Menu
Home Explore People Places Arts History Plants & Animals Science Life & Culture Technology
On this page
Azure Data Explorer
Cloud-based data analytics service

Azure Data Explorer is a fully-managed cloud analytics platform developed by Microsoft that ingests structured, semi-structured (like JSON), and unstructured data. It enables fast ad hoc queries with low latency, supports full-text indexing, time series analysis, and regular expression evaluation for data parsing. Offered as a platform as a service within the Microsoft Azure ecosystem, it was announced in 2018 to address big data and data exploration needs efficiently.

We don't have any images related to Azure Data Explorer yet.
We don't have any YouTube videos related to Azure Data Explorer yet.
We don't have any PDF documents related to Azure Data Explorer yet.
We don't have any Books related to Azure Data Explorer yet.
We don't have any archived web articles related to Azure Data Explorer yet.

History

The development of the product began in 2014 as a grassroots incubation project in the Israeli R&D center of Microsoft,14 with the internal code name 'Kusto'1516 (named after Jacques Cousteau, as a reference to "exploring the ocean of data"). The project aim was to address Azure services' needs for fast and scalable log and telemetry analytics.

In 2016 it became the backend big-data and analytics service for Application Insights Analytics.17

The product was announced as a Public Preview product at the Microsoft Ignite 2018 conference,18 and was announced as a generally available at the Microsoft Ignite conference of February 2019.19

In March 2021, "Kusto EngineV3", Azure Data Explorer's next generation storage and query engine, became generally available. It was designed to provide unparalleled performance for ingesting and querying telemetry, logs, and time series data.20

Features

Azure Data Explorer offers an optimized query language and visualizing options21 of its data with a SQL-like language called KQL (Kusto Query Language222324).2526

Azure Data Explorer can ingest 200 MB per second per node.27 Data Ingestion methods are pipelines and connectors to common services like Azure Event Grid or Azure Event Hub,28 or programmatic ingestion using SDKs.

Data visualization can be achieved using their native dashboard offering, or with tools like Power BI2930 or Grafana.3132

Design

Azure Data Explorer is a distributed database running on a cluster of compute nodes in Microsoft Azure. It is based on relational database management systems (RDBMS), supporting entities such as databases, tables, functions, and columns. It supports complex analytics query operators, such as calculated columns, searching and filtering on rows, group by-aggregates and joins.33

The engine service exposes a relational data model: At the top level (cluster) there is a collection of databases, each database contains a collection of tables and stored functions. Each table defines a schema (ordered list of typed fields).

In Azure Data Explorer, unlike a typical relational database management systems (RDBMS), there are no constraints like key uniqueness, primary and foreign key.34 The necessary relationships are established at the query time.35 The data in Azure Data Explorer generally follows this pattern:36 Creating Database, Ingesting data, Query the database.

References

  1. Serra, James (2019-03-14). "Azure Data Explorer". James Serra's Blog. Retrieved 2020-04-09. https://www.jamesserra.com/archive/2019/03/azure-data-explorer/ ↩

  2. "Microsoft Rolls Out Azure Data Lake Storage, Azure Data Explorer". eWEEK. Retrieved 2020-04-10. https://www.eweek.com/cloud/microsoft-rolls-out-azure-data-lake-storage-azure-data-explorer ↩

  3. Mogenis, Max. "What is Azure Data Explorer?". blog.pragmaticworks.com. Retrieved 2020-03-21. https://blog.pragmaticworks.com/what-is-azure-data-explorer ↩

  4. "Creating An Azure Data Explorer Cluster And Database In Azure". www.c-sharpcorner.com. Retrieved 2020-04-09. https://www.c-sharpcorner.com/article/creating-an-azure-data-explorer-cluster-and-database-in-azure/ ↩

  5. Mackie, Kurt (2019-09-20). "Updated Tools for Office 365 and Microsoft Azure Arriving Soon -- Redmondmag.com". Redmondmag. Retrieved 2020-04-10. https://redmondmag.com/articles/2019/09/20/updated-tools-office-365-microsoft-azure.aspx ↩

  6. "Microsoft Azure Data Explorer vs. Microsoft Azure SQL Database Comparison". db-engines.com. Retrieved 2020-04-09. https://db-engines.com/en/system/Microsoft+Azure+Data+Explorer%3BMicrosoft+Azure+SQL+Database ↩

  7. Mahajan, Gauri (2020-02-27). "Azure Data Explorer for beginners". SQL Shack - articles about database auditing, server performance, data recovery, and more. Retrieved 2020-04-09. https://www.sqlshack.com/azure-data-explorer-for-beginners/ ↩

  8. "What is Azure Data Explorer". docs.microsoft.com. Retrieved 2019-12-06. https://docs.microsoft.com/en-us/azure/data-explorer/data-explorer-overview ↩

  9. Mahajan, Gauri (2020-02-27). "Azure Data Explorer for beginners". SQL Shack - articles about database auditing, server performance, data recovery, and more. Retrieved 2020-03-21. https://www.sqlshack.com/azure-data-explorer-for-beginners/ ↩

  10. Lai, Alex (2019-01-14). "What is Azure Data Explorer and Kusto Querying Language (KQL)?". Adatis. Retrieved 2020-04-10. https://adatis.co.uk/what-is-azure-data-explorer-and-kusto-querying-language-kql/ ↩

  11. "Microsoft Azure Data Explorer vs. Microsoft Azure SQL Database Comparison". db-engines.com. Retrieved 2020-04-09. https://db-engines.com/en/system/Microsoft+Azure+Data+Explorer%3BMicrosoft+Azure+SQL+Database ↩

  12. Mahajan, Gauri (2020-02-27). "Azure Data Explorer for beginners". SQL Shack - articles about database auditing, server performance, data recovery, and more. Retrieved 2020-04-09. https://www.sqlshack.com/azure-data-explorer-for-beginners/ ↩

  13. "Azure Data Explorer - Digital Marketplace". www.digitalmarketplace.service.gov.uk. Retrieved 2020-03-21. https://www.digitalmarketplace.service.gov.uk/g-cloud/services/517576167550270 ↩

  14. "Microsoft R&D". www.microsoftrnd.co.il. Retrieved 2019-12-06. https://www.microsoftrnd.co.il/whatwedo#BigData ↩

  15. Mahajan, Gauri (2020-02-27). "Azure Data Explorer for beginners". SQL Shack - articles about database auditing, server performance, data recovery, and more. Retrieved 2020-03-21. https://www.sqlshack.com/azure-data-explorer-for-beginners/ ↩

  16. Mahajan, Gauri (2020-02-27). "Azure Data Explorer for beginners". SQL Shack - articles about database auditing, server performance, data recovery, and more. Retrieved 2020-04-09. https://www.sqlshack.com/azure-data-explorer-for-beginners/ ↩

  17. orspod. "Introducing Application Insights Analytics". devblogs.microsoft.com. Retrieved 2019-12-06. https://devblogs.microsoft.com/bharry/introducing-application-analytics/ ↩

  18. "Introducing Azure Data Explorer". azure.microsoft.com. Retrieved 2019-12-06. https://azure.microsoft.com/en-us/blog/introducing-azure-data-explorer/ ↩

  19. "General Availability: Azure Data Explorer | Azure updates | Microsoft Azure". azure.microsoft.com. Retrieved 2019-12-06. https://azure.microsoft.com/en-us/updates/general-availability-azure-data-explorer/ ↩

  20. orspod. "Azure Data Explorer Kusto EngineV3". docs.microsoft.com. Retrieved 2021-04-13. https://docs.microsoft.com/en-us/azure/data-explorer/engine-v3 ↩

  21. Brust, Andrew. "Fastly, Microsoft partner on real-time analytics with Azure Data Explorer". ZDNet. Retrieved 2020-04-09. https://www.zdnet.com/article/fastly-microsoft-partner-on-real-time-analytics-with-azure-data-explorer/ ↩

  22. Mackie, Kurt (2019-07-18). "Microsoft Previews Prometheus Data in Azure Monitor for Containers -- Redmondmag.com". Redmondmag. Retrieved 2020-04-09. https://redmondmag.com/articles/2019/07/18/prometheus-preview-in-azure-monitor-for-containers.aspx ↩

  23. "Getting Started with the Kusto Query Language (KQL) – System.Blog.Martens.Ben". blogs.msdn.microsoft.com. Retrieved 2019-12-06. https://blogs.msdn.microsoft.com/ben/2018/10/11/getting-started-with-the-kusto-query-language/ ↩

  24. "Exploring Data in Microsoft Azure Using Kusto Query Language and Azure Data Explorer". www.pluralsight.com. Retrieved 2020-03-21. https://www.pluralsight.com/courses/microsoft-azure-data-exploring ↩

  25. Mahajan, Gauri (2020-02-27). "Azure Data Explorer for beginners". SQL Shack - articles about database auditing, server performance, data recovery, and more. Retrieved 2020-04-09. https://www.sqlshack.com/azure-data-explorer-for-beginners/ ↩

  26. "What is Azure Data Explorer". docs.microsoft.com. Retrieved 2019-12-06. https://docs.microsoft.com/en-us/azure/data-explorer/data-explorer-overview ↩

  27. "Introducing Azure Data Explorer". azure.microsoft.com. Retrieved 2019-12-06. https://azure.microsoft.com/en-us/blog/introducing-azure-data-explorer/ ↩

  28. "Architecting Data and Analytics Pipelines in Azure". Gartner. Retrieved 2020-04-10. https://www.gartner.com/en/documents/3939788/architecting-data-and-analytics-pipelines-in-azure ↩

  29. "Architecting Data and Analytics Pipelines in Azure". Gartner. Retrieved 2020-04-10. https://www.gartner.com/en/documents/3939788/architecting-data-and-analytics-pipelines-in-azure ↩

  30. Mahajan, Gauri (2020-02-27). "Azure Data Explorer for beginners". SQL Shack - articles about database auditing, server performance, data recovery, and more. Retrieved 2020-04-10. https://www.sqlshack.com/azure-data-explorer-for-beginners/ ↩

  31. "Grafana vs Azure Dashboards - Which One to Use & When?". CloudIQ Tech. 2018-12-10. Retrieved 2020-04-10. https://cloudiqtech.com/grafana-vs-azure-dashboards/ ↩

  32. "Azure Data Explorer Datasource plugin for Grafana". Grafana Labs. Retrieved 2020-04-10. https://grafana.com/grafana/plugins/grafana-azure-data-explorer-datasource ↩

  33. orspod. "Getting started with Kusto - Azure Data Explorer". docs.microsoft.com. Retrieved 2019-12-06. https://docs.microsoft.com/en-us/azure/kusto/concepts/ ↩

  34. "GraphDB vs. Microsoft Azure Data Explorer Comparison". db-engines.com. Retrieved 2020-04-10. https://db-engines.com/en/system/GraphDB%3BMicrosoft+Azure+Data+Explorer ↩

  35. "Azure Data Explorer: a big data analytics cloud platform" (PDF). Archived (PDF) from the original on 2019-12-06. https://azure.microsoft.com/mediahandler/files/resourcefiles/azure-data-explorer/Azure_Data_Explorer_white_paper.pdf ↩

  36. Serra, James (2019-03-14). "Azure Data Explorer". James Serra's Blog. Retrieved 2020-03-21. https://www.jamesserra.com/archive/2019/03/azure-data-explorer/ ↩