Microsoft Sentinel SIEM and data lake calculator

Currently, historical data that is archived for long-term storage is not moved or mirrored from Sentinel to data lake. Your existing archived data will remain in Sentinel and will not be accessible in data lake.

However, since the billing meter changes to the new data lake-based approach, you will benefit from the 6:1 Storage compression discount regardless. Consequently, the cost of your long-term retained data will decrease to one-sixth of its previous price.

According to Microsoft's latest guidance, the data processing fee applies to uncompressed data that enters the ingestion pipelines. This fee does not apply to data mirrored from your analytics tier—mirrored data incurs no ingestion or processing charges.

If you use a Data Collection Rule (DCR) to filter data, you can reduce ingestion costs, but data processing costs remain unaffected.

Example:
If you send 100 GB of data to your DCR and drop 90% of it, you will pay ingestion costs on only 10% of the data, but the processing fee will still apply to the full 100%.

Microsoft addresses this scenario in their guidance documents. The data lake feature only mirrors new data moving forward.

For example, if you enable data lake and after 30 days reduce interactive (analytics) retention from 180 to 90 days, then all the queryable data you will have is 90 days in analytics (interactive) and 30 days (overlapping with the first 30 days of analytics) in data lake. This means you will lose the ability to run interactive queries on your last 90 days of data.

Be sure to carefully plan your retention policy and query requirements before making changes.

This is a Microsoft Sentinel platform requirement to ensure data continuity and consistency. Since analytics tier data can transition to data lake tier after the analytics retention period (long-term retention), the data lake must be configured to store data for at least as long as analytics.

This prevents gaps in your security data and maintains a continuous timeline for compliance and forensic investigations.

Think of it as a data flow: analytics (hot) → data lake (cold/archived), where the cold tier must be able to receive and retain data from the hot tier.

The data not kept in data lake tier longer than its analytics retention won't generate data lake storage costs.

All newly ingested analytics data is mirrored to the data lake storage at no additional charge. Therefore, during the interactive retention period, data is accessible both in the free-to-query analytics tier and in the pay-to-query data lake storage.

However, if you specifically query your data lake - for example, from the KQL queries page or via KQL jobs - you will incur query costs, even if the same data exists in analytics. If you access the data through Sentinel within the analytics retention window, no additional fee is applied. At present, Microsoft does not waive query fees for data lake storage data even if the same data is available for free in analytics; your costs will depend on which tool or interface you use to access the data.

Query charges are incurred when you directly query the data lake table using KQL jobs, KQL queries, or via the API (for example, through the Sentinel MCP server). All standard workloads that access the data lake in this manner will be subject to these charges.

The advanced data insights charge applies when you utilize managed workboos Notebooks. Whether you run Notebooks interactively or as scheduled jobs, charges are based on the CPU hours used by the given execution. When using Notebooks for querying, you will not incur regular query charges. In summary, Notebook-based exploration of the data lake is exempt from query charges but will generate advanced data insights charges for the compute resources consumed.

The most reliable method for minimizing query charges in the Sentinel data lake tier is to leverage the TimeGenerated field. Explicitly define time windows for your queries to limit data scanning to the intervals required. Be careful when running queries via the Sentinel MCP server - it tends to query ALL your existing data -, or when you run a KQL query in the GUI - easy to misconfigure, and due to a bug you can easily query more data than expected.

With Notebooks, billing is based on CPU hours consumed. To minimize costs, focus on developing efficient, fast-executing code. Applying filters on the TimeGenerated field speeds up queries due to partitioning. Additionally, selecting only the necessary columns (field pruning) reduces data processed, improving query performance and lowering compute time.

To simplify pricing, Microsoft applies a fixed 6x discount on data lake storage charges, effectively assuming that your stored data achieves an average compression ratio of 6:1. This discount applies exclusively to data lake storage fees and does not impact other costs such as query charges, which are calculated based on uncompressed data.

In reality, your data's actual compression ratio may be better or worse than 6:1. If the compression is more effective, you may pay slightly more than the true compressed cost; if less effective, you may pay less. Microsoft uses the fixed 6x factor as a standard assumption across all data.
Overall, this 6x discount represents a new cost-saving benefit that was not available previously.

Note that the Azure Cost Management page displays raw data storage in GB but does not factor in the 6x compression, so reported storage size will appear larger than what the discounted pricing reflects.

Only advanced data insights charges apply to this activity. Although you are querying data in the data lake, no separate data query charges are incurred. Instead, data query costs are incorporated into the advanced data insights billing.

This comparison focuses solely on cost and does not consider feature differences.

- In North Europe, analytics ingestion costs $5.16 USD per GB (Pay-as-you-Go pricing).
- Ingesting data into the data lake costs $0.06 USD per GB (Ingestion) plus $0.12 USD per GB (data processing), totaling $0.18 USD per GB.
- Querying data lake data costs $0.006 USD per GB.

Therefore, ingesting data into the data lake is $4.98 USD per GB cheaper than analytics ingestion.
From a cost perspective, you can run 4.98 / 0.006 ≈ 830 queries on the same amount of data in the data lake before analytics becomes the more economical option (excluding data lake storage costs).

If your team runs queries that look back 90 days, with one query per day, the same data will be queried about 90 times. This means you can conduct roughly 830 / 90 ≈ 9 queries per day. After that it will be more expensive than pushing the data into analytics tier.

If your team only looks back 7 days (a more realistic scenario), you could run roughly 830 / 7 ≈ 120 queries per day. While that may seem high, interactive query execution often involves many trial runs or incorrect queries, generating additional charges. It's important to carefully monitor and manage query patterns in your environment.