You may want to forward Azure resource logs to a different tenant from time to time. Fortunately, using the Diagnostic settings option in Azure to forward -at least some of the- logs to another tenant is quite simple. I needed to test out some of the interesting scenarios because I couldn’t find a detailed description of how the Diagnostic settings work. I’ll share my findings with you here.
I had a specific scenario in mind to put to the test. Cross-tenant log forwarding is simple to set up, but how do I disable it in a specific scenario?
What should I do if I lose access to the source tenant? The Diagnostic settings config is created in this tenant. Without access to the source tenant, how can I stop the log forwarding? This situation can easily happen if the source tenant is managed by a client of yours or by a team that gets outsourced.
Initial setup
I used the setup described in this section to test the aforementioned scenario.
The following capabilities are required when configuring a resource to send ‘Diagnostic logs’ to a Log Analytics Workspace:
- Configure Diagnostic settings on the source resource. The permission that allows this is ‘Microsoft.Insights/DiagnosticSettings/*’. Diagnostic settings can be configured, for instance, by a Contributor for a specific resource or a Monitoring Contributor for any resource.
- Have access to the destination Log Analytics workspace and its key. To see the workspace you need the ‘Microsoft.OperationalInsights/workspaces/read’ permission. The workspace key can be obtained with this permission: ‘Microsoft.OperationalInsights/workspaces/sharedKeys/action’ and used to configure the Diagnostic settings (this happens automatically during Diagnostic settings configuration).
I usually use the Owner role in the source Tenant and Log Analytics contributor in the destination Tenant. The Owner of the subscriptions in tenant A can be useful in case I have to configure policies for multiple subscriptions. The Log Analytics contributor has the ‘Microsoft.OperationalInsights/workspaces/read’ and ‘../sharedKeys/action’ permissions but it can be an overkill in some environments.
So, to configure cross-tenant diagnostic settings your setup will look like this:
-
Tenant A: The source tenant in which you want to configure the resource to forward logs to the Log Analytics Workspace.
-
Tenant B: The destination tenant in which the Log Analytics Workspace is located.
-
A user and a group in Tenant A. You can assign the permissions with Azure Lighthouse to the group, and you can add the user to the given group. Also the group or the user itself needs permission to be able to configure Diagnostic settings. All the required permissoins/roles are going to be assigned to this user/group and nothing else is needed.
-
The Lighthouse configuration that grants the ‘Microsoft.OperationalInsights/workspaces/sharedKeys/action’ and ‘Microsoft.OperationalInsights/workspaces/read’ permissions over the Log Analytics workspace in Tenant B to the group in Tenant A.
Here is a Lighthouse parameter file that provides Log Analytics contributor (92aaf0da-9dab-42b6-94a3-d43ce8d16293) role to every memeber of group A in tenant A. You can use this parameter file together with Microsoft’s Lighthouse ARM template which needs to be deployed in tenant B.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"mspOfferName": {
"value": "DelegateAccount cross test"
},
"mspOfferDescription": {
"value": "Diagnostic settings test setup"
},
"managedByTenantId": {
"value": "[Tenat A ID]"
},
"authorizations": {
"value": [
{
"principalId": "[Group A ID]",
"roleDefinitionId": "92aaf0da-9dab-42b6-94a3-d43ce8d16293",
"principalIdDisplayName": "DelegateGroup_LAContributor"
}
]
}
}
}Testing
The scenario is to try to stop the log forwarding from tenant A to tenant B while not having any access to tenant A.
1. Remove the Lighthouse projection
The first thing I tried was removing the Lighthouse assignment in Azure by going to the Service Providers tab in tenant B. Using the GUI option, I removed this access.
I theorized that if no one has access to tenant B, log forwarding might no longer work. However, this was not the case. Even after the Lighthouse access was removed, the logs were still relayed to the Sentinel in tenant A without any disruption.
2. Changing the Log Analytics workspace keys
You require the “Microsoft.OperationalInsights/workspaces/sharedKeys/” permission over the target Log Analytics workspace when configuring logging through diagnostic settings. This suggested to me that the workspace key was being used, which meant that the forwarding might continue to function with the keys even after the Lighthouse setup was removed. However, I hoped recreating the key(s) will solve the problem.
Thus, I went ahead and changed both keys (primary and secondary). Unfortunately, the logs continued to be forwarded even after this. This behavior suggests that the keys are not being utilized directly.
According to Microsoft
Because I was unable to stop cross-tenant logging from tenant B (the destination tenant), I contacted Microsoft. They shared the following information:
-
In short, it is not possible to stop log forwarding from the Destination tenant. To halt the flow of events, you have to gain access to the source tenant and remove the Diagnostic settings from there.
-
Because the keys are not directly used in log forwarding, changing them does not stop it. The keys are used to generate a certificate, which is then used to configure the Diagnostic settings. This means that log forwarding will continue until either the Diagnostic settings configuration is removed or the certificate expires. Changing the key does not have any effect on this.
-
Furthermore, we are unable to revoke this certificate from tenant B, so we are once again out of luck.
-
The certificate is valid for one year after the initial Diagnostic settings configuration.
-
And their advice on how to deal with a situation in which you no longer have access to a client’s environment: “In addition - I reviewed your case with a few other colleagues and we all agree that the best course of action in such a scenario would be for you to reach out to said potential client (to whose tenant you no longer have access) and ask them to stop the logs collection (to remove the Diagnostic setting in question).”
Potential implications
It is not always necessary to forward logs between tenants. However, if you do, you should always keep the aforementioned constraints in mind when planning your offboarding process. If you lose access to the source tenant for any reason, you will not be able to easily stop the log forwarding.
-
Client tenants: If you forward your client’s log to your own tenant and then leave that client, you don’t want to see those logs in your tenant anymore. Unwanted events can have financial consequences, but they can also be a privacy or legal concern.
-
Multi-tenant company: Multiple tenants can also be used within your company. All departments or company functions, can have their own tenants. In this case, outsourcing a function or simply eliminating one of the departments can result in the same situation. And then having these logs might be expensive for you and difficult to get rid of.
-
Attack: While this is an unlikely scenario, the inability to disable log forwarding from the destination tenant can be abused against that tenant as well. Once the connection is established, someone can simply initiate a massive amount of log generation, which will be free or nearly free in the source tenant but fairly expensive in the destination tenant. It can also trigger some rules, overloading a SOC with meaningless alerts. However, the Diagnostic settings must first be configured with the appropriate permissions, so this activity is unlikely to be a goal of any attacker (perhaps a disgruntled client or employee?).
Conclusion
To enable cross-tenant Diagnostic settings, you must first create a resource projection from tenant A (source tenant) using Azure Lighthouse. Because the registration is configured in tenant A, so that tenat can also remove this registration any time. By removing the registration it also removes your options (in the destination tenant) to stop the log forwarding entirely.
As a result, you should be aware of this risk, identify and define the party (or parties) responsible for removing the Diagnostic settings, and ensure that your offboarding process addresses the cross-tenant logging issue discussed in this article.