Documenti di Didattica
Documenti di Professioni
Documenti di Cultura
Azure Monitor maximizes the availability and performance of your applications and services by delivering a
comprehensive solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises
environments. It helps you understand how your applications are performing and proactively identifies issues
affecting them and the resources they depend on.
Just a few examples of what you can do with Azure Monitor include:
Detect and diagnose issues across applications and dependencies with Application Insights.
Correlate infrastructure issues with Azure Monitor for VMs and Azure Monitor for Containers.
Drill into your monitoring data with Log Analytics for troubleshooting and deep diagnostics.
Support operations at scale with smart alerts and automated actions.
Create visualizations with Azure dashboards and workbooks.
Overview
The following diagram gives a high-level view of Azure Monitor. At the center of the diagram are the data stores
for metrics and logs, which are the two fundamental types of data use by Azure Monitor. On the left are the
sources of monitoring data that populate these data stores. On the right are the different functions that Azure
Monitor performs with this collected data such as analysis, alerting, and streaming to external systems.
Log data collected by Azure Monitor can be analyzed with queries to quickly retrieve, consolidate, and analyze
collected data. You can create and test queries using Log Analytics in the Azure portal and then either directly
analyze the data using these tools or save queries for use with visualizations or alert rules.
Azure Monitor uses a version of the Kusto query language used by Azure Data Explorer that is suitable for
simple log queries but also includes advanced functionality such as aggregations, joins, and smart analytics. You
can quickly learn the query language using multiple lessons. Particular guidance is provided to users who are
already familiar with SQL and Splunk.
Insights
Monitoring data is only useful if it can increase your visibility into the operation of your computing environment.
Azure Monitor includes several features and tools that provide valuable insights into your applications and other
resources that they depend on. Monitoring solutions and features such as Application Insights and Azure
Monitor for containers provide deep insights into different aspects of your application and specific Azure
services.
Application Insights
Application Insights monitors the availability, performance, and usage of your web applications whether they're
hosted in the cloud or on-premises. It leverages the powerful data analysis platform in Azure Monitor to provide
you with deep insights into your application's operations and diagnose errors without waiting for a user to
report them. Application Insights includes connection points to a variety of development tools and integrates
with Visual Studio to support your DevOps processes.
Monitoring solutions
Monitoring solutions in Azure Monitor are packaged sets of logic that provide insights for a particular
application or service. They include logic for collecting monitoring data for the application or service, queries to
analyze that data, and views for visualization. Monitoring solutions are available from Microsoft and partners to
provide monitoring for various Azure services and other applications.
Autoscale
Autoscale allows you to have the right amount of resources running to handle the load on your application. It
allows you to create rules that use metrics collected by Azure Monitor to determine when to automatically add
resources to handle increases in load and also save money by removing resources that are sitting idle. You
specify a minimum and maximum number of instances and the logic for when to increase or decrease resources.
Views
Views visually present log data in Azure Monitor. Each view includes a single tile that drills down to a
combination of visualizations such as bar and line charts in addition to lists summarizing critical data.
Monitoring solutions include views that summarize data for a particular application, and you can create your
own views to present data from any log query. Like other elements in Azure Monitor, views can be added to
Azure dashboards.
Power BI
Power BI is a business analytics service that provides interactive visualizations across a variety of data sources
and is an effective means of making data available to others within and outside your organization. You can
configure Power BI to automatically import log data from Azure Monitor to take advantage of these additional
visualizations.
Next steps
Learn more about:
Metrics and logs for the data collected by Azure Monitor.
Data sources for how the different components of your application send telemetry.
Log queries for analyzing collected data.
Best practices for monitoring cloud applications and services.
Azure Monitor naming and terminology changes
12/6/2019 • 2 minutes to read • Edit Online
Significant changes have been made to Azure Monitor recently, with different services being consolidated in order
to simplify monitoring for Azure customers. This article describes recent name and terminology changes in Azure
Monitor documentation.
Next steps
Read an overview of Azure Monitor that describes its different components and features.
Learn about the transition of the OMS portal.
Azure Monitor Frequently Asked Questions
1/24/2020 • 39 minutes to read • Edit Online
This Microsoft FAQ is a list of commonly asked questions about Azure Monitor.
General
What is Azure Monitor?
Azure Monitor is a service in Azure that provides performance and availability monitoring for applications and
services in Azure, other cloud environments, or on-premises. Azure Monitor collects data from multiple sources
into a common data platform where it can be analyzed for trends and anomalies. Rich features in Azure Monitor
assist you in quickly identifying and responding to critical situations that may affect your application.
What's the difference between Azure Monitor, Log Analytics, and Application Insights?
In September 2018, Microsoft combined Azure Monitor, Log Analytics, and Application Insights into a single
service to provide powerful end-to-end monitoring of your applications and the components they rely on. Features
in Log Analytics and Application Insights have not changed, although some features have been rebranded to Azure
Monitor in order to better reflect their new scope. The log data engine and query language of Log Analytics is now
referred to as Azure Monitor Logs. See Azure Monitor terminology updates.
What does Azure Monitor cost?
Features of Azure Monitor that are automatically enabled such as collection of metrics and activity logs are
provided at no cost. There is a cost associated with other features such as log queries and alerting. See the Azure
Monitor pricing page for detailed pricing information.
How do I enable Azure Monitor?
Azure Monitor is enabled the moment that you create a new Azure subscription, and Activity log and platform
metrics are automatically collected. Create diagnostic settings to collect more detailed information about the
operation of your Azure resources, and add monitoring solutions and insights to provide additional analysis on
collected data for particular services.
How do I access Azure Monitor?
Access all Azure Monitor features and data from the Monitor menu in the Azure portal. The Monitoring section
of the menu for different Azure services provides access to the same tools with data filtered to a particular
resource. Azure Monitor data is also accessible for a variety of scenarios using CLI, PowerShell, and a REST API.
Is there an on-premises version of Azure Monitor?
No. Azure Monitor is a scalable cloud service that processes and stores large amounts of data, although Azure
Monitor can monitor resources that are on-premises and in other clouds.
Can Azure Monitor monitor on-premises resources?
Yes, in addition to collecting monitoring data from Azure resources, Azure Monitor can collect data from virtual
machines and applications in other clouds and on-premises. See Sources of monitoring data for Azure Monitor.
Does Azure Monitor integrate with System Center Operations Manager?
You can connect your existing System Center Operations Manager management group to Azure Monitor to collect
data from agents into Azure Monitor Logs. This allows you to use log queries and solution to analyze data collected
from agents. You can also configure existing System Center Operations Manager agents to send data directly to
Azure Monitor. See Connect Operations Manager to Azure Monitor.
What IP addresses does Azure Monitor use?
See IP addresses used by Application Insights and Log Analytics for a listing of the IP addresses and ports required
for agents and other external resources to access Azure Monitor.
Monitoring data
Where does Azure Monitor get its data?
Azure Monitor collects data from a variety of sources including logs and metrics from Azure platform and
resources, custom applications, and agents running on virtual machines. Other services such as Azure Security
Center and Network Watcher collect data into a Log Analytics workspace so it can be analyzed with Azure Monitor
data. You can also send custom data to Azure Monitor using the REST API for logs or metrics. See Sources of
monitoring data for Azure Monitor.
What data is collected by Azure Monitor?
Azure Monitor collects data from a variety of sources into logs or metrics. Each type of data has its own relative
advantages, and each supports a particular set of features in Azure Monitor. There is a single metrics database for
each Azure subscription, while you can create multiple Log Analytics workspaces to collect logs depending on your
requirements. See Azure Monitor data platform.
Is there a maximum amount of data that I can collect in Azure Monitor?
There is no limit to the amount of metric data you can collect, but this data is stored for a maximum of 93 days. See
Retention of Metrics. There is no limit on the amount of log data that you can collected, but it may be affected by
the pricing tier you choose for the Log Analytics workspace. See pricing details.
How do I access data collected by Azure Monitor?
Insights and solutions provide a custom experience for working with data stored in Azure Monitor. You can work
directly with log data using a log query written in Kusto Query Language (KQL ). In the Azure portal, you can write
and run queries and interactively analyze data using Log Analytics. Analyze metrics in the Azure portal with the
Metrics Explorer. See Analyze log data in Azure Monitor and Getting started with Azure Metrics Explorer.
Logs
What's the difference between Azure Monitor Logs and Azure Data Explorer?
Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. Azure Monitor
Logs is built on top of Azure Data Explorer and uses the same Kusto Query Language (KQL ) with some minor
differences. See Azure Monitor log query language differences.
How do I retrieve log data?
All data is retrieved from a Log Analytics workspace using a log query written using Kusto Query Language (KQL ).
You can write your own queries or use solutions and insights that include log queries for a particular application or
service. See Overview of log queries in Azure Monitor.
What is a Log Analytics workspace?
All log data collected by Azure Monitor is stored in a Log Analytics workspace. A workspace is essentially a
container where log data is collected from a variety of sources. You may have a single Log Analytics workspace for
all your monitoring data or may have requirements for multiple workspaces. See Designing your Azure Monitor
Logs deployment.
Can you move an existing Log Analytics workspace to another Azure subscription?
You can move a workspace between resource groups or subscriptions but not to a different region. See Move a Log
Analytics workspace to different subscription or resource group.
Why can't I see Query Explorer and Save buttons in Log Analytics?
Query Explorer, Save and New alert rule buttons are not available when the query scope is set to a specific
resource. To create alerts, save or load a query, Log Analytics must be scoped to a workspace. To open Log
Analytics in workspace context, select Logs from the Azure Monitor menu. The last used workspace is selected,
but you can select any other workspace. See Log query scope and time range in Azure Monitor Log Analytics
Why am I getting the error: "Register resource provider 'Microsoft.Insights' for this subscription to enable this
query" when opening Log Analytics from a VM?
Many resource providers are automatically registered, but you may need to manually register some resource
providers. The scope for registration is always the subscription. See Resource providers and types for more
information.
Why am I am getting no access error message when opening Log Analytics from a VM?
To view VM Logs, you need to be granted with read permission to the workspaces that stores the VM logs. In these
cases, your administrator must grant you with to permissions in Azure.
Alerts
What is an alert in Azure Monitor?
Alerts proactively notify you when important conditions are found in your monitoring data. They allow you to
identify and address issues before the users of your system notice them. There are multiple kinds of alerts:
Metric - Metric value exceeds a threshold.
Log query - Results of a log query match defined criteria.
Activity log - Activity log event matches defined criteria.
Web test - Results of availability test match defined criteria.
See Overview of alerts in Microsoft Azure.
What is an action group?
An action group is a collection of notifications and actions that can be triggered by an alert. Multiple alerts can use
a single action group allowing you to leverage common sets of notifications and actions. See Create and manage
action groups in the Azure portal.
What is an action rule?
An action rule allows you to modify the behavior of a set of alerts that match a certain criteria. This allows you to to
perform such requirements as disable alert actions during a maintenance window. You can also apply an action
group to a set of alerts rather than applying them directly to the alert rules. See Action rules.
Agents
Does Azure Monitor require an agent?
An agent is only required to collect data from the operating system and workloads in virtual machines. The virtual
machines can be located in Azure, another cloud environment, or on-premises. See Overview of the Azure Monitor
agents.
What's the difference between the Azure Monitor agents?
Azure Diagnostic extension is for Azure virtual machines and collects data to Azure Monitor Metrics, Azure
Storage, and Azure Event Hubs. The Log Analytics agent is for virtual machines in Azure, another cloud
environment, or on-premises and collects data to Azure Monitor Logs. The Dependency agent requires the Log
Analytics agent and collected process details and dependencies. See Overview of the Azure Monitor agents.
Does my agent traffic use my ExpressRoute connection?
Traffic to Azure Monitor uses the Microsoft peering ExpressRoute circuit. See ExpressRoute documentation for a
description of the different types of ExpressRoute traffic.
How can I confirm that the Log Analytics agent is able to communicate with Azure Monitor?
From Control Panel on the agent computer, select Security & Settings, Microsoft Monitoring Agent . Under
the Azure Log Analytics (OMS ) tab, a green check mark icon confirms that the agent is able to communicate with
Azure Monitor. A yellow warning icon means the agent is having issues. One common reason is the Microsoft
Monitoring Agent service has stopped. Use service control manager to restart the service.
How do I stop the Log Analytics agent from communicating with Azure Monitor?
For agents connected to Log Analytics directly, open the Control Panel and select Security & Settings, Microsoft
Monitoring Agent. Under the Azure Log Analytics (OMS ) tab, remove all workspaces listed. In System Center
Operations Manager, remove the computer from the Log Analytics managed computers list. Operations Manager
updates the configuration of the agent to no longer report to Log Analytics.
How much data is sent per agent?
The amount of data sent per agent depends on:
The solutions you have enabled
The number of logs and performance counters being collected
The volume of data in the logs
See Manage usage and costs with Azure Monitor Logs for details.
For computers that are able to run the WireData agent, use the following query to see how much data is being sent:
WireData
| where ProcessName == "C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe"
| where Direction == "Outbound"
| summarize sum(TotalBytes) by Computer
How much network bandwidth is used by the Microsoft Management Agent (MMA ) when sending data to Azure
Monitor?
Bandwidth is a function on the amount of data sent. Data is compressed as it is sent over the network.
How can I be notified when data collection from the Log Analytics agent stops?
Use the steps described in create a new log alert to be notified when data collection stops. Use the following
settings for the alert rule:
Define alert condition: Specify your Log Analytics workspace as the resource target.
Alert criteria
Signal Name: Custom log search
Search query:
Heartbeat | summarize LastCall = max(TimeGenerated) by Computer | where LastCall < ago(15m)
Alert logic: Based on number of results, Condition Greater than, Threshold value 0
Evaluated based on: Period (in minutes) 30, Frequency (in minutes) 10
Define alert details
Name: Data collection stopped
Severity: Warning
Specify an existing or new Action Group so that when the log alert matches criteria, you are notified if you have a
heartbeat missing for more than 15 minutes.
What are the firewall requirements for Azure Monitor agents?
See Network firewall requirementsfor details on firewall requirements.
Visualizations
Why can't I can’t see View Designer?
View Designer is only available for users assigned with Contributor permissions or higher in the Log Analytics
workspace.
Application Insights
Configuration problems
I'm having trouble setting up my:
.NET app
Monitoring an already-running app
Azure diagnostics
Java web app
I get no data from my server
Set firewall exceptions
Set up an ASP.NET server
Set up a Java server
Can I use Application Insights with ...?
Web apps on an IIS server in Azure VM or Azure virtual machine scale set
Web apps on an IIS server - on-premises or in a VM
Java web apps
Node.js apps
Web apps on Azure
Cloud Services on Azure
App servers running in Docker
Single-page web apps
SharePoint
Windows desktop app
Other platforms
Is it free?
Yes, for experimental use. In the basic pricing plan, your application can send a certain allowance of data each
month free of charge. The free allowance is large enough to cover development, and publishing an app for a small
number of users. You can set a cap to prevent more than a specified amount of data from being processed.
Larger volumes of telemetry are charged by the Gb. We provide some tips on how to limit your charges.
The Enterprise plan incurs a charge for each day that each web server node sends telemetry. It is suitable if you
want to use Continuous Export on a large scale.
Read the pricing plan.
How much does it cost?
Open the Usage and estimated costs page in an Application Insights resource. There's a chart of recent
usage. You can set a data volume cap, if you want.
Open the Azure Billing blade to see your bills across all resources.
What does Application Insights modify in my project?
The details depend on the type of project. For a web application:
Adds these files to your project:
ApplicationInsights.config
ai.js
Installs these NuGet packages:
Application Insights API - the core API
Application Insights API for Web Applications - used to send telemetry from the server
Application Insights API for JavaScript Applications - used to send telemetry from the client
The packages include these assemblies:
Microsoft.ApplicationInsights
Microsoft.ApplicationInsights.Platform
Inserts items into:
Web.config
packages.config
(New projects only - if you add Application Insights to an existing project, you have to do this manually.) Inserts
snippets into the client and server code to initialize them with the Application Insights resource ID. For example,
in an MVC app, code is inserted into the master page Views/Shared/_Layout.cshtml
How do I upgrade from older SDK versions?
See the release notes for the SDK appropriate to your type of application.
How can I change which Azure resource my project sends data to?
In Solution Explorer, right-click ApplicationInsights.config and choose Update Application Insights. You can
send the data to an existing or new resource in Azure. The update wizard changes the instrumentation key in
ApplicationInsights.config, which determines where the server SDK sends your data. Unless you deselect "Update
all," it will also change the key where it appears in your web pages.
What is Status Monitor?
A desktop app that you can use in your IIS web server to help configure Application Insights in web apps. It doesn't
collect telemetry: you can stop it when you are not configuring an app.
Learn more.
What telemetry is collected by Application Insights?
From server web apps:
HTTP requests
Dependencies. Calls to: SQL Databases; HTTP calls to external services; Azure Cosmos DB, table, blob storage,
and queue.
Exceptions and stack traces.
Performance Counters - If you use Status Monitor, Azure monitoring for App Services, Azure monitoring for
VM or virtual machine scale set, or the Application Insights collectd writer.
Custom events and metrics that you code.
Trace logs if you configure the appropriate collector.
From client web pages:
Page view counts
AJAX calls Requests made from a running script.
Page view load data
User and session counts
Authenticated user IDs
From other sources, if you configure them:
Azure diagnostics
Import to Analytics
Log Analytics
Logstash
Can I filter out or modify some telemetry?
Yes, in the server you can write:
Telemetry Processor to filter or add properties to selected telemetry items before they are sent from your app.
Telemetry Initializer to add properties to all items of telemetry.
Learn more for ASP.NET or Java.
How are city, country/region, and other geo location data calculated?
We look up the IP address (IPv4 or IPv6) of the web client using GeoLite2.
Browser telemetry: We collect the sender's IP address.
Server telemetry: The Application Insights module collects the client IP address. It is not collected if
X-Forwarded-For is set.
To learn more about how IP address and geolocation data is collected in Application Insights refer to this article.
You can configure the ClientIpHeaderTelemetryInitializer to take the IP address from a different header. In some
systems, for example, it is moved by a proxy, load balancer, or CDN to X-Originating-IP . Learn more.
You can use Power BI to display your request telemetry on a map.
How long is data retained in the portal? Is it secure?
Take a look at Data Retention and Privacy.
Could personal data be sent in the telemetry?
This is possible if your code sends such data. It can also happen if variables in stack traces include personal data.
Your development team should conduct risk assessments to ensure that personal data is properly handled. Learn
more about data retention and privacy.
All octets of the client web address are always set to 0 after the geo location attributes are looked up.
My Instrumentation Key is visible in my web page source.
This is common practice in monitoring solutions.
It can't be used to steal your data.
It could be used to skew your data or trigger alerts.
We have not heard that any customer has had such problems.
You could:
Use two separate Instrumentation Keys (separate Application Insights resources), for client and server data. Or
Write a proxy that runs in your server, and have the web client send data through that proxy.
How do I see POST data in Diagnostic search?
We don't log POST data automatically, but you can use a TrackTrace call: put the data in the message parameter.
This has a longer size limit than the limits on string properties, though you can't filter on it.
Should I use single or multiple Application Insights resources?
Use a single resource for all the components or roles in a single business system. Use separate resources for
development, test, and release versions, and for independent applications.
See the discussion here
Example - cloud service with worker and web roles
How do I dynamically change the instrumentation key?
Discussion here
Example - cloud service with worker and web roles
What are the User and Session counts?
The JavaScript SDK sets a user cookie on the web client, to identify returning users, and a session cookie to
group activities.
If there is no client-side script, you can set cookies at the server.
If one real user uses your site in different browsers, or using in-private/incognito browsing, or different
machines, then they will be counted more than once.
To identify a logged-in user across machines and browsers, add a call to setAuthenticatedUserContext().
Have I enabled everything in Application Insights?
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Server app perf: response times, ... Add Application Insights to your project Detect perf issues
or Install AI Status Monitor on server
(or write your own code to track
dependencies)
Dependency telemetry Install AI Status Monitor on server Diagnose issues with databases or other
external components
Get stack traces from exceptions Insert TrackException calls in your code Detect and diagnose exceptions
(but some are reported automatically)
Search log traces Add a logging adapter Diagnose exceptions, perf issues
Client usage basics: page views, JavaScript initializer in web pages Usage analytics
sessions, ...
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Client custom metrics Tracking calls in web pages Enhance user experience
Automation
Configuring Application Insights
You can write PowerShell scripts using Azure Resource Monitor to:
Create and update Application Insights resources.
Set the pricing plan.
Get the instrumentation key.
Add a metric alert.
Add an availability test.
You can't set up a Metric Explorer report or set up continuous export.
Querying the telemetry
Use the REST API to run Analytics queries.
How can I set an alert on an event?
Azure alerts are only on metrics. Create a custom metric that crosses a value threshold whenever your event
occurs. Then set an alert on the metric. You'll get a notification whenever the metric crosses the threshold in either
direction; you won't get a notification until the first crossing, no matter whether the initial value is high or low; there
is always a latency of a few minutes.
Are there data transfer charges between an Azure web app and Application Insights?
If your Azure web app is hosted in a data center where there is an Application Insights collection endpoint, there
is no charge.
If there is no collection endpoint in your host data center, then your app's telemetry will incur Azure outgoing
charges.
This doesn't depend on where your Application Insights resource is hosted. It just depends on the distribution of
our endpoints.
Can I send telemetry to the Application Insights portal?
We recommend you use our SDKs and use the SDK API. There are variants of the SDK for various platforms.
These SDKs handle buffering, compression, throttling, retries, and so on. However, the ingestion schema and
endpoint protocol are public.
Can I monitor an intranet web server?
Yes, but you will need to allow traffic to our services by either firewall exceptions or proxy redirects.
QuickPulse https://rt.services.visualstudio.com:443
ApplicationIdProvider https://dc.services.visualstudio.com:443
TelemetryChannel https://dc.services.visualstudio.com:443
<ApplicationInsights>
...
<TelemetryModules>
<Add
Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule,
Microsoft.AI.PerfCounterCollector">
<QuickPulseServiceEndpoint>https://rt.services.visualstudio.com/QuickPulseService.svc</QuickPulseServiceEndpoin
t>
</Add>
</TelemetryModules>
...
<TelemetryChannel>
<EndpointAddress>https://dc.services.visualstudio.com/v2/track</EndpointAddress>
</TelemetryChannel>
...
<ApplicationIdProvider
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplicationId
Provider, Microsoft.ApplicationInsights">
<ProfileQueryEndpoint>https://dc.services.visualstudio.com/api/profiles/{0}/appId</ProfileQueryEndpoint>
</ApplicationIdProvider>
...
</ApplicationInsights>
NOTE
ApplicationIdProvider is available starting in v2.6.0.
Proxy passthrough
Proxy passthrough can be achieved by configuring either a machine level or application level proxy. For more
information see dotnet's article on DefaultProxy.
Example Web.config:
<system.net>
<defaultProxy>
<proxy proxyaddress="http://xx.xx.xx.xx:yyyy" bypassonlocal="true"/>
</defaultProxy>
</system.net>
Option 2
Re-enable collection for these properties for every container log line.
If the first option is not convenient due to query changes involved, you can re-enable collecting these fields by
enabling the setting log_collection_settings.enrich_container_logs in the agent config map as described in the
data collection configuration settings.
NOTE
The second option is not recommended with large clusters that have more than 50 nodes because it generates API server
calls from every node in the cluster to perform this enrichment. This option also increases data size for every log line collected.
console.log(json.stringify({
"Hello": "This example has multiple lines:",
"Docker/Moby": "will not break this into multiple lines",
"and you will receive":"all of them in log analytics",
"as one": "log entry"
}));
This data will look like the following example in Azure Monitor for logs when you query for it:
LogEntry : ({“Hello": "This example has multiple lines:","Docker/Moby": "will not break this into multiple
lines", "and you will receive":"all of them in log analytics", "as one": "log entry"}
For a detailed look at the issue, review the following GitHub link.
How do I resolve Azure AD errors when I enable live logs?
You may see the following error: The reply url specified in the request does not match the reply urls
configured for the application: '<application ID>'. The solution to solve it can be found in the article How to
view container data in real time with Azure Monitor for containers.
Why can't I upgrade cluster after onboarding?
If after you enable Azure Monitor for containers for an AKS cluster, you delete the Log Analytics workspace the
cluster was sending its data to, when attempting to upgrade the cluster it will fail. To work around this, you will have
to disable monitoring and then re-enable it referencing a different valid workspace in your subscription. When you
try to perform the cluster upgrade again, it should process and complete successfully.
Which ports and domains do I need to open/whitelist for the agent?
See the Network firewall requirements for the proxy and firewall configuration information required for the
containerized agent with Azure, Azure US Government, and Azure China 21Vianet clouds.
NOTE
We configure performance counters for the workspace that affects all VMs that report to the workspace, whether or not you
have chosen to onboard them to Azure Monitor for VMs. For more details on how performance counters are configured for
the workspace, please refer to our documentation. For information about the counters configured for Azure Monitor for VMs,
please refer to our enable Azure Monitor for VMs article.
Next steps
If your question isn't answered here, you can refer to the following forums to additional questions and answers.
Log Analytics
Application Insights
For general feedback on Azure Monitor please visit the feedback forum.
Quickstart: Monitor an Azure resource with Azure
Monitor
1/10/2020 • 2 minutes to read • Edit Online
Azure Monitor starts collecting data from Azure resources the moment that they're created. This quickstart
provides a brief walkthrough of the data that's automatically collected for a resource and how to view it in the
Azure portal for a particular resource. Later, you can add configuration to collect additional data and can go to the
Azure Monitor menu to use the same tools to access data collected for all the resources in your subscription.
For more detailed descriptions of monitoring data collected from Azure resources see Monitoring Azure resources
with Azure Monitor.
Overview page
Many services will include monitoring data on their Overview page as a quick glance to their operation. This will
typically be based on a subset of platform metrics stored in Azure Monitor Metrics.
1. Locate an Azure resource in your subscription.
2. Go to the Overview page and note if there's any performance data displayed. This data will be provided by
Azure Monitor. The example below is the Overview page for an Azure storage account, and you can see
that there are multiple metrics displayed.
3. You can click on any of the graphs to open the data in metrics explorer which is described below.
3. If you want to see events from other resources in your subscription, either change criteria in the filter or
even remove filter properties.
View metrics
Metrics are numerical values that describe some aspect of your resource at a particular time. Azure Monitor
automatically collects platform metrics at one minute intervals from all Azure resources. You can view these
metrics using metrics explorer.
1. Under the Monitoring section of your resource's menu, select Metrics. This opens metrics explorer with
the scope set to your resource.
2. Click Add metric to add a metric to the chart.
3. Select a Metric from the dropdown list and then an Aggregation. This defines how the collected values will
be sampled over each time interval.
4. Click Add metric to add additional metric and aggregation combinations to the chart.
Next steps
In this quickstart, you viewed the Activity log and metrics for an Azure resource which are automatically collected
by Azure Monitor. Resource logs provide insight into the detailed operation of the resource but must be configured
in order to be collected. Continue to the tutorial for collecting resource logs into a Log Analytics workspace where
they can be analyzed using log queries.
Collect and analyze resource logs with Azure Monitor
Collect data from an Azure virtual machine with
Azure Monitor
12/13/2019 • 5 minutes to read • Edit Online
Azure Monitor can collect data directly from your Azure virtual machines into a Log Analytics workspace for
detailed analysis and correlation. Installing the Log Analytics VM extension for Windows and Linux allows Azure
Monitor to collect data from your Azure VMs. This quickstart shows you how to configure and collect data from
your Azure Linux or Windows VMs using the VM extension with a few easy steps.
This quickstart assumes you have an existing Azure virtual machine. If not you can create a Windows VM or
create a Linux VM following our VM quickstarts.
Create a workspace
1. In the Azure portal, select All services. In the list of resources, type Log Analytics. As you begin typing,
the list filters based on your input. Select Log Analytics workspaces.
2. Select Create, and then select choices for the following items:
Provide a name for the new Log Analytics workspace, such as DefaultLAWorkspace.
Select a Subscription to link to by selecting from the drop-down list if the default selected is not
appropriate.
For Resource Group, select an existing resource group that contains one or more Azure virtual
machines.
Select the Location your VMs are deployed to. For additional information, see which regions Log
Analytics is available in.
If you are creating a workspace in a new subscription created after April 2, 2018, it will
automatically use the Per GB pricing plan and the option to select a pricing tier will not be available.
If you are creating a workspace for an existing subscription created before April 2, or to subscription
that was tied to an existing EA enrollment, select your preferred pricing tier. For additional
information about the particular tiers, see Log Analytics Pricing Details.
3. After providing the required information on the Log Analytics workspace pane, select OK.
While the information is verified and the workspace is created, you can track its progress under Notifications
from the menu.
For Windows and Linux virtual machines already deployed in Azure, you install the Log Analytics agent with the
Log Analytics VM Extension. Using the extension simplifies the installation process and automatically configures
the agent to send data to the Log Analytics workspace that you specify. The agent is also upgraded automatically
when a newer version is released, ensuring that you have the latest features and fixes. Before proceeding, verify
the VM is running otherwise the process will fail to complete successfully.
NOTE
The Log Analytics agent for Linux cannot be configured to report to more than one Log Analytics workspace.
1. In the Azure portal, select All services found in the upper left-hand corner. In the list of resources, type
Log Analytics. As you begin typing, the list filters based on your input. Select Log Analytics
workspaces.
2. In your list of Log Analytics workspaces, select DefaultLAWorkspace created earlier.
3. On the left-hand menu, under Workspace Data Sources, select Virtual machines.
4. In the list of Virtual machines, select a virtual machine you want to install the agent on. Notice that the
Log Analytics connection status for the VM indicates that it is Not connected.
5. In the details for your virtual machine, select Connect. The agent is automatically installed and configured
for your Log Analytics workspace. This process takes a few minutes, during which time the Status shows
Connecting.
6. After you install and connect the agent, the Log Analytics connection status will be updated with This
workspace.
For example, the query in the following image returned 10,000 performance records. Your results will be
significantly less.
Clean up resources
When no longer needed, delete the Log Analytics workspace. To do so, select the Log Analytics workspace you
created earlier and on the resource page select Delete.
Next steps
Now that you are collecting operational and performance data from your Windows or Linux virtual machines,
you can easily begin exploring, analyzing, and taking action on data that you collect for free.
To learn how to view and analyze the data, continue to the tutorial.
View or analyze data in Log Analytics
Quickstart: Collect data from a Linux computer in a
hybrid environment with Azure Monitor
12/27/2019 • 6 minutes to read • Edit Online
Azure Monitor can collect data directly from your physical or virtual Linux computers in your environment into a
Log Analytics workspace for detailed analysis and correlation. Installing the Log Analytics agent allows Azure
Monitor to collect data from a datacenter or other cloud environment. This quickstart shows you how to
configure and collect data from your Linux server with a few easy steps. For information about Azure Linux VMs,
see Collect data about Azure virtual machines.
To understand the supported configuration, see Supported Windows operating systems and Network firewall
configuration.
If you don't have an Azure subscription, create a free account before you begin.
Create a workspace
1. In the Azure portal, select All services. In the list of resources, type Log Analytics. As you begin typing,
the list filters based on your input. Select Log Analytics workspaces.
2. Select Create, and then select choices for the following items:
Provide a name for the new Log Analytics workspace, such as DefaultLAWorkspace.
Select a Subscription to link to by selecting from the drop-down list if the default selected is not
appropriate.
For Resource Group, select an existing resource group that contains one or more Azure virtual
machines.
Select the Location your VMs are deployed to. For additional information, see which regions Log
Analytics is available in.
If you are creating a workspace in a new subscription created after April 2, 2018, it will
automatically use the Per GB pricing plan and the option to select a pricing tier will not be available.
If you are creating a workspace for an existing subscription created before April 2, or to subscription
that was tied to an existing EA enrollment, select your preferred pricing tier. For additional
information about the particular tiers, see Log Analytics Pricing Details.
3. After providing the required information on the Log Analytics workspace pane, select OK.
While the information is verified and the workspace is created, you can track its progress under Notifications
from the menu.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log
Analytics agent for Linux.
1. In the upper-left corner of the Azure portal, select All services. In the search box, enter Log Analytics. As
you type, the list filters based on your input. Select Log Analytics workspaces.
2. In your list of Log Analytics workspaces, select the workspace you created earlier. (You might have named
it DefaultLAWorkspace.)
3. Select Advanced settings:
4. Select Connected Sources, and then select Linux Servers.
5. The value to the right of Workspace ID and Primary Key. Copy and paste both into your favorite editor.
NOTE
The Log Analytics agent for Linux cannot be configured to report to more than one Log Analytics workspace.
If your Linux computer needs to communicate through a proxy server to Log Analytics, the proxy configuration
can be specified on the command line by including -p [protocol://][user:password@]proxyhost[:port] . The
proxyhost property accepts a fully qualified domain name or IP address of the proxy server.
For example: https://user01:password@proxy01.contoso.com:30443
1. To configure the Linux computer to connect to a Log Analytics workspace, run the following command
providing the workspace ID and primary key copied earlier. The following command downloads the agent,
validates its checksum, and installs it.
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -w <YOUR WORKSPACE ID> -s <YOUR
WORKSPACE PRIMARY KEY>
The following command includes the -p proxy parameter and example syntax when authentication is
required by your proxy server:
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -p [protocol://]
[user:password@]proxyhost[:port] -w <YOUR WORKSPACE ID> -s <YOUR WORKSPACE PRIMARY KEY>
2. To configure the Linux computer to connect to Log Analytics workspace in Azure Government cloud, run
the following command providing the workspace ID and primary key copied earlier. The following
command downloads the agent, validates its checksum, and installs it.
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -w <YOUR WORKSPACE ID> -s <YOUR
WORKSPACE PRIMARY KEY> -d opinsights.azure.us
The following command includes the -p proxy parameter and example syntax when authentication is
required by your proxy server:
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -p [protocol://]
[user:password@]proxyhost[:port] -w <YOUR WORKSPACE ID> -s <YOUR WORKSPACE PRIMARY KEY> -d
opinsights.azure.us
For example, the query in the following image returned 10,000 Performance records. Your results will be
significantly less.
Clean up resources
When no longer needed, you can remove the agent from the Linux computer and delete the Log Analytics
workspace.
To remove the agent, run the following command on the Linux computer. The --purge argument completely
removes the agent and its configuration.
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-Linux/master/installer/scripts/onboard_agent.sh
&& sh onboard_agent.sh --purge
To delete the workspace, select the Log Analytics workspace you created earlier and on the resource page select
Delete.
Next steps
Now that you are collecting operational and performance data from your on-premises Linux computer, you can
easily begin exploring, analyzing, and taking action on data that you collect for free.
To learn how to view and analyze the data, continue to the tutorial.
View or analyze data in Log Analytics
Collect data from a Windows computer in a hybrid
environment with Azure Monitor
12/13/2019 • 5 minutes to read • Edit Online
Azure Monitor can collect data directly from your your physical or virtual Windows computers in your
environment into a Log Analytics workspace for detailed analysis and correlation. Installing the Log Analytics
agent allows Azure Monitor to collect data from a datacenter or other cloud environment. This quickstart shows
you how to configure and collect data from your Windows computer with a few easy steps. For information about
Azure Windows VMs, see Collect data about Azure virtual machines.
To understand the supported configuration, see Supported Windows operating systems and Network firewall
configuration.
If you don't have an Azure subscription, create a free account before you begin.
Create a workspace
1. In the Azure portal, select All services. In the list of resources, type Log Analytics. As you begin typing,
the list filters based on your input. Select Log Analytics workspaces.
2. Select Create, and then select choices for the following items:
Provide a name for the new Log Analytics workspace, such as DefaultLAWorkspace.
Select a Subscription to link to by selecting from the drop-down list if the default selected is not
appropriate.
For Resource Group, select an existing resource group that contains one or more Azure virtual
machines.
Select the Location your VMs are deployed to. For additional information, see which regions Log
Analytics is available in.
If you are creating a workspace in a new subscription created after April 2, 2018, it will automatically
use the Per GB pricing plan and the option to select a pricing tier will not be available. If you are
creating a workspace for an existing subscription created before April 2, or to subscription that was
tied to an existing EA enrollment, select your preferred pricing tier. For additional information about
the particular tiers, see Log Analytics Pricing Details.
3. After providing the required information on the Log Analytics workspace pane, select OK.
While the information is verified and the workspace is created, you can track its progress under Notifications
from the menu.
For example, the query in this image returned 10,000 Performance records. Your results will be significantly
less.
Clean up resources
You can remove the agent from your computer and delete the Log Analytics workspace if you no longer need
them.
To remove the agent, complete these steps:
1. Open Control Panel.
2. Open Programs and Features.
3. In Programs and Features, select Microsoft Monitoring Agent and then select Uninstall.
To delete the Log Analytics workspace you created earlier, select it, and, on the resource page, select Delete:
Next steps
Now that you're collecting operational and performance data from your Windows computer, you can easily begin
exploring, analyzing, and acting on the data you collect, for free.
To learn how to view and analyze the data, continue to the tutorial:
View or analyze data in Log Analytics
Start monitoring your ASP.NET Web Application
10/24/2019 • 4 minutes to read • Edit Online
With Azure Application Insights, you can easily monitor your web application for availability, performance, and
usage. You can also quickly identify and diagnose errors in your application without waiting for a user to report
them. With the information that you collect from Application Insights about the performance and effectiveness of
your app, you can make informed choices to maintain and improve your application.
This quickstart shows how to add Application Insights to an existing ASP.NET web application and start analyzing
live statistics, which is just one of the various methods you can use to analyze your application. If you do not have
an ASP.NET web application, you can create one following the Create an ASP.NET Web App quickstart.
Prerequisites
To complete this quickstart:
Install Visual Studio 2019 with the following workloads:
ASP.NET and web development
Azure development
If you don't have an Azure subscription, create a free account before you begin.
IMPORTANT
The process to add Application Insights varies by ASP.NET template type. If you are using the Empty or Azure
Mobile App template select Project > Add Application Insights Telemetry. For all other ASP.NET templates
consult the instructions in the step above.
3. Click Get Started (earlier versions of Visual Studio have a Start Free button instead).
4. Select your subscription and click Register.
5. Select Project > Manage NuGet Packages > Package source: nuget.org > Update the Application
Insights SDK packages to the latest stable release.
6. Run your application by either selecting Start Debugging from the Debug menu or by pressing the F5
key.
3. Click on the App Analytics icon View in Logs (Analytics) on one of the application components. This
opens Logs (Analytics), which provides a rich query language for analyzing all data collected by
Application Insights. In this case, a query is generated for you that renders the request count as a chart. You
can write your own queries to analyze other data.
4. Click on Live Metrics Stream on the left under investigate. This shows live statistics about your application
as it's running. This includes such information as the number of incoming requests, the duration of those
requests, and any failures that occur. You can also inspect critical performance metrics such as processor and
memory.
If you are ready to host your application in Azure, you can publish it now. Follow the steps described in
Create an ASP.NET Web App Quickstart.
5. If you use Visual Studio to add Application Insights monitoring, you can automatically add client-side
monitoring. To add client-side monitoring manually to an application add the following JavaScript to your
application:
<!--
To collect user behavior analytics about your application,
insert the following script into each page you want to track.
Place this code immediately before the closing </head> tag,
and before any other scripts. Your first data will appear
automatically in just a few seconds.
-->
<script type="text/javascript">
var appInsights=window.appInsights||function(a){
function b(a){c[a]=function(){var b=arguments;c.queue.push(function(){c[a].apply(c,b)})}}var c=
{config:a},d=document,e=window;setTimeout(function(){var
b=d.createElement("script");b.src=a.url||"https://az416426.vo.msecnd.net/scripts/a/ai.0.js",d.getElementsByTagN
ame("script")[0].parentNode.appendChild(b)});try{c.cookie=d.cookie}catch(a){}c.queue=[];for(var f=
["Event","Exception","Metric","PageView","Trace","Dependency"];f.length;)b("track"+f.pop());if(b("setAuthentica
tedUserContext"),b("clearAuthenticatedUserContext"),b("startTrackEvent"),b("stopTrackEvent"),b("startTrackPage"
),b("stopTrackPage"),b("flush"),!a.disableExceptionTracking){f="onerror",b("_"+f);var
g=e[f];e[f]=function(a,b,d,e,h){var i=g&&g(a,b,d,e,h);return!0!==i&&c["_"+f](a,b,d,e,h),i}}return c
}({
instrumentationKey:"<your instrumentation key>"
});
window.appInsights=appInsights,appInsights.queue&&0===appInsights.queue.length&&appInsights.trackPageView();
</script>
To learn more, visit the GitHub repository for our open-source JavaScript SDK.
Video
External step-by-step video about configuring Application Insights with a .NET application from scratch.
Clean up resources
When you are done testing, you can delete the resource group and all related resources. To do so follow the steps
below.
1. From the left-hand menu in the Azure portal, click Resource groups and then click myResourceGroup.
2. On your resource group page, click Delete, type myResourceGroup in the text box, and then click Delete.
Next steps
In this quickstart, you’ve enabled your application for monitoring by Azure Application Insights. Continue to the
tutorials to learn how to use it to monitor statistics and detect issues in your application.
Azure Application Insights tutorials
Start Monitoring Your ASP.NET Core Web Application
12/23/2019 • 3 minutes to read • Edit Online
With Azure Application Insights, you can easily monitor your web application for availability, performance, and
usage. You can also quickly identify and diagnose errors in your application without waiting for a user to report
them.
This quickstart guides you through adding the Application Insights SDK to an existing ASP.NET Core web
application. To learn about configuring Application Insights without Visual Studio checkout this article.
Prerequisites
To complete this quickstart:
Install Visual Studio 2019 with the following workloads:
ASP.NET and web development
Azure development
Install .NET Core 2.0 SDK
You will need an Azure subscription and an existing .NET Core web application.
If you don't have an ASP.NET Core web application, you can use our step-by-step guide to create an ASP.NET
Core app and add Application Insights.
If you don't have an Azure subscription, create a free account before you begin.
NOTE
If this is your first time creating an Application Insights resource you can learn more by visiting the Create an
Application Insights Resource doc.
A configuration box appears; use the following table to fill out the input fields.
Name Globally Unique Value Name that identifies the app you are
monitoring
SETTINGS VALUE DESCRIPTION
2. Click Create.
3. Click on the App Analytics icon View in Analytics. This opens Application Insights Analytics,
which provides a rich query language for analyzing all data collected by Application Insights. In this case, a
query is generated for you that renders the request count as a chart. You can write your own queries to
analyze other data.
4. Return to the Overview page and examine the KPI Dashboards. This dashboard provides statistics about
your application health, including the number of incoming requests, the duration of those requests, and any
failures that occur.
5. On the left click on Metrics. Use the metrics explorer to investigate the health and utilization of your
resource. You can click Add new chart to create additional custom views or select Edit to modify the
existing chart types, height, color palette, groupings, and metrics. For example, you can make a chart that
displays the average browser page load time by picking "Browser page load time" from the metrics drop
down and "Avg" from aggregation. To learn more about Azure Metrics Explorer visit Getting started with
Azure Metrics Explorer.
Video
External step-by-step video about configuring Application Insights with .NET Core and Visual Studio from
scratch.
External step-by-step video about configuring Application Insights with .NET Core and Visual Studio Code from
scratch.
Clean up resources
When you are done testing, you can delete the resource group and all related resources. To do so follow the steps
below.
NOTE
If you used an existing resource group the instructions below will not work and you will need to just delete the individual
Application Insights resource. Keep in mind anytime you delete a resource group all underyling resources that are members
of that group will be deleted.
1. From the left-hand menu in the Azure portal, click Resource groups and then click myResourceGroup.
2. On your resource group page, click Delete, type myResourceGroup in the text box, and then click Delete.
Next steps
Find and diagnose run-time exceptions
Quickstart: Start monitoring your Node.js Web
application with Azure Application Insights
12/23/2019 • 4 minutes to read • Edit Online
This quickstart guides you through adding the version 0.22 Application Insights SDK for Node.js to an existing
Node.js web application.
With Azure Application Insights, you can easily monitor your web application for availability, performance, and
usage. You can also quickly identify and diagnose errors in your application without waiting for a user to report
them. With the version 0.20 SDK release onward, you can monitor common third-party packages, including
MongoDB, MySQL, and Redis.
Prerequisites
To complete this quickstart:
You need an Azure Subscription and an existing Node.js web application.
If you don't have a Node.js web application, you can create one by following the Create a Node.js web app
quickstart.
If you don't have an Azure subscription, create a free account before you begin.
A configuration page appears; use the following table to fill out the input fields.
Name Globally Unique Value Name that identifies the app you are
monitoring
2. Select Create.
2. Add the Application Insights SDK for Node.js to your application. From your app's root folder run:
3. Edit your app's first .js file and add the two lines below to the topmost part of your script. If you are using
the Node.js quickstart app, you would modify the index.js file. Replace <instrumentation_key> with your
application's instrumentation key.
NOTE
It takes 3-5 minutes before data begins appearing in the portal. If this app is a low-traffic test app, keep in mind that most
metrics are only captured when there are active requests or operations occurring.
3. Select the App Analytics icon View in Analytics. This opens Application Insights Analytics, which
provides a rich query language for analyzing all data collected by Application Insights. In this case, a query
is generated for you that renders the request count as a chart. You can write your own queries to analyze
other data.
4. Return to the Overview page and examine the KPI graphs. This dashboard provides statistics about your
application health, including the number of incoming requests, the duration of those requests, and any
failures that occur.
To enable the Page View Load Time chart to populate with client-side telemetry data, add this script to
each page that you want to track:
<!--
To collect user behavior analytics tools about your application,
insert the following script into each page you want to track.
Place this code immediately before the closing </head> tag,
and before any other scripts. Your first data will appear
automatically in just a few seconds.
-->
<script type="text/javascript">
var appInsights=window.appInsights||function(config){
function i(config){t[config]=function(){var i=arguments;t.queue.push(function()
{t[config].apply(t,i)})}}var t=
{config:config},u=document,e=window,o="script",s="AuthenticatedUserContext",h="start",c="stop",l="Track"
,a=l+"Event",v=l+"Page",y=u.createElement(o),r,f;y.src=config.url||"https://az416426.vo.msecnd.net/scrip
ts/a/ai.0.js";u.getElementsByTagName(o)[0].parentNode.appendChild(y);try{t.cookie=u.cookie}catch(p)
{}for(t.queue=[],t.version="1.0",r=
["Event","Exception","Metric","PageView","Trace","Dependency"];r.length;)i("track"+r.pop());return
i("set"+s),i("clear"+s),i(h+a),i(c+a),i(h+v),i(c+v),i("flush"),config.disableExceptionTracking||
(r="onerror",i("_"+r),f=e[r],e[r]=function(config,i,u,e,o){var s=f&&f(config,i,u,e,o);return
s!==!0&&t["_"+r](config,i,u,e,o),s}),t
}({
instrumentationKey:"<insert instrumentation key>"
});
window.appInsights=appInsights;
appInsights.trackPageView();
</script>
5. On the left, select Metrics. Use the metrics explorer to investigate the health and utilization of your
resource. You can select Add new chart to create additional custom views or select Edit to modify the
existing chart types, height, color palette, groupings, and metrics. For example, you can make a chart that
displays the average browser page load time by selecting "Browser page load time" from the metrics drop
down and "Avg" from aggregation. To learn more about Azure Metrics Explorer visit Getting started with
Azure Metrics Explorer.
To learn more about monitoring Node.js, check out the additional App Insights Node.js documentation.
Clean up resources
When you are done testing, you can delete the resource group and all related resources. To do so follow the steps
below.
NOTE
If you used an existing resource group the instructions below will not work and you will need to just delete the individual
Application Insights resource. Keep in mind anytime you delete a resource group all underyling resources that are members
of that group will be deleted.
1. From the left-hand menu in the Azure portal, select Resource groups and then select myResourceGroup.
2. On your resource group page, select Delete, enter myResourceGroup in the text box, and then select Delete.
Next steps
Find and diagnose performance problems
Start analyzing your mobile app with App Center
and Application Insights
12/23/2019 • 6 minutes to read • Edit Online
This quickstart guides you through connecting your app's App Center instance to Application Insights. With
Application Insights, you can query, segment, filter, and analyze your telemetry with more powerful tools than are
available from the Analytics service of App Center.
Prerequisites
To complete this quickstart, you need:
An Azure subscription.
An iOS, Android, Xamarin, Universal Windows, or React Native app.
If you don't have an Azure subscription, create a free account before you begin.
MSAnalytics.trackEvent("Video clicked")
To send custom events from Android apps, use the trackEvent method in the App Center SDK. Learn more about
tracking events from Android apps.
Analytics.trackEvent("Video clicked")
To send custom events from other app platforms, use the trackEvent methods in their App Center SDKs.
To make sure your custom events are being received, go to the Events tab under the Analytics section in App
Center. It can take a couple minutes for events to show up from when they're sent from your app.
NOTE
If this is your first time creating an Application Insights resource you can learn more by visiting the Create an
Application Insights Resource doc.
A configuration box will appear. Use the table below to fill out the input fields.
Name Some globally unique value, like Name that identifies the app you are
"myApp-iOS" monitoring
Resource Group A new resource group, or an existing The resource group in which to
one from the menu create the new Application Insights
resource
Location A location from the menu Choose a location near you, or near
where your app is hosted
3. Click Create.
If your app supports multiple platforms (iOS, Android, etc.), it's best to create separate Application Insights
resources, one for each platform.
customEvents
| where timestamp >= ago(24h)
| summarize dcount(user_Id) by name
| order by dcount_user_Id desc
a. Select the query by clicking anywhere on the query in the text editor.
b. Then click Go to run the query.
Learn more about Application Insights Analytics and the Log Analytics query language.
2. Segment and filter your custom event telemetry. From the Application Insights Overview page,
choose Users in the table of contents.
The Users tool shows how many users of your app clicked certain buttons, visited certain screens, or
performed any other action that you are tracking as an event with the App Center SDK. If you've been
looking for a way to segment and filter your App Center events, the Users tool is a great choice.
For example, segment your usage by geography by choosing Country or region in the Split by
dropdown menu.
3. Analyze conversion, retention, and navigation patterns in your app. From the Application Insights
Overview page, choose User Flows in the table of contents.
The User Flows tool visualizes which events users send after some starting event. It's useful for getting an
overall picture of how users navigate through your app. It can also reveal places where many users are
churning from your app, or repeating the same actions over and over.
In addition to User Flows, Application Insights has several other user behavior analytics tools to answer
specific questions:
Funnels for analyzing and monitoring conversion rates.
Retention for analyzing how well your app retains users over time.
Workbooks for combining visualizations and text into a shareable report.
Cohorts for naming and saving specific groups of users or events so they can be easily referenced from
other analytics tools.
Clean up resources
If you do not want to continue using Application Insights with App Center, turn off export in App Center and
delete the Application Insights resource. This will prevent you from being charged further by Application Insights
for this resource.
To turn off export in App Center:
1. In App Center, go to Settings and choose Export.
2. Click the Application Insights export you want to delete, then click Delete export at the bottom and confirm.
To delete the Application Insights resource:
1. In the left-hand menu of the Azure portal, click Resource groups and then choose the resource group in
which your Application Insights resource was created.
2. Open the Application Insights resource you want to delete. Then click Delete in the top menu of the resource
and confirm. This will permanently delete the copy of the data that was exported to Application Insights.
Next steps
Understand how customers are using your app
Start monitoring your website
1/8/2020 • 4 minutes to read • Edit Online
With Azure Monitor Application Insights, you can easily monitor your website for availability, performance, and
usage. You can also quickly identify and diagnose errors in your application without waiting for a user to report
them. Application Insights provides both server-side monitoring as well as client/browser-side monitoring
capabilities.
This quickstart guides you through adding the open source Application Insights JavaScript SDK which allows you
to understand the client/browser-side experience for visitors to your website.
Prerequisites
To complete this quickstart:
You need an Azure Subscription.
If you don't have an Azure subscription, create a free account before you begin.
NOTE
If this is your first time creating an Application Insights resource you can learn more by visiting the Create an
Application Insights Resource article.
A configuration box appears; use the following table to fill out the input fields.
Name Globally Unique Value Name that identifies the app you are
monitoring
2. Click Create.
Create an HTML file
1. On your local computer, create a file called hello_world.html . For this example the file will be placed on the
root of the C: drive at C:\hello_world.html .
2. Copy the script below into hello_world.html :
<!DOCTYPE html>
<html>
<head>
<title>Azure Monitor Application Insights</title>
</head>
<body>
<h1>Azure Monitor Application Insights Hello World!</h1>
<p>You can use the Application Insights JavaScript SDK to perform client/browser-side monitoring of your
website. To learn about more advanced JavaScript SDK configurations visit the <a
href="https://github.com/Microsoft/ApplicationInsights-JS/blob/master/API-reference.md" title="API
Reference">API reference</a>.</p>
</body>
</html>
2. Add the following script to your hello_world.html before the closing </head> tag:
<script type="text/javascript">
var sdkInstance="appInsightsSDK";window[sdkInstance]="appInsights";var
aiName=window[sdkInstance],aisdk=window[aiName]||function(e){function n(e){t[e]=function(){var
n=arguments;t.queue.push(function(){t[e].apply(t,n)})}}var t={config:e};t.initialize=!0;var
i=document,a=window;setTimeout(function(){var
n=i.createElement("script");n.src=e.url||"https://az416426.vo.msecnd.net/scripts/b/ai.2.min.js",i.getEle
mentsByTagName("script")[0].parentNode.appendChild(n)});try{t.cookie=i.cookie}catch(e){}t.queue=
[],t.version=2;for(var r=
["Event","PageView","Exception","Trace","DependencyData","Metric","PageViewPerformance"];r.length;)n("tr
ack"+r.pop());n("startTrackPage"),n("stopTrackPage");var
s="Track"+r[0];if(n("start"+s),n("stop"+s),n("setAuthenticatedUserContext"),n("clearAuthenticatedUserCon
text"),n("flush"),!
(!0===e.disableExceptionTracking||e.extensionConfig&&e.extensionConfig.ApplicationInsightsAnalytics&&!0=
==e.extensionConfig.ApplicationInsightsAnalytics.disableExceptionTracking)){n("_"+(r="onerror"));var
o=a[r];a[r]=function(e,n,i,a,s){var c=o&&o(e,n,i,a,s);return!0!==c&&t["_"+r]
({message:e,url:n,lineNumber:i,columnNumber:a,error:s}),c},e.autoExceptionInstrumented=!0}return t}(
{
instrumentationKey:"INSTRUMENTATION_KEY"
}
);window[aiName]=aisdk,aisdk.queue&&0===aisdk.queue.length&&aisdk.trackPageView({});
</script>
3. Go back to the Overview page. Click on Browser from under the Investigate header, then select
Performance Here you find metrics related to the performance of your website. There is also a
corresponding view for analyzing failures and exceptions in your website. You can click Samples to drill into
individual transaction details. From here, you can access the end-to-end transaction details experience.
4. To begin exploring the user behavior analytics tools, from the main Application Insights menu select Users
under the Usage header. Since we are testing from a single machine, we will only see data for one user. For
a live website, the distribution of users might look as follows:
5. If we had instrumented a more complex website with multiple pages, another useful tool is User Flows.
With User Flows you can track the pathway visitors takes through the various parts of your website.
To learn more advanced configurations for monitoring websites, check out the JavaScript SDK API reference.
Clean up resources
If you plan to continue on to work with subsequent quickstarts or with the tutorials, do not clean up the resources
created in this quickstart. Otherwise, if you do not plan to continue, use the following steps to delete all resources
created by this quickstart in the Azure portal.
NOTE
If you used an existing resource group the instructions below will not work and you will need to just delete the individual
Application Insights resource. Keep in mind anytime you delete a resource group all underyling resources that are members
of that group will be deleted.
1. From the left-hand menu in the Azure portal, click Resource groups and then click myResourceGroup.
2. On your resource group page, click Delete, type myResourceGroup in the text box, and then click Delete.
Next steps
Find and diagnose performance problems
Tutorial: Create a metrics chart in Azure Monitor
1/6/2020 • 3 minutes to read • Edit Online
Metrics explorer is a feature of Azure Monitor in the Azure portal that allows you to create charts from metric
values, visually correlate trends, and investigate spikes and dips in metric values. Use the metrics explorer to
investigate the health and utilization of your Azure resources or to plot charts from custom metrics.
In this tutorial, you learn how to:
Select a metric for which you want to plot a chart
Perform different aggregations of metric values
Modify the time range and granularity for the chart
Prerequisites
To complete this tutorial you need an Azure resource to monitor. You can use any resource in your Azure
subscription that supports metrics. To determine whether a resource supports metrics, go to its menu in the Azure
portal and verify that there's a Metrics option in the Monitoring section of the menu.
Log in to Azure
Log in to the Azure portal at https://portal.azure.com.
3. Select a Namespace if the scope has more than one. The namespace is just a way to organize metrics so
that you can easily find them. For example, storage accounts have separate namespaces for storing Files,
Tables, Blobs, and Queues metrics. Many resource types only have one namespace.
4. Select a metric from a list of available metrics for the selected scope and namespace.
5. Optionally, change the metric Aggregation. This defines how the metric values will aggregated across the
time granularity for the graph. For example, if the time granularity is set to 15 minutes and the aggregation
is set to sum, then each point in the graph will be the sum of all collected values over each 15 minute
segment.
6. Use the Add metric button and repeat these steps if you want to see multiple metrics plotted in the same
chart. For multiple charts in one view, select the New chart button.
Next steps
Now that you've learned how to work with metrics in Azure Monitor, learn how to use metrics to send proactive
alerts.
Create, view, and manage metric alerts using Azure Monitor
Tutorial: Collect and analyze resource logs from an
Azure resource
12/27/2019 • 4 minutes to read • Edit Online
Resource logs provide insight into the detailed operation of an Azure resource and are useful for monitoring their
health and availability. Azure resources generate resource logs automatically, but you must configure where they
should be collected. This tutorial takes you through the process of creating a diagnostic setting to collect resource
logs for a resource in your Azure subscription and analyzing it with a log query.
In this tutorial, you learn how to:
Create a Log Analytics workspace in Azure Monitor
Create a diagnostic setting to collect resource logs
Create a simple log query to analyze logs
Prerequisites
To complete this tutorial you need an Azure resource to monitor. You can use any resource in your Azure
subscription that supports diagnostic settings. To determine whether a resource supports diagnostic settings, go to
its menu in the Azure portal and verify that there's a Diagnostic settings option in the Monitoring section of the
menu.
Log in to Azure
Log in to the Azure portal at https://portal.azure.com.
Create a workspace
A Log Analytics workspace in Azure Monitor collects and indexes log data from a variety of sources and allows
advanced analysis using a powerful query language. The Log Analytics workspace needs to exist before you create
a diagnostic setting to send data into it. You can use an existing workspace in your Azure subscription or create
one with the following procedure.
NOTE
While you can work with data in Log Analytics workspaces in the Azure Monitor menu, you create and manage workspaces
in the Log Analytics workspaces menu.
NOTE
If you opened Logs from the Azure Monitor menu, the scope would be set to the Log Analytics workspace. In this
case, any queries will include all records in the workspace.
3. The service shown in the example writes resource logs to the AzureDiagnostics table, but other services
may write to other tables. See Supported services, schemas, and categories for Azure Resource Logs for
tables used by different Azure services.
NOTE
Multiple services write resource logs to the AzureDiagnostics table. If you start Log Analytics from the Azure Monitor
menu, then you would need to add a where statement with the ResourceProvider column to specify your
particular service. When you start Log Analytics from a resource's menu, then the scope is set to only records from
this resource so this column isn't required. See the service's documentation for sample queries.
Next steps
Now that you've learned how to collect resource logs into a Log Analytics workspace, complete a tutorial on
writing log queries to analyze this data.
Get started with log queries in Azure Monitor
Get started with log queries in Azure Monitor
12/13/2019 • 7 minutes to read • Edit Online
NOTE
You can work through this exercise in your own environment if you are collecting data from at least one virtual machine. If
not then use our Demo environment, which includes plenty of sample data.
In this tutorial you will learn to write log queries in Azure Monitor. It will teach you how to:
Understand query structure
Sort query results
Filter query results
Specify a time range
Select which fields to include in the results
Define and use custom fields
Aggregate and group results
For a tutorial on using Log Analytics in the Azure portal, see Get started with Azure Monitor Log Analytics.
For more details on log queries in Azure Monitor, see Overview of log queries in Azure Monitor.
Follow along with a video version of this tutorial below:
NOTE
The Kusto query language used by Azure Monitor is case-sensitive. Language keywords are typically written in lower-case.
When using names of tables or columns in a query, make sure to use the correct case, as shown on the schema pane.
SecurityEvent
| take 10
The query shown above returns 10 results from the SecurityEvent table, in no specific order. This is a very common
way to take a glance at a table and understand its structure and content. Let's examine how it's built:
The query starts with the table name SecurityEvent - this part defines the scope of the query.
The pipe (|) character separates commands, so the output of the first one in the input of the following command.
You can add any number of piped elements.
Following the pipe is the take command, which returns a specific number of arbitrary records from the table.
We could actually run the query even without adding | take 10 - that would still be valid, but it could return up to
10,000 results.
Search queries
Search queries are less structured, and generally more suited for finding records that include a specific value in any
of their columns:
This query searches the SecurityEvent table for records that contain the phrase "Cryptographic". Of those records,
10 records will be returned and displayed. If we omit the in (SecurityEvent) part and just run
search "Cryptographic" , the search will go over all tables, which would take longer and be less efficient.
WARNING
Search queries are typically slower than table-based queries because they have to process more data.
SecurityEvent
| sort by TimeGenerated desc
That could return too many results though and might also take some time. The above query sorts the entire
SecurityEvent table by the TimeGenerated column. The Analytics portal then limits the display to show only 10,000
records. This approach is of course not optimal.
The best way to get only the latest 10 records is to use top, which sorts the entire table on the server side and then
returns the top records:
SecurityEvent
| top 10 by TimeGenerated
Descending is the default sorting order, so we typically omit the desc argument.The output will look like this:
Where: filtering on a condition
Filters, as indicated by their name, filter the data by a specific condition. This is the most common way to limit
query results to relevant information.
To add a filter to a query, use the where operator followed by one or more conditions. For example, the following
query returns only SecurityEvent records where Level equals 8:
SecurityEvent
| where Level == 8
When writing filter conditions, you can use the following expressions:
SecurityEvent
| where Level == 8 and EventID == 4672
SecurityEvent
| where Level == 8
| where EventID == 4672
NOTE
Values can have different types, so you might need to cast them to perform comparison on the correct type. For example,
SecurityEvent Level column is of type String, so you must cast it to a numerical type such as int or long, before you can use
numerical operators on it: SecurityEvent | where toint(Level) >= 10
SecurityEvent
| where TimeGenerated > ago(30m)
| where toint(Level) >= 10
In the above time filter ago(30m) means "30 minutes ago" so this query only returns records from the last 30
minutes. Other units of time include days (2d), minutes (25m), and seconds (10s).
SecurityEvent
| top 10 by TimeGenerated
| project TimeGenerated, Computer, Activity
You can also use project to rename columns and define new ones. The following example uses project to do the
following:
Select only the Computer and TimeGenerated original columns.
Rename the Activity column to EventDetails.
Create a new column named EventCode. The substring() function is used to get only the first four characters
from the Activity field.
SecurityEvent
| top 10 by TimeGenerated
| project Computer, TimeGenerated, EventDetails=Activity, EventCode=substring(Activity, 0, 4)
extend keeps all original columns in the result set and defines additional ones. The following query uses extend to
add the EventCode column. Note that this column may not display at the end of the table results in which case you
would need to expand the details of a record to view it.
SecurityEvent
| top 10 by TimeGenerated
| extend EventCode=substring(Activity, 0, 4)
Perf
| where TimeGenerated > ago(1h)
| summarize count() by ObjectName
Sometimes it makes sense to define groups by multiple dimensions. Each unique combination of these values
defines a separate group:
Perf
| where TimeGenerated > ago(1h)
| summarize count() by ObjectName, CounterName
Another common use is to perform mathematical or statistical calculations on each group. For example, the
following calculates the average CounterValue for each computer:
Perf
| where TimeGenerated > ago(1h)
| summarize avg(CounterValue) by Computer
Unfortunately, the results of this query are meaningless since we mixed together different performance counters.
To make this more meaningful, we should calculate the average separately for each combination of CounterName
and Computer:
Perf
| where TimeGenerated > ago(1h)
| summarize avg(CounterValue) by Computer, CounterName
Perf
| where TimeGenerated > ago(7d)
| where Computer == "ContosoAzADDS2"
| where CounterName == "Available MBytes"
| summarize avg(CounterValue) by bin(TimeGenerated, 1h)
To make the output clearer, you select to display it as a time-chart, showing the available memory over time:
Next steps
Learn more about using string data in a log query with Work with strings in Azure Monitor log queries.
Learn more about aggregating data in a log query with Advanced aggregations in Azure Monitor log queries.
Learn how to join data from multiple tables with Joins in Azure Monitor log queries.
Get documentation on the entire Kusto query language in the KQL language reference.
Create and share dashboards of Log Analytics data
12/13/2019 • 3 minutes to read • Edit Online
Log Analytics dashboards can visualize all of your saved log queries, giving you the ability to find, correlate, and
share IT operational data in the organization. This tutorial covers creating a log query that will be used to support a
shared dashboard that will be accessed by your IT operations support team. You learn how to:
Create a shared dashboard in the Azure portal
Visualize a performance log query
Add a log query to a shared dashboard
Customize a tile in a shared dashboard
To complete the example in this tutorial, you must have an existing virtual machine connected to the Log Analytics
workspace.
Here you can bring together operational data that is most important to IT across all your Azure resources, including
telemetry from Azure Log Analytics. Before we step into visualizing a log query, let's first create a dashboard and
share it. We can then focus on our example performance log query, which will render as a line chart, and add it to
the dashboard.
To create a dashboard, select the New dashboard button next to the current dashboard's name.
This action creates a new, empty, private dashboard and puts you into customization mode where you can name
your dashboard and add or rearrange tiles. Edit the name of the dashboard and specify Sample Dashboard for this
tutorial, and then select Done customizing.
When you create a dashboard, it is private by default, which means you are the only person who can see it. To make
it visible to others, use the Share button that appears alongside the other dashboard commands.
You are asked to choose a subscription and resource group for your dashboard to be published to. For convenience,
the portal's publishing experience guides you towards a pattern where you place dashboards in a resource group
called dashboards. Verify the subscription selected and then click Publish. Access to the information displayed in
the dashboard is controlled with Azure Resource Based Access Control.
Perf
| where CounterName == "% Processor Time" and ObjectName == "Processor" and InstanceName == "_Total"
| summarize AggregatedValue = avg(CounterValue) by bin(TimeGenerated, 1hr), Computer
| render timechart
Save the query by selecting the Save button from the top of the page.
In the Save Query control panel, provide a name such as Azure VMs - Processor Utilization and a category such as
Dashboards and then click Save. This way you can create a library of common queries that you can use and modify.
Finally, pin this to the shared dashboard created earlier by selecting the Pin to dashboard button from the top
right corner of the page and then selecting the dashboard name.
Now that we have a query pinned to the dashboard, you will notice it has a generic title and comment below it.
We should rename it to something meaningful that can be easily understood by those viewing it. Click the edit
button to customize the title and subtitle for the tile, and then click Update. A banner will appear asking you to
publish changes or discard. Click Save a copy.
Next steps
In this tutorial, you learned how to create a dashboard in the Azure portal and add a log query to it. Advance to the
next tutorial to learn the different responses you can implement based on log query results.
Respond to events with Log Analytics Alerts
Respond to events with Azure Monitor Alerts
12/13/2019 • 5 minutes to read • Edit Online
Alerts in Azure Monitor can identify important information in your Log Analytics repository. They are created by
alert rules that automatically run log searches at regular intervals, and if results of the log search match particular
criteria, then an alert record is created and it can be configured to perform an automated response. This tutorial is
a continuation of the Create and share dashboards of Log Analytics data tutorial.
In this tutorial, you learn how to:
Create an alert rule
Configure an Action Group to send an e-mail notification
To complete the example in this tutorial, you must have an existing virtual machine connected to the Log Analytics
workspace.
Create alerts
Alerts are created by alert rules in Azure Monitor and can automatically run saved queries or custom log searches
at regular intervals. You can create alerts based on specific performance metrics or when certain events are
created, absence of an event, or a number of events are created within a particular time window. For example,
alerts can be used to notify you when average CPU usage exceeds a certain threshold, when a missing update is
detected, or when an event is generated upon detecting that a specific Windows service or Linux daemon is not
running. If the results of the log search match particular criteria, then an alert is created. The rule can then
automatically run one or more actions, such as notify you of the alert or invoke another process.
In the following example, you create a metric measurement alert rule based off of the Azure VMs - Processor
Utilization query saved in the Visualize data tutorial. An alert is created for each virtual machine that exceeds a
threshold of 90%.
1. In the Azure portal, click All services. In the list of resources, type Log Analytics. As you begin typing, the
list filters based on your input. Select Log Analytics.
2. In the left-hand pane, select Alerts and then click New Alert Rule from the top of the page to create a new
alert.
3. For the first step, under the Create Alert section, you are going to select your Log Analytics workspace as
the resource, since this is a log based alert signal. Filter the results by choosing the specific Subscription
from the drop-down list if you have more than one, which contains the VM and Log Analytics workspace
created earlier. Filter the Resource Type by selecting Log Analytics from the drop-down list. Finally, select
the Resource DefaultLAWorkspace and then click Done.
4. Under the section Alert Criteria, click Add Criteria to select our saved query and then specify logic that
the alert rule follows. From the Configure signal logic pane, select Azure VMs - Processor Utilization from
the list. The pane updates to present the configuration settings for the alert. On the top, it shows the results
for the last 30 minutes of the selected signal and the search query itself.
5. Configure the alert with the following information:
a. From the Based on drop-down list, select Metric measurement. A metric measurement will create an
alert for each object in the query with a value that exceeds our specified threshold.
b. For the Condition, select Greater than and enter 90 for Threshold.
c. Under Trigger Alert Based On section, select Consecutive breaches and from the drop-down list select
Greater than enter a value of 3.
d. Under Evaluation based on section, modify the Period value to 30 minutes. The rule will run every five
minutes and return records that were created within the last thirty minutes from the current time. Setting
the time period to a wider window accounts for the potential of data latency, and ensures the query returns
data to avoid a false negative where the alert never fires.
6. Click Done to complete the alert rule.
7. Now moving onto the second step, provide a name of your alert in the Alert rule name field, such as
Percentage CPU greater than 90 percent. Specify a Description detailing specifics for the alert, and
select Critical(Sev 0) for the Severity value from the options provided.
8. To immediately activate the alert rule on creation, accept the default value for Enable rule upon creation.
9. For the third and final step, you specify an Action Group, which ensures that the same actions are taken
each time an alert is triggered and can be used for each rule you define. Configure a new action group with
the following information:
a. Select New action group and the Add action group pane appears.
b. For Action group name, specify a name such as IT Operations - Notify and a Short name such as
itops-n.
c. Verify the default values for Subscription and Resource group are correct. If not, select the correct one
from the drop-down list.
d. Under the Actions section, specify a name for the action, such as Send Email and under Action Type
select Email/SMS/Push/Voice from the drop-down list. The Email/SMS/Push/Voice properties pane
will open to the right in order to provide additional information.
e. On the Email/SMS/Push/Voice pane, enable Email and provide a valid email SMTP address to deliver
the message to.
f. Click OK to save your changes.
Next steps
In this tutorial, you learned how alert rules can proactively identify and respond to an issue when they run log
searches at scheduled intervals and match a particular criteria.
Follow this link to see pre-built Log Analytics script samples.
Log Analytics script samples
Create an Autoscale Setting for Azure resources
based on performance data or a schedule
3/15/2019 • 5 minutes to read • Edit Online
Autoscale settings enable you to add/remove instances of service based on preset conditions. These settings can
be created through the portal. This method provides a browser-based user interface for creating and configuring
an autoscale setting.
In this tutorial, you will
Create a Web App and App Service Plan
Configure autoscale rules for scale-in and scale out based on the number of requests a Web App receives
Trigger a scale-out action and watch the number of instances increase
Trigger a scale-in action and watch the number of instances decrease
Clean up your resources
If you don't have an Azure subscription, create a free account before you begin.
Clean up resources
1. From the left-hand menu in the Azure portal, click All resources and then select the Web App created in
this tutorial.
2. On your resource page, click Delete, confirm delete by typing yes in the text box, and then click Delete.
3. Then select the App Service Plan resource and click Delete.
4. Confirm delete by typing yes in the text box, and then click Delete.
Next steps
In this tutorial, you
Created a Web App and App Service Plan
Configured autoscale rules for scale-in and scale out based on the number of requests the Web App received
Triggered a scale-out action and watched the number of instances increase
Triggered a scale-in action and watched the number of instances decrease
Cleaned up your resources
To learn more about autoscale settings, continue on to the autoscale overview.
Archive your monitoring data
Find and diagnose run-time exceptions with Azure
Application Insights
12/13/2019 • 4 minutes to read • Edit Online
Azure Application Insights collects telemetry from your application to help identify and diagnose run-time
exceptions. This tutorial takes you through this process with your application. You learn how to:
Modify your project to enable exception tracking
Identify exceptions for different components of your application
View details of an exception
Download a snapshot of the exception to Visual Studio for debugging
Analyze details of failed requests using query language
Create a new work item to correct the faulty code
Prerequisites
To complete this tutorial:
Install Visual Studio 2019 with the following workloads:
ASP.NET and web development
Azure development
Download and install the Visual Studio Snapshot Debugger.
Enable Visual Studio Snapshot Debugger
Deploy a .NET application to Azure and enable the Application Insights SDK.
The tutorial tracks the identification of an exception in your application, so modify your code in your
development or test environment to generate an exception.
Log in to Azure
Log in to the Azure portal at https://portal.azure.com.
Analyze failures
Application Insights collects any failures in your application and lets you view their frequency across different
operations to help you focus your efforts on those with the highest impact. You can then drill down on details of
these failures to identify root cause.
1. Select Application Insights and then your subscription.
2. To open the Failures panel either select Failures under the Investigate menu or click the Failed requests
graph.
3. The Failed requests panel shows the count of failed requests and the number of users affected for each
operation for the application. By sorting this information by user you can identify those failures that most
impact users. In this example, the GET Employees/Create and GET Customers/Details are likely
candidates to investigate because of their large number of failures and impacted users. Selecting an
operation shows further information about this operation in the right panel.
4. Reduce the time window to zoom in on the period where the failure rate shows a spike.
5. See the related samples by clicking on the button with the number of filtered results. The "suggested"
samples have related telemetry from all components, even if sampling may have been in effect in any of
them. Click on a search result to see the details of the failure.
6. The details of the failed request shows the Gantt chart which shows that there were two dependency
failures in this transaction, which also attributed to over 50% of the total duration of the transaction. This
experience presents all telemetry, across components of a distributed application that are related to this
operation ID. Learn more about the new experience. You can select any of the items to see its details on the
right side.
7. The operations detail also shows a FormatException which appears to have caused the failure. You can see
that it's due to an invalid zip code. You can open the debug snapshot to see code level debug information in
Visual Studio.
4. You then have the option to download this snapshot into Visual Studio where we can locate the actual code
that needs to be corrected. To do so, click Download Snapshot.
5. The snapshot is loaded into Visual Studio.
6. You can now run a debug session in Visual Studio Enterprise that quickly identifies the line of code that
caused the exception.
Use analytics data
All data collected by Application Insights is stored in Azure Log Analytics, which provides a rich query language
that allows you to analyze the data in a variety of ways. We can use this data to analyze the requests that generated
the exception we're researching.
1. Click the CodeLens information above the code to view telemetry provided by Application Insights.
2. Click Analyze impact to open Application Insights Analytics. It's populated with several queries that
provide details on failed requests such as impacted users, browsers, and regions.
Add work item
If you connect Application Insights to a tracking system such as Azure DevOps or GitHub, you can create a work
item directly from Application Insights.
1. Return to the Exception Properties panel in Application Insights.
2. Click New Work Item.
3. The New Work Item panel opens with details about the exception already populated. You can add any
additional information before saving it.
Next steps
Now that you've learned how to identify run-time exceptions, advance to the next tutorial to learn how to identify
and diagnose performance issues.
Identify performance issues
Find and diagnose performance issues with Azure
Application Insights
12/19/2019 • 5 minutes to read • Edit Online
Azure Application Insights collects telemetry from your application to help analyze its operation and performance.
You can use this information to identify problems that may be occurring or to identify improvements to the
application that would most impact users. This tutorial takes you through the process of analyzing the
performance of both the server components of your application and the perspective of the client. You learn how to:
Identify the performance of server-side operations
Analyze server operations to determine the root cause of slow performance
Identify slowest client-side operations
Analyze details of page views using query language
Prerequisites
To complete this tutorial:
Install Visual Studio 2019 with the following workloads:
ASP.NET and web development
Azure development
Deploy a .NET application to Azure and enable the Application Insights SDK.
Enable the Application Insights profiler for your application.
Log in to Azure
Log in to the Azure portal at https://portal.azure.com.
4. The graph currently shows the average duration of the selected operations over time. You can switch to the
95th percentile to find the performance issues. Add the operations that you're interested in by pinning them
to the graph. This shows that there are some peaks worth investigating. Isolate this further by reducing the
time window of the graph.
5. The performance panel on the right shows distribution of durations for different requests for the selected
operation. Reduce the window to start around the 95th percentile. The "Top 3 dependencies" insights card,
can tell you at a glance that the external dependencies are likely contributing to the slow transactions. Click
on the button with number of samples to see a list of the samples. You can then select any sample to see
transaction details.
6. You can see at a glance that the call to Fabrikamaccount Azure Table is contributing most to the total
duration of the transaction. You can also see that an exception caused it to fail. You can click on any item in
the list to see its details on the right side. Learn more about the transaction diagnostics experience
7. The Profiler helps get further with code level diagnostics by showing the actual code that ran for the
operation and the time required for each step. Some operations may not have a trace since the profiler runs
periodically. Over time, more operations should have traces. To start the profiler for the operation, click
Profiler traces.
8. The trace shows the individual events for each operation so you can diagnose the root cause for the
duration of the overall operation. Click one of the top examples, which have the longest duration.
9. Click Hot Path to highlight the specific path of events that most contribute to the total duration of the
operation. In this example, you can see that the slowest call is from
FabrikamFiberAzureStorage.GetStorageTableData method. The part that takes most time is the
CloudTable.CreateIfNotExist method. If this line of code is executed every time the function gets called,
unnecessary network call and CPU resource will be consumed. The best way to fix your code is to put this
line in some startup method that only executes once.
10. The Performance Tip at the top of the screen supports the assessment that the excessive duration is due
to waiting. Click the waiting link for documentation on interpreting the different types of events.
11. For further analysis, you can click Download Trace to download the trace. You can view this data using
PerfView.
2. Select on one of the operation names then click the blue samples button in the bottom right and select an
operation. This will bring up the end-to-end transaction details and on the right side you can view the Page
View Properties. This allows you to view details of the client requesting the page including the type of
browser and its location. This information can assist you in determining whether there are performance
issues related to particular types of clients.
Use logs data for client
Like the data collected for server performance, Application Insights makes all client data available for deep
analysis using Logs.
1. Return to the browser summary and click View in Logs (Analytics)
2. Logs opens with a query for each of the views in the panel. The first query shows the duration for different
page views over time.
3. Smart Diagnostics is a feature of Logs identifies unique patterns in the data. When you click the Smart
Diagnostics dot in the line chart, the same query is run without the records that caused the anomaly. Details
of those records are shown in the comment section of the query so you can identify the properties of those
page views that are causing the excessive duration.
Next steps
Now that you've learned how to identify run-time exceptions, advance to the next tutorial to learn how to create
alerts in response to failures.
Alert on application health
Monitor and alert on application health with Azure
Application Insights
12/13/2019 • 2 minutes to read • Edit Online
Azure Application Insights allows you to monitor your application and send you alerts when it is either unavailable,
experiencing failures, or suffering from performance issues. This tutorial takes you through the process of creating
tests to continuously check the availability of your application.
You learn how to:
Create availability test to continuously check the response of the application
Send mail to administrators when a problem occurs
Prerequisites
To complete this tutorial:
Create an Application Insights resource.
Sign in to Azure
Sign in to the Azure portal at https://portal.azure.com.
3. Type in a name for the test and leave the other defaults. This selection will trigger requests for the
application url every 5 minutes from five different geographic locations.
4. Select Alerts to open the Alerts dropdown where you can define details for how to respond if the test fails.
Choose Near-realtime and set the status to Enabled.
Type in an email address to send when the alert criteria is met. You could optionally type in the address of a
webhook to call when the alert criteria is met.
5. Return to the test panel, select the ellipses and edit alert to enter the configuration for your near-realtime
alert.
6. Set failed locations to greater than or equal to 3. Create an action group to configure who gets notified
when your alert threshold is breached.
7. Once you have configured your alert, click on the test name to view details from each location. Tests can be
viewed in both line graph and scatter plot format to visualize the success/failures for a given time range.
8. You can drill down into the details of any test by clicking on its dot in the scatter chart. This will launch the
end-to-end transaction details view. The example below shows the details for a failed request.
Next steps
Now that you've learned how to alert on issues, advance to the next tutorial to learn how to analyze how users are
interacting with your application.
Understand users
Use Azure Application Insights to understand how
customers are using your application
10/24/2019 • 6 minutes to read • Edit Online
Azure Application Insights collects usage information to help you understand how your users interact with your
application. This tutorial walks you through the different resources that are available to analyze this information.
You will learn how to:
Analyze details about users accessing your application
Use session information to analyze how customers use your application
Define funnels that let you compare your desired user activity to their actual activity
Create a workbook to consolidate visualizations and queries into a single document
Group similar users to analyze them together
Learn which users are returning to your application
Inspect how users navigate through your application
Prerequisites
To complete this tutorial:
Install Visual Studio 2019 with the following workloads:
ASP.NET and web development
Azure development
Download and install the Visual Studio Snapshot Debugger.
Deploy a .NET application to Azure and enable the Application Insights SDK.
Send telemetry from your application for adding custom events/page views
Send user context to track what a user does over time and fully utilize the usage features.
Log in to Azure
Log in to the Azure portal at https://portal.azure.com.
5. Click the Split by dropdown to add a breakdown by a user property to the graph. Select Country or
region. The graph includes the same data but allows you to view a breakdown of the number of users for
each country/region.
6. Position the cursor over different bars in the chart and note that the count for each country/region reflects
only the time window represented by that bar.
7. Have a look at the Insights column at the right that perform analysis on your user data. This provides
information such as the number of unique sessions over the time period and records with common
properties that make up significant of the user data
Analyze user sessions
The Sessions panel is similar to the Users panel. Where Users helps you understand details about the users
accessing your application, Sessions helps you understand how those users used your application.
1. Select Sessions in the menu.
2. Have a look at the graph and note that you have the same options to filter and break down the data as in
the Users panel.
3. The Sample of these sessions pane on the right lists sessions that include a large number of events. These
are interesting sessions to analyze.
4. Click on one of the sessions to view its Session Timeline, which shows every action in the sessions. This
can help you identify information such as the sessions with a large number of exceptions.
4. Click Save to save the funnel and then view its results. The window to the right of the funnel shows the
most common events before the first activity and after the last activity to help you understand user
tendencies around the particular sequence.
Learn which customers return
Retention helps you understand which users are coming back to your application.
1. Select Retention in the menu.
2. By default, the analyzed information includes users who performed any action and then returned to perform
any action. You can change this filter to any include, for example, only those users who returned after
completing a purchase.
3. The returning users that match the criteria are shown in graphical and table form for different time
durations. The typical pattern is for a gradual drop in returning users over time. A sudden drop from one
time period to the next might raise a concern.
Analyze user navigation
A User flow visualizes how users navigate between the pages and features of your application. This helps you
answer questions such as where users typically move from a particular page, how they typically exit your
application, and if there are any actions that are regularly repeated.
1. Select User flows in the menu.
2. Click New to create a new user flow and then click Edit to edit its details.
3. Increase the Time Range to 7 days and then select an initial event. The flow will track user sessions that
start with that event.
4. The user flow is displayed, and you can see the different user paths and their session counts. Blue lines
indicate an action that the user performed after the current action. A red line indicates the end of the user
session.
5. To remove an event from the flow, click the x in the corner of the action and then click Create Graph. The
graph is redrawn with any instances of that event removed. Click Edit to see that the event is now added to
Excluded events.
6. Click Add users to add a graph with user information. Edit the details of the graph if you want and then
click Done editing to save it.
Next steps
Now that you've learned how to analyze your users, advance to the next tutorial to learn how to create custom
dashboards that combine this information with other useful data about your application.
Create custom dashboards
Create custom KPI dashboards using Azure
Application Insights
10/24/2019 • 5 minutes to read • Edit Online
You can create multiple dashboards in the Azure portal that each include tiles visualizing data from multiple Azure
resources across different resource groups and subscriptions. You can pin different charts and views from Azure
Application Insights to create custom dashboards that provide you with complete picture of the health and
performance of your application. This tutorial walks you through the creation of a custom dashboard that includes
multiple types of data and visualizations from Azure Application Insights. You learn how to:
Create a custom dashboard in Azure
Add a tile from the Tile Gallery
Add standard metrics in Application Insights to the dashboard
Add a custom metric chart Application Insights to the dashboard
Add the results of a Logs (Analytics) query to the dashboard
Prerequisites
To complete this tutorial:
Deploy a .NET application to Azure and enable the Application Insights SDK.
Sign in to Azure
Sign in to the Azure portal at https://portal.azure.com.
6. Click Done customizing at the top of the screen to exit tile customization mode.
2. Keep the Dashboard name the same and select the Subscription Name to share the dashboard. Click
Publish. The dashboard is now available to other services and subscriptions. You can optionally define
specific users who should have access to the dashboard.
3. Select your Application Insights resource in the home screen.
4. Click Logs (Analytics) on the left under monitoring to open the Logs (Analytics) portal.
5. Type the following query, which returns the top 10 most requested pages and their request count:
requests
| summarize count() by name
| sort by count_ desc
| take 10
exceptions
| summarize count() by operation_Name
| sort by count_ desc
| take 10
10. Click the pin icon on the top right to pin the chart to your dashboard and this time select the link to return
to your dashboard.
11. The results of the queries are now added to your dashboard in the format that you selected. Click and drag
each into position and then click Done customizing.
12. Select the pencil icon on each title to give them a descriptive title.
13. Select Share to republish your changes to your dashboard that now includes a variety of charts and
visualizations from Application Insights.
Next steps
Now that you've learned how to create custom dashboards, have a look at the rest of the Application Insights
documentation including a case study.
Deep diagnostics
Azure Monitor PowerShell samples
12/13/2019 • 2 minutes to read • Edit Online
The following table includes links to PowerShell scripts samples to perform various functions in Azure Monitor.
Create workspace
Create a Log Analytics workspace Creates a Log Analytics workspace in Azure Monitor.
What is monitored by Azure Monitor?
1/17/2020 • 9 minutes to read • Edit Online
This article describes the different applications and services that are monitored by Azure Monitor.
INSIGHT DESCRIPTION
Azure Monitor for Containers Monitors the performance of container workloads deployed to
either Azure Container Instances or managed Kubernetes
clusters hosted on Azure Kubernetes Service (AKS).
Azure Monitor for Cosmos DB (preview) Provides a view of the overall performance, failures, capacity,
and operational health of all your Azure Cosmos DB resources
in a unified interactive experience.
Azure Monitor for Networks (preview) Provides a comprehensive view of health and metrics for all
your network resource. The advanced search capability helps
you identify resource dependencies, enabling scenarios like
identifying resource that are hosting your website, by simply
searching for your website name.
Azure Monitor for Resource Groups (preview) Triage and diagnose any problems your individual resources
encounter, while offering context as to the health and
performance of the resource group as a whole.
Azure Monitor for Storage (preview) Provides comprehensive monitoring of your Azure Storage
accounts by delivering a unified view of your Azure Storage
services performance, capacity, and availability.
Azure Monitor for VMs (preview) Monitors your Azure virtual machines (VM) and virtual
machine scale sets at scale. It analyzes the performance and
health of your Windows and Linux VMs, and monitors their
processes and dependencies on other resources and external
processes.
Core solutions
Solutions are based on log queries and views customized for a particular application or service. They collect and
analyze logs only and are being deprecated over time in favor of insights.
SOLUTION DESCRIPTION
Agent health Analyze the health and configuration of Log Analytics agents.
Azure services
The following table lists Azure services and the data they collect into Azure Monitor.
Metrics - The service automatically collects metrics into Azure Monitor Metrics.
Logs - The service supports diagnostic settings which can collect platform logs and metrics to Azure Monitor
Logs.
Insight - There is an insight available for the service which provides a customized monitoring experience for the
service.
Advanced Threat No No No
Protection
Advisor No No No
AI Builder No No No
AppConfig No No No
Attestation Service No No No
SERVICE METRICS LOGS INSIGHT NOTES
Azure Service No No No
Manager (RDFE)
Backup No Yes No
Bastion No No No
Batch AI No No No
Blueprints No No No
Bot Service No No No
Cloud Shell No No No
Cost Management No No No
Data Box No No No
Data Share No No No
SERVICE METRICS LOGS INSIGHT NOTES
Database Migration No No No
Service
Databricks No Yes No
DevOps No No No
DNS Yes No No
Domain names No No No
DPS No No No
Dynamics 365 No No No
Customer
Engagement
Dynamics 365 No No No
Finance and
Operations
HDInsight No Yes No
HPC Cache No No No
Information No Yes No
Protection
Intune No Yes No
SERVICE METRICS LOGS INSIGHT NOTES
IoT Central No No No
Machine Learning No No No
Service
Managed Applications No No No
Maps No No No
Microsoft Flow No No No
Microsoft Managed No No No
Desktop
Microsoft PowerApps No No No
Microsoft Social No No No
Engagement
Migrate No No No
Multi-Factor No Yes No
Authentication
Open Datasets No No No
Policy No No No
Power BI Embedded No No No
SERVICE METRICS LOGS INSIGHT NOTES
Private Link No No No
Project Spool No No No
Communication
Platform
Resource Graph No No No
Resource Manager No No No
Retail Search – by No No No
Bing
Signup Portal No No No
Stack No No No
Storage Cache No No No
TINA No No No
Universal Print No No No
Windows Virtual No No No
Desktop
Product integrations
The services and solutions in the following table store their data in a Log Analytics workspace so that it can be
analyzed with other log data collected by Azure Monitor.
PRODUCT/SERVICE DESCRIPTION
Azure Information Protection Classify and optionally protect documents and emails. See
Central reporting for Azure Information Protection.
Azure Security Center Collect and analyze security events and perform threat
analysis. See Data collection in Azure Security Center
Microsoft Intune Create a diagnostic setting to send logs to Azure Monitor. See
Send log data to storage, event hubs, or log analytics in
Intune (preview).
PRODUCT/SERVICE DESCRIPTION
Office 365 Monitor your Office 365 environment. Updated version with
improved onboarding available through Azure Sentinel.
Surface Hub Track the health and usage of Surface Hub devices.
System Center Operations Manager Collect data from Operations Manager agents by connecting
their management group to Azure Monitor. See Connect
Operations Manager to Azure Monitor
Assess the risk and health of your System Center Operations
Manager management group with Operations Manager
Assessment solution.
Visual Studio App Center Build, test, and distribute applications and then monitor their
status and usage. See Start analyzing your mobile app with
App Center and Application Insights.
Other solutions
Other solutions are available for monitoring different applications and services, but active development has
stopped and they may not be available in all regions. They are covered by the Azure Log Analytics data ingestion
service level agreement.
SOLUTION DESCRIPTION
Active Directory assessment Assess the risk and health of your Active Directory
environments.
Active Directory replication status Regularly monitors your Active Directory environment for any
replication failures.
Activity log analytics Analyze Activity log entries using predefined log queries and
views.
SOLUTION DESCRIPTION
DNS Analytics (preview) Collects, analyzes, and correlates Windows DNS analytic and
audit logs and other related data from your DNS servers.
Cloud Foundry Collect, view, and analyze your Cloud Foundry system health
and performance metrics, across multiple deployments.
On-Demand Assessments Assess and optimize the availability, security, and performance
of your on-premises, hybrid, and cloud Microsoft technology
environments.
SQL assessment Assess the risk and health of your SQL Server environments.
RESOURCE METHOD
Virtual machines Use the Log Analytics agent to collect data from the guest
operating system of virtual machines in other cloud
environments or on-premises. See Collect log data with the
Log Analytics agent.
REST API Client Separate APIs are available to write data to Azure Monitor
Logs and Metrics from any REST API client. See Send log data
to Azure Monitor with the HTTP Data Collector API for Logs
and Send custom metrics for an Azure resource to the Azure
Monitor metric store by using a REST API for Metrics.
Next steps
Read more about the Azure Monitor data platform which stores the logs and metrics collected by insights and
solutions.
Complete a tutorial on monitoring an Azure resource.
Complete a tutorial on writing a log query to analyze data in Azure Monitor Logs.
Complete a tutorial on creating a metrics chart to analyze data in Azure Monitor Metrics.
Azure Management - Monitoring
1/14/2020 • 3 minutes to read • Edit Online
Monitoring in Azure is one aspect of Azure Management. This article briefly describes the different areas of
management required to deploy and maintain your applications and resources in Azure with links to
documentation to get you started.
Management in Azure
Management refers to the tasks and processes required to maintain your business applications and the resources
that support them. Azure has multiple services and tools that work together to provide complete management for
not only your applications running in Azure but also in other clouds and on-premises. Understanding the different
tools available and how they can be used together for a variety of management scenarios is the first step in
designing a complete management environment.
The following diagram illustrates the different areas of management that are required to maintain any application
or resource. These different areas can be thought of in terms of a lifecycle where each is required in continuous
succession over the lifespan of a resource. This starts with its initial deployment, through its continued operation,
and finally when it's retired.
The following sections briefly describe the different management areas and provide links to detailed content on the
main Azure services intended to address them.
Monitor
Monitoring is the act of collecting and analyzing data to determine the performance, health, and availability of your
business application and the resources it depends on. An effective monitoring strategy will help you understand the
detailed operation of the different components of your application and to increase your uptime by proactively
notifying you of critical issues so that you can resolve them before they become problems. Monitoring in Azure is
primarily provided by Azure Monitor which provides common stores for storing monitoring data, multiple data
sources for collecting data from the different tiers supporting your application, and features for analyzing and
responding to collected data.
Configure
Configure refers to the initial deployment and configuration of applications and resources and their ongoing
maintenance with patches and updates. Automation of these tasks through script and policy allows you to
eliminate redundancy, minimizing your time and effort and increasing your accuracy and efficiency. Azure
Automation provides the bulk of services for automating configuration tasks. In addition to runbooks for
automating processes, it provides configuration and update management, which assist you in managing
configuration through policy and in identifying and deploying updates.
Govern
Governance provides mechanisms and processes to maintain control over your applications and resources in
Azure. It involves planning your initiatives and setting strategic priorities. Governance in Azure is primarily
implemented with two services. Azure Policy allows you to create, assign and, manage policy definitions that
enforce different rules and actions over your resources, so those resources stay compliant with your corporate
standards and service level agreements. Azure Cost Management allows you to track cloud usage and
expenditures for your Azure resources and other cloud providers including AWS and Google.
Secure
Managing security of your applications, resources, and data involves a combination of assessing threats, collecting
and analyzing security data, and ensuring that your applications and resources are designed and configured in a
secure fashion. Security monitoring and threat analysis are provided by Azure Security Center which includes
unified security management and advanced threat protection across hybrid cloud workloads. You should also see
Introduction to Azure Security for comprehensive information on security in Azure and guidance on securely
configuring Azure resources.
Protect
Protection refers to ensuring that your applications and data are always available, even in the case of outages
beyond your control. Protection in Azure is provided by two services. Azure Backup provides backup and recovery
of your data, either in the cloud or on-premises. Azure Site Recovery ensures high availability of your application
by providing business continuity and immediate recovery in the case of disaster.
Migrate
Migration refers to transitioning workloads currently running on-premises to the Azure cloud. Azure Migrate is a
service that helps you assess the migration suitability, including performance-based sizing and cost estimates, of
on-premises virtual machines to Azure. Azure Site Recovery can help you perform the actual migration of virtual
machines from on-premises or from Amazon Web Services. Azure Database Migration will assist you in migrating
multiple database sources to Azure Data platforms.
Continuous monitoring with Azure Monitor
12/13/2019 • 5 minutes to read • Edit Online
Continuous monitoring refers to the process and technology required to incorporate monitoring across each phase
of your DevOps and IT operations lifecycles. It helps to continuously ensure the health, performance, and reliability
of your application and infrastructure as it moves from development to production. Continuous monitoring builds
on the concepts of Continuous Integration and Continuous Deployment (CI/CD ) which help you develop and
deliver software faster and more reliably to provide continuous value to your users.
Azure Monitor is the unified monitoring solution in Azure that provides full-stack observability across applications
and infrastructure in the cloud and on-premises. It works seamlessly with Visual Studio and Visual Studio Code
during development and test and integrates with Azure DevOps for release management and work item
management during deployment and operations. It even integrates across the ITSM and SIEM tools of your choice
to help track issues and incidents within your existing IT processes.
This article describes specific steps for using Azure Monitor to enable continuous monitoring throughout your
workflows. It includes links to other documentation that provides details on implementing different features.
Continuously optimize
Monitoring is one of the fundamental aspects of the popular Build-Measure-Learn philosophy, which recommends
continuously tracking your KPIs and user behavior metrics and then striving to optimize them through planning
iterations. Azure Monitor helps you collect metrics and logs relevant to your business and to add new data points
in the next deployment as required.
Use tools in Application Insights to track end-user behavior and engagement.
Use Impact Analysis to help you prioritize which areas to focus on to drive to important KPIs.
Next steps
Learn about the difference components of Azure Monitor.
Add continuous monitoring to your release pipeline.
Sources of monitoring data for Azure Monitor
1/8/2020 • 12 minutes to read • Edit Online
Azure Monitor is based on a common monitoring data platform that includes Logs and Metrics. Collecting data
into this platform allows data from multiple resources to be analyzed together using a common set of tools in
Azure Monitor. Monitoring data may also be sent to other locations to support certain scenarios, and some
resources may write to other locations before they can be collected into Logs or Metrics.
This article describes the different sources of monitoring data collected by Azure Monitor in addition to the
monitoring data created by Azure resources. Links are provided to detailed information on configuration
required to collect this data to different locations.
Application tiers
Sources of monitoring data from Azure applications can be organized into tiers, the highest tiers being your
application itself and the lower tiers being components of Azure platform. The method of accessing data from
each tier varies. The application tiers are summarized in the table below, and the sources of monitoring data in
each tier are presented in the following sections. See Monitoring data locations in Azure for a description of
each data location and how you can access its data.
Azure
The following table briefly describes the application tiers that are specific to Azure. Following the link for
further details on each in the sections below.
Azure Tenant Data about the operation of tenant- View AAD data in portal or configure
level Azure services, such as Azure collection to Azure Monitor using a
Active Directory. tenant diagnostic setting.
TIER DESCRIPTION COLLECTION METHOD
Azure subscription Data related to the health and View in portal or configure collection
management of cross-resource to Azure Monitor using a log profile.
services in your Azure subscription
such as Resource Manager and Service
Health.
Azure resources Data about the operation and Metrics collected automatically, view in
performance of each Azure resource. Metrics Explorer.
Configure diagnostic settings to collect
logs in Azure Monitor.
Monitoring solutions and Insights
available for more detailed monitoring
for specific resource types.
Operating system (guest) Data about the operating system on Install Log Analytics agent to collect
compute resources. client data sources into Azure Monitor
and Dependency agent to collect
dependencies supporting Azure
Monitor for VMs.
For Azure virtual machines, install
Azure Diagnostic Extension to collect
logs and metrics into Azure Monitor.
Application Code Data about the performance and Instrument your code to collect data
functionality of the actual application into Application Insights.
and code, including performance
traces, application logs, and user
telemetry.
Custom sources Data from external services or other Collect log or metrics data into Azure
components or devices. Monitor from any REST client.
Azure tenant
Telemetry related to your Azure tenant is collected from tenant-wide services such as Azure Active Directory.
Azure Active Directory Audit Logs
Azure Active Directory reporting contains the history of sign-in activity and audit trail of changes made within a
particular tenant.
Azure Monitor Logs Configure Azure AD logs to be Integrate Azure AD logs with Azure
collected in Azure Monitor to analyze Monitor logs (preview)
them with other monitoring data.
Azure Storage Export Azure AD logs to Azure Storage Tutorial: Archive Azure AD logs to an
for archiving. Azure storage account (preview)
Event Hub Stream Azure AD logs to other Tutorial: Stream Azure Active Directory
locations using Event Hubs. logs to an Azure event hub (preview).
Azure subscription
Telemetry related to the health and operation of your Azure subscription.
Activity log The Activity log is collected into its Query the Activity log in the Azure
own data store that you can view from portal
the Azure Monitor menu or use to
create Activity log alerts.
Azure Monitor Logs Configure Azure Monitor Logs to Collect and analyze Azure activity logs
collect the Activity log to analyze it in Log Analytics workspace in Azure
with other monitoring data. Monitor
Azure Storage Export the Activity log to Azure Archive Activity log
Storage for archiving.
Event Hubs Stream the Activity log to other Stream Activity log to Event Hub.
locations using Event Hubs
Activity log Service Health records are stored in View service health notifications by
Azure Monitor Logs the Azure Activity log, so you can view using the Azure portal
them in the Azure portal or perform
any other activities you can perform
with the Activity log.
Azure resources
Metrics and resource logs provide information about the internal operation of Azure resources. These are
available for most Azure services, and monitoring solutions and insights collect additional data for particular
services.
Platform metrics
Most Azure services will send platform metrics that reflect their performance and operation directly to the
metrics database. The specific metrics will vary for each type of resource.
DESTINATION DESCRIPTION REFERENCE
Azure Monitor Metrics Platform metrics will write to the Azure Getting started with Azure Metrics
Monitor metrics database with no Explorer
configuration. Access platform metrics Supported metrics with Azure Monitor
from Metrics Explorer.
Azure Monitor Logs Copy platform metrics to Logs for Azure diagnostics direct to Log
trending and other analysis using Log Analytics
Analytics.
Event Hubs Stream metrics to other locations Stream Azure monitoring data to an
using Event Hubs. event hub for consumption by an
external tool
Resource logs
Resource logs provide insights into the internal operation of an Azure resource. Resource logs are created
automatically, but you must create a diagnostic setting to specify a destination for them to collected for each
resource.
The configuration requirements and content of resource logs vary by resource type, and not all services yet
create them. See Supported services, schemas, and categories for Azure resource logs for details on each
service and links to detailed configuration procedures. If the service isn’t listed in this article, then that service
doesn’t currently create resource logs.
Azure Monitor Logs Send resource logs to Azure Monitor Collect Azure resource logs in Log
Logs for analysis with other collected Analytics workspace in Azure Monitor
log data.
Storage Send resource logs to Azure Storage Archive Azure resource logs
for archiving.
Event Hubs Stream resource logs to other Stream Azure resource logs to an
locations using Event Hubs. event hub
Storage When you enable the Diagnostics Store and view diagnostic data in
Extension, it will write to a storage Azure Storage
account by default.
Azure Monitor Metrics When you configure the Diagnostics Send Guest OS metrics to the Azure
Extension to collect performance Monitor metric store using a Resource
counters, they are written to the Azure Manager template for a Windows
Monitor metrics database. virtual machine
Application Insights Logs Collect logs and performance counters Send Cloud Service, Virtual Machine,
from the compute resource or Service Fabric diagnostic data to
supporting your application to be Application Insights
analyzed with other application data.
Event Hubs Configure the Diagnostics Extension to Streaming Azure Diagnostics data in
stream the data to other locations the hot path by using Event Hubs
using Event Hubs.
Azure Monitor Logs The Log Analytics agent connects to Agent data sources in Azure Monitor
Azure Monitor either directly or Connect Operations Manager to Azure
through System Center Operations Monitor
Manager and allows you to collect
data from data sources that you
configure or from monitoring solutions
that provide additional insights into
applications running on the virtual
machine.
Azure Monitor Logs Stores data about processes and Using Azure Monitor for VMs
dependencies on the agent. (preview) Map to understand
application components
VM Storage Azure Monitor for VMs uses the Log Understand the health of your Azure
Analytics agent to store heath state virtual machines
information in a custom location. This Azure Resource health REST API
is only available to Azure Monitor for
VMs in the Azure portal in addition to
the Azure Resource health REST API.
Application Code
Detailed application monitoring in Azure Monitor is done with Application Insights which collects data from
applications running on a variety of platforms. The application can be running in Azure, another cloud, or on-
premises.
Application data
When you enable Application Insights for an application by installing an instrumentation package, it collects
metrics and logs related to the performance and operation of the application. Application Insights stores the
data it collects in the same Azure Monitor data platform used by other data sources. It includes extensive tools
for analyzing this data, but you can also analyze it with data from other sources using tools such as Metrics
Explorer and Log Analytics.
Azure Monitor Logs Operational data about your Analyze log data in Azure Monitor
application including page views,
application requests, exceptions, and
traces.
Azure Monitor Metrics Application Insights collects metrics Log-based and pre-aggregated
describing the performance and metrics in Application Insights
operation of the application in addition Application Insights API for custom
to custom metrics that you define in events and metrics
your application into the Azure
Monitor metrics database.
Azure Storage Send application data to Azure Storage Export telemetry from Application
for archiving. Insights
Azure Monitor Logs Monitoring solutions collect data into Data collection details for monitoring
Azure Monitor logs where it may be solutions in Azure
analyzed using the query language or
views that are typically included in the
solution.
Azure Monitor Logs Stores monitoring data for AKS Understand AKS cluster performance
including inventory, logs, and events. with Azure Monitor for containers
Metric data is also stored in Logs in
order to leverage its analysis
functionality in the portal.
Azure Monitor Metrics Metric data is stored in the metric View container metrics in metrics
database to drive visualization and explorer
alerts.
Azure Kubernetes Service Provides direct access to your Azure How to view Kubernetes logs, events,
Kubernetes Service (AKS) container and pod metrics in real-time
logs (stdout/stderror), events, and pod
metrics in the portal.
Custom sources
In addition to the standard tiers of an application, you may need to monitor other resources that have telemetry
that can't be collected with the other data sources. For these resources, write this data to either Metrics or Logs
using an Azure Monitor API.
DESTINATION METHOD DESCRIPTION REFERENCE
Azure Monitor Logs Data Collector API Collect log data from any Send log data to Azure
REST client and store in Log Monitor with the HTTP
Analytics workspace. Data Collector API (public
preview)
Azure Monitor Metrics Custom Metrics API Collect metric data from Send custom metrics for an
any REST client and store in Azure resource to the Azure
Azure Monitor metrics Monitor metric store by
database. using a REST API
Other services
Other services in Azure write data to the Azure Monitor data platform. This allows you to analyze data collected
by these services with data collected by Azure Monitor and leverage the same analysis and visualization tools.
Azure Security Center Azure Monitor Logs Azure Security Center Data collection in Azure
stores the security data it Security Center
collects in a Log Analytics
workspace which allows it
to be analyzed with other
log data collected by Azure
Monitor.
Azure Sentinel Azure Monitor Logs Azure Sentinel stores the Connect data sources
data it collects from
different data sources in a
Log Analytics workspace
which allows it to be
analyzed with other log
data collected by Azure
Monitor.
Next steps
Learn more about the types of monitoring data collected by Azure Monitor and how to view and analyze
this data.
List the different locations where Azure resources store data and how you can access it.
Azure Monitor data platform
1/8/2020 • 6 minutes to read • Edit Online
Enabling observability across today's complex computing environments running distributed applications that
rely on both cloud and on-premises services, requires collection of operational data from every layer and
every component of the distributed system. You need to be able to perform deep insights on this data and
consolidate it into a single pane of glass with different perspectives to support the multitude of stakeholders
in your organization.
Azure Monitor collects and aggregates data from a variety of sources into a common data platform where it
can be used for analysis, visualization, and alerting. It provides a consistent experience on top of data from
multiple sources, which gives you deep insights across all your monitored resources and even with data from
other services that store their data in Azure Monitor.
NOTE
It's important to distinguish between Azure Monitor Logs and sources of log data in Azure. For example, subscription
level events in Azure are written to an activity log that you can view from the Azure Monitor menu. Most resources will
write operational information to a resource log that you can forward to different locations. Azure Monitor Logs is a log
data platform that collects activity logs and resource logs along with other monitoring data to provide deep analysis
across your entire set of resources.
You can work with log queries interactively with Log Analytics in the Azure portal or add the results to an
Azure dashboard for visualization in combination with other data. You can also create log alerts which will
trigger an alert based on the results of a schedule query.
Read more about Azure Monitor Logs including their sources of data in Logs in Azure Monitor.
Distributed traces
Traces are series of related events that follow a user request through a distributed system. They can be used to
determine behavior of application code and the performance of different transactions. While logs will often be
created by individual components of a distributed system, a trace measures the operation and performance of
your application across the entire set of components.
Distributed tracing in Azure Monitor is enabled with the Application Insights SDK, and trace data is stored
with other application log data collected by Application Insights. This makes it available to the same analysis
tools as other log data including log queries, dashboards, and alerts.
Read more about distributed tracing at What is Distributed Tracing?.
Benefits Lightweight and capable of near-real Analyzed with rich query language.
time scenarios such as alerting. Ideal Ideal for deep analysis and identifying
for fast detection of issues. root cause.
Data sources include Platform metrics collected from Azure Application and resource logs.
resources. Monitoring solutions.
Applications monitored by Application Agents and VM extensions.
Insights. Application requests and exceptions.
Custom defined by application or API. Azure Security Center.
Data Collector API.
Next steps
Read more about Metrics in Azure Monitor.
Read more about Logs in Azure Monitor.
Learn about the monitoring data available for different resources in Azure.
Monitoring data locations in Azure Monitor
10/25/2019 • 2 minutes to read • Edit Online
Azure Monitor is based on a data platform of Logs and Metrics as described in Azure Monitor data platform.
Monitoring data from Azure resources may be written to other locations though, either before they are copied to
Azure Monitor or to support additional scenarios.
Azure Monitor Logs Log Analytics workspace that's based Log Analytics
on Azure Data Explorer which provides Log Analytics API
a powerful analysis engine and rich Application Insights API
query language.
Activity log Data from the Activity log is most Azure portal
useful when sent to Azure Monitor Azure Monitor Events API
Logs to analyze it with other data, but
it's also collected on its own so it can be
directly viewed in the Azure portal.
Azure Storage Some data sources will write directly to Storage Analytics
Azure storage and require configuration Server Explorer
to move data into Logs. You can also Storage Explorer
send data to Azure storage for
archiving and for integration with
external systems.
Azure Monitor for VMs Azure Monitor for VMs stores workload Azure portal
health data in a custom location that's Workload monitor REST API
used by its monitoring experience in the Azure Resource health REST API
Azure portal.
Next steps
See the different sources of monitoring data collected by Azure Monitor.
Learn about the types of monitoring data collected by Azure Monitor and how to view and analyze this data.
Metrics in Azure Monitor
9/24/2019 • 5 minutes to read • Edit Online
NOTE
The Azure Monitor data platform is based on two fundamental data types: Metrics and Logs. This article describes
Metrics. Refer to Logs in Azure Monitor for a detailed description of logs and to Azure Monitor data platform for a
comparison of the two.
Metrics in Azure Monitor are lightweight and capable of supporting near real-time scenarios making them
particularly useful for alerting and fast detection of issues. This article describes how metrics are structured,
what you can do with them, and identifies different data sources that store data in metrics.
Multi-dimensional metrics
One of the challenges to metric data is that it often has limited information to provide context for collected
values. Azure Monitor addresses this challenge with multi-dimensional metrics. Dimensions of a metric are
name-value pairs that carry additional data to describe the metric value. For example, a metric Available disk
space could have a dimension called Drive with values C:, D:, which would allow viewing either available disk
space across all drives or for each drive individually.
The example below illustrates two datasets for a hypothetical metric called Network Throughput. The first
dataset has no dimensions. The second dataset shows the values with two dimensions, IP Address and
Direction:
Network Throughput
TIMESTAMP METRIC VALUE
This non-dimensional metric can only answer a basic question like "what was my network throughput at a
given time?”
Network Throughput + two dimensions ("IP" and "Direction")
TIMESTAMP DIMENSION "IP" DIMENSION "DIRECTION" METRIC VALUE
This metric can answer questions such as "what was the network throughput for each IP address?", and "how
much data was sent versus received?" Multi-dimensional metrics carry additional analytical and diagnostic
value compared to non-dimensional metrics.
Retention of Metrics
For most resources in Azure, metrics are stored for 93 days. There are some exceptions:
Guest OS metrics
Classic guest OS metrics. These are performance counters collected by the Windows Diagnostic
Extension (WAD ) or the Linux Diagnostic Extension (LAD ) and routed to an Azure storage account.
Retention for these metrics is 14 days.
Guest OS metrics sent to Azure Monitor Metrics. These are performance counters collected by the
Windows Diagnostic Extension (WAD ) and send to the Azure Monitor Sink, or via the InfluxData Telegraf
Agent on Linux machines. Retention for these metrics is 93 days.
Guest OS metrics collected by Log Analytics agent. These are performance counters collected by the
Log Analytics agent and sent to a Log Analytics workspace. Retention for these metrics is 31 days, and can
be extended up to 2 years.
Application Insights log-based metrics.
Behind the scene, log-based metrics translate into log queries. Their retention matches the retention of
events in underlying logs. For Application Insights resources, logs are stored for 90 days.
NOTE
You can send platform metrics for Azure Monitor resources to a Log Analytics workspace for long term trending.
Next steps
Learn more about the Azure Monitor data platform.
Learn about log data in Azure Monitor.
Learn about the monitoring data available for different resources in Azure.
Logs in Azure Monitor
1/8/2020 • 7 minutes to read • Edit Online
NOTE
All data collected by Azure Monitor fits into one of two fundamental types, Metrics and Logs. This article describes
Logs. Refer to Metrics in Azure Monitor for a detailed description of metrics and to Monitoring data collected by Azure
Monitor for a comparison of the two.
Logs in Azure Monitor are especially useful for performing complex analysis across data from a variety of
sources. This article describes how Logs are structured in Azure Monitor, what you can do with the data, and
identifies different data sources that store data in Logs.
NOTE
It's important to distinguish between Azure Monitor Logs and sources of log data in Azure. For example, subscription
level events in Azure are written to an activity log that you can view from the Azure Monitor menu. Most resources will
write operational information to a resource log that you can forward to different locations. Azure Monitor Logs is a log
data platform that collects activity logs and resource logs along with other monitoring data to provide deep analysis
across your entire set of resources.
Analyze Use Log Analytics in the Azure portal to write log queries
and interactively analyze log data using the powerful Data
Explorer analysis engine.
Use the Application Insights analytics console in the Azure
portal to write log queries and interactively analyze log
data from Application Insights.
Visualize Pin query results rendered as tables or charts to an Azure
dashboard.
Create a workbook to combine with multiple sets of data in
an interactive report.
Export the results of a query to Power BI to use different
visualizations and share with users outside of Azure.
Export the results of a query to Grafana to leverage its
dashboarding and combine with other data sources.
Retrieve Access log query results from a command line using Azure
CLI.
Access log query results from a command line using
PowerShell cmdlets.
Access log query results from a custom application using
REST API.
Open Log Analytics from Application Insights to analyze Application Insights data.
You can also retrieve log data by using the Log Analytics API and the Application Insights REST API.
Azure Active Directory audit logs Configured through Diagnostic settings for each directory.
See Integrate Azure AD logs with Azure Monitor logs.
Activity logs Stored separately by default and can be used for near real
time alerts. Install Activity log Analytics solution to write to
Log Analytics workspace. See Collect and analyze Azure
activity logs in Log Analytics.
Azure resources
DATA DESCRIPTION
Monitoring solutions Monitoring solutions write data they collect to their Log
Analytics workspace. See Data collection details for
management solutions in Azure for a list of solutions. See
Monitoring solutions in Azure Monitor for details on
installing and using solutions.
Azure table storage Collect data from Azure storage where some Azure
resources write monitoring data. See Use Azure blob
storage for IIS and Azure table storage for events with Log
Analytics.
Virtual Machines
DATA DESCRIPTION
Agent data sources Data sources collected from Windows and Linux agents
include events, performance data, and custom logs. See
Agent data sources in Azure Monitor for a list of data
sources and details on configuration.
Monitoring solutions Monitoring solutions write data they collect from agents to
their Log Analytics workspace. See Data collection details
for management solutions in Azure for a list of solutions.
See Monitoring solutions in Azure Monitor for details on
installing and using solutions.
System Center Operations Manager Connect Operations Manager management group to Azure
Monitor to collect event and performance data from on-
premises agents into logs. See Connect Operations
Manager to Log Analytics for details on this configuration.
Applications
DATA DESCRIPTION
Requests and exceptions Detailed data about application requests and exceptions
are in the requests, pageViews, and exceptions tables. Calls
to external components are in the dependencies table.
Usage and performance Performance for the application is available in the requests,
browserTimings and performanceCounters tables. Data for
custom metrics is in the customMetrics table.
Trace data Results from distributed tracing are stored in the traces
table.
Insights
DATA DESCRIPTION
Azure Monitor for containers Inventory and performance data collected by Azure
Monitor for containers. See Container data-collection
details for a list of the tables.
Azure Monitor for VMs Map and performance data collected by Azure Monitor for
VMs. See How to query logs from Azure Monitor for VMs
for details on querying this data.
Custom
DATA DESCRIPTION
REST API Write data to a Log Analytics workspace from any REST
client. See Send log data to Azure Monitor with the HTTP
Data Collector API for details.
Logic App Write any data to a Log Analytics workspace from a Logic
App workflow with the Azure Log Analytics Data
Collector action.
Security
DATA DESCRIPTION
Azure Security Center Azure Security Center stores data that it collects in a Log
Analytics workspace where it can be analyzed with other
log data. See Data collection in Azure Security Center for
details on workspace configuration.
Azure Sentinel Azure Sentinel stores data from data sources into a Log
Analytics workspace. See Connect data sources.
Next steps
Learn more about the Azure Monitor data platform.
Learn about metrics in Azure Monitor.
Learn about the monitoring data available for different resources in Azure.
Log data ingestion time in Azure Monitor
12/6/2019 • 7 minutes to read • Edit Online
Azure Monitor is a high scale data service that serves thousands of customers sending terabytes of data each
month at a growing pace. There are often questions about the time it takes for log data to become available after
it's collected. This article explains the different factors that affect this latency.
Typical latency
Latency refers to the time that data is created on the monitored system and the time that it comes available for
analysis in Azure Monitor. The typical latency to ingest log data is between 2 and 5 minutes. The specific latency
for any particular data will vary depending on a variety of factors explained below.
Heartbeat
| where TimeGenerated > ago(8h)
| extend E2EIngestionLatency = ingestion_time() - TimeGenerated
| extend AgentLatency = _TimeReceived - TimeGenerated
| summarize percentiles(E2EIngestionLatency,50,95), percentiles(AgentLatency,50,95) by Computer
| top 20 by percentile_E2EIngestionLatency_95 desc
The preceding percentile checks are good for finding general trends in latency. To identify a short-term spike in
latency, using the maximum ( max() ) might be more effective.
If you want to drill down on the ingestion time for a specific computer over a period of time, use the following
query, which also visualizes the data from the past day in a graph:
Heartbeat
| where TimeGenerated > ago(24h) //and Computer == "ContosoWeb2-Linux"
| extend E2EIngestionLatencyMin = todouble(datetime_diff("Second",ingestion_time(),TimeGenerated))/60
| extend AgentLatencyMin = todouble(datetime_diff("Second",_TimeReceived,TimeGenerated))/60
| summarize percentiles(E2EIngestionLatencyMin,50,95), percentiles(AgentLatencyMin,50,95) by
bin(TimeGenerated,30m)
| render timechart
Use the following query to show computer ingestion time by the country/region they are located in which is based
on their IP address:
Heartbeat
| where TimeGenerated > ago(8h)
| extend E2EIngestionLatency = ingestion_time() - TimeGenerated
| extend AgentLatency = _TimeReceived - TimeGenerated
| summarize percentiles(E2EIngestionLatency,50,95),percentiles(AgentLatency,50,95) by RemoteIPCountry
Different data types originating from the agent might have different ingestion latency time, so the previous
queries could be used with other types. Use the following query to examine the ingestion time of various Azure
services:
AzureDiagnostics
| where TimeGenerated > ago(8h)
| extend E2EIngestionLatency = ingestion_time() - TimeGenerated
| extend AgentLatency = _TimeReceived - TimeGenerated
| summarize percentiles(E2EIngestionLatency,50,95), percentiles(AgentLatency,50,95) by ResourceProvider
Heartbeat
| where TimeGenerated > ago(1d) //show only VMs that were active in the last day
| summarize NoHeartbeatPeriod = now() - max(TimeGenerated) by Computer
| top 20 by NoHeartbeatPeriod desc
Next steps
Read the Service Level Agreement (SLA) for Azure Monitor.
Overview of Insights in Azure Monitor
11/8/2019 • 2 minutes to read • Edit Online
Insights provide a customized monitoring experience for particular applications and services. They store data in
the Azure Monitor data platform and leverage other Azure Monitor features for analysis and alerting but may
collect additional data and provide a unique user experience in the Azure portal. Access insights from the
Insights section of the Azure Monitor menu in the Azure portal.
The following sections provide a brief description of the insights that are currently available in Azure Monitor.
See the detailed documentation for details on each.
Application Insights
Application Insights is an extensible Application Performance Management (APM ) service for web developers on
multiple platforms. Use it to monitor your live web application. It works for applications on a wide variety of
platforms including .NET, Node.js and Java EE, hosted on-premises, hybrid, or any public cloud. It also integrates
with your DevOps process and has connection points to a variety of development tools.
See What is Application Insights?.
Next steps
Learn more about the Azure Monitor data platform leveraged by insights.
Learn about the different data sources used by Azure Monitor and the different kinds of data collected by each
of the insights.
Inventory and data collection details for monitoring
solutions in Azure
12/13/2019 • 4 minutes to read • Edit Online
Monitoring solutions leverage services in Azure to provide additional insight into the operation of a particular
application or service. Monitoring solutions typically collect log data and provide queries and views to analyze
collected data. You can add monitoring solutions to Azure Monitor for any applications and services that you use.
They are typically available at no cost but collect data that could invoke usage charges.
This article includes a list of montioring solutions available from Microsoft with links to their detailed
documentation. It also provides information on their method and frequency of data collection into Azure Monitor.
You can use the information in this article to identify the different solutions available and to understand the data
flow and connection requirements for different monitoring solutions.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a
Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the
terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
AD Windows • • • 7 days
Assessment
AD Windows • • • 5 days
Replication
Status
Application Azure on
Insights notification
Connector
(Deprecate
d)
Azure Azure on
Application notification
Gateway
Analytics
Azure Azure on
Network notification
Security
Group
Analytics
(Deprecate
d)
Backup Azure on
notification
Next steps
Learn how to install and use monitoring solutions.
Learn how to create queries to analyze data collected by monitoring solutions.
Log Analytics data security
12/13/2019 • 13 minutes to read • Edit Online
This document is intended to provide information specific to Log Analytics, which is a feature of Azure Monitor, to
supplement the information on Azure Trust Center.
This article explains how data is collected, processed, and secured by Log Analytics. You can use agents to connect
to the web service, use System Center Operations Manager to collect operational data, or retrieve data from Azure
diagnostics for use by Log Analytics.
The Log Analytics service manages your cloud-based data securely by using the following methods:
Data segregation
Data retention
Physical security
Incident management
Compliance
Security standards certifications
Contact us with any questions, suggestions, or issues about any of the following information, including our security
policies at Azure support options.
Windows 8.0 - 10 Supported, and enabled by default. To confirm that you are still using the
default settings.
Windows Server 2012 - 2016 Supported, and enabled by default. To confirm that you are still using the
default settings
PLATFORM/LANGUAGE SUPPORT MORE INFORMATION
Windows 7 SP1 and Windows Server Supported, but not enabled by default. See the Transport Layer Security (TLS)
2008 R2 SP1 registry settings page for details on
how to enable.
Data segregation
After your data is ingested by the Log Analytics service, the data is kept logically separate on each component
throughout the service. All data is tagged per workspace. This tagging persists throughout the data lifecycle, and it
is enforced at each layer of the service. Your data is stored in a dedicated database in the storage cluster in the
region you have selected.
Data retention
Indexed log search data is stored and retained according to your pricing plan. For more information, see Log
Analytics Pricing.
As part of your subscription agreement, Microsoft will retain your data per the terms of the agreement. When
customer data is removed, no physical drives are destroyed.
The following table lists some of the available solutions and provides examples of the type of data they collect.
Log Management User-defined event logs, Windows Event Logs and/or IIS Logs
SQL and Active Directory Assessment WMI data, registry data, performance data, and SQL Server
dynamic management view results
Physical security
The Log Analytics service is managed by Microsoft personnel and all activities are logged and can be audited. Log
Analytics is operated as an Azure Service and meets all Azure Compliance and Security requirements. You can
view details about the physical security of Azure assets on page 18 of the Microsoft Azure Security Overview.
Physical access rights to secure areas are changed within one business day for anyone who no longer has
responsibility for the Log Analytics service, including transfer and termination. You can read about the global
physical infrastructure we use at Microsoft Datacenters.
Incident management
Log Analytics has an incident management process that all Microsoft services adhere to. To summarize, we:
Use a shared responsibility model where a portion of security responsibility belongs to Microsoft and a portion
belongs to the customer
Manage Azure security incidents:
Start an investigation upon detection of an incident
Assess the impact and severity of an incident by an on-call incident response team member. Based on
evidence, the assessment may or may not result in further escalation to the security response team.
Diagnose an incident by security response experts to conduct the technical or forensic investigation,
identify containment, mitigation, and workaround strategies. If the security team believes that customer
data may have become exposed to an unlawful or unauthorized individual, parallel execution of the
Customer Incident Notification process begins in parallel.
Stabilize and recover from the incident. The incident response team creates a recovery plan to mitigate
the issue. Crisis containment steps such as quarantining impacted systems may occur immediately and in
parallel with diagnosis. Longer term mitigations may be planned which occur after the immediate risk
has passed.
Close the incident and conduct a post-mortem. The incident response team creates a post-mortem that
outlines the details of the incident, with the intention to revise policies, procedures, and processes to
prevent a recurrence of the event.
Notify customers of security incidents:
Determine the scope of impacted customers and to provide anybody who is impacted as detailed a
notice as possible
Create a notice to provide customers with detailed enough information so that they can perform an
investigation on their end and meet any commitments they have made to their end users while not
unduly delaying the notification process.
Confirm and declare the incident, as necessary.
Notify customers with an incident notification without unreasonable delay and in accordance with any
legal or contractual commitment. Notifications of security incidents are delivered to one or more of a
customer's administrators by any means Microsoft selects, including via email.
Conduct team readiness and training:
Microsoft personnel are required to complete security and awareness training, which helps them to
identify and report suspected security issues.
Operators working on the Microsoft Azure service have addition training obligations surrounding their
access to sensitive systems hosting customer data.
Microsoft security response personnel receive specialized training for their roles
If loss of any customer data occurs, we notify each customer within one day. However, customer data loss has
never occurred with the service.
For more information about how Microsoft responds to security incidents, see Microsoft Azure Security Response
in the Cloud.
Compliance
The Log Analytics software development and service team's information security and governance program
supports its business requirements and adheres to laws and regulations as described at Microsoft Azure Trust
Center and Microsoft Trust Center Compliance. How Log Analytics establishes security requirements, identifies
security controls, manages, and monitors risks are also described there. Annually, we review polices, standards,
procedures, and guidelines.
Each development team member receives formal application security training. Internally, we use a version control
system for software development. Each software project is protected by the version control system.
Microsoft has a security and compliance team that oversees and assesses all services in Microsoft. Information
security officers make up the team and they are not associated with the engineering teams that develops Log
Analytics. The security officers have their own management chain and conduct independent assessments of
products and services to ensure security and compliance.
Microsoft's board of directors is notified by an annual report about all information security programs at Microsoft.
The Log Analytics software development and service team are actively working with the Microsoft Legal and
Compliance teams and other industry partners to acquire various certifications.
NOTE
In some certifications/attestations, Log Analytics is listed under its former name of Operational Insights.
Next steps
Learn how to collect data with Log Analytics for your Azure VMs following the Azure VM quickstart.
If you are looking to collect data from physical or virtual Windows or Linux computers in your environment,
see the Quickstart for Linux computers or Quickstart for Windows computers
Azure Monitor customer-managed key configuration
1/16/2020 • 16 minutes to read • Edit Online
This article provides background information and steps to configure Customer-Managed Keys (CMK) for your Log
Analytics workspaces and Application Insights components. Once configured, any data sent to your workspaces or
components is encrypted with your Azure Key Vault key.
We recommend you review [Limitations and constraints](#Limitations and constraints) below before configuration.
Disclaimers
Azure Monitor CMK is an early access feature and enabled for registered subscriptions.
The CMK deployment described in this article is delivered in production quality and supported as such
although it's an early access feature.
The CMK capability is delivered on a dedicated data-store-cluster, which is an Azure Data Explorer (ADX)
cluster and suitable for customers sending 1TB per day or more.
The CMK pricing model isn't available currently and it isn't covered in this article. A pricing model for
dedicated ADX cluster is expected in the second quarter of calendar year (CY ) 2020 and will apply to any
existing CMK deployments.
This article describes the CMK configuration for Log Analytics workspaces. CMK for Application Insights
components is also supported using this article while differences are listed in the Appendix.
NOTE
Log Analytics and Application Insights are using the same data-store platform and query engine. We are bringing these two
stores together via integration of Application Insights into Log Analytics to create a single unified logs store under Azure
Monitor. This change is planned for the second quarter of calendar year 2020. If you don’t have to deploy CMK for your
Application Insights data by then, we recommend waiting for the completion of the consolidation since such deployments will
be disrupted by the consolidation and you will have to re-configure CMK after the migration to Log Analytics workspace. The
1TB per day minimum applies at the cluster level and until the consolidation completes during second quarter, Application
Insights and Log Analytics require separate clusters.
IMPORTANT
Any API request must include a Bearer authorization token in the request header.
For example:
GET
https://management.azure.com/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Micros
oft.OperationalInsights/workspaces/<workspaceName>?api-version=2015-11-01-preview
Authorization: Bearer eyJ0eXAiO....
IMPORTANT
CMK capability is regional. Your Azure Key Vault, Storage Account, Cluster resource and associated Log Analytics workspaces
must be in the same region, but they can be in different subscriptions.
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-
name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2019-08-01-preview
Authorization: Bearer <token>
Content-type: application/json
{
"location": "<region-name>",
"properties": {
"clusterType": "LogAnalytics" //Should be "ApplicationInsights" for Application Insights CMK
},
"identity": {
"type": "systemAssigned"
}
}
Response
Identity is assigned to the Cluster resource at creation time.
{
"identity": {
"type": "SystemAssigned",
"tenantId": "tenant-id",
"principalId": "principle-id" //A GUID that was generated by the managed identity service
},
"properties": {
"provisioningState": "Succeeded",
"clusterType": "LogAnalytics",
"clusterId": "cluster-id" //A GUID that Log Analytics generates for the cluster
},
"id": "/subscriptions/subscription-id/resourceGroups/resource-group-
name/providers/Microsoft.OperationalInsights/clusters/cluster-name", //The cluster resource Id
"name": "cluster-name",
"type": "Microsoft.OperationalInsights/clusters",
"location": "region-name"
}
IMPORTANT
Copy and keep the "cluster-id" value since you will need it in next steps.
If you what to delete the Cluster resource for any reason -- for example, create it with a different name or
clusterType, use this API call:
DELETE
https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-
name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2019-08-01-preview
The Get permission is required to verify that your Key Vault is configured as recoverable to protect your key and
the access to your Azure Monitor data.
It takes a few minutes until the Cluster resource is propagated in Azure Resource Manager. When configuring this
Access Policy immediately after the Cluster resource creation, a transient error may occur. In this case, try again
after a few minutes.
Update Cluster resource with Key identifier details
This step applies following future key version updates in your Key Vault. Update the Cluster resource with Key Vault
Key identifier details, to allow Azure Monitor Storage to use the new key version. Select the current version of your
key in Azure Key Vault to get the Key identifier details.
Update the Cluster resource KeyVaultProperties with Key Identifier details.
Update
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-
name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2019-08-01-preview
Authorization: Bearer <token>
Content-type: application/json
{
"properties": {
"KeyVaultProperties": { //Key Vault key identifier details taken from Key identifier URI
KeyVaultUri: "https://<key-vault-name>.vault.azure.net",
KeyName: "<key-name>",
KeyVersion: "<current-version>"
},
},
"location":"<region-name>",
"identity": {
"type": "systemAssigned"
}
}
Response
{
"identity": {
"type": "SystemAssigned",
"tenantId": "tenant-id",
"principalId": "principle-id" //A GUID that was generated by the managed identity service
},
"properties": {
"KeyVaultProperties": { // Key Vault key identifier
KeyVaultUri: "https://key-vault-name.vault.azure.net",
KeyName: "key-name",
KeyVersion: "current-version"
},
"provisioningState": "Succeeded",
"clusterType": "LogAnalytics",
"clusterId": "cluster-id"
},
"id": "/subscriptions/subscription-id/resourceGroups/resource-group-
name/providers/Microsoft.OperationalInsights/clusters/cluster-name", //The cluster resource Id
"name": "cluster-name",
"type": "Microsoft.OperationalInsights/clusters",
"location": "region-name" //Example: Switzerland North
}
GET https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-
name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2019-08-01-preview
Authorization: Bearer <token>
Response
{
"identity": {
"type": "SystemAssigned",
"tenantId": "tenant-id",
"principalId": "principal-Id"
},
"properties": {
"KeyVaultProperties": { // Key Vault key identifier
KeyVaultUri: "https://key-vault-name.vault.azure.net",
KeyName: "key-name",
KeyVersion: "current-version"
},
"provisioningState": "Succeeded",
"clusterType": "LogAnalytics",
"clusterId": "cluster-id"
},
"id": "/subscriptions/subscription-id/resourceGroups/resource-group-
name/providers/Microsoft.OperationalInsights/clusters/cluster-name",
"name": "cluster-name",
"type": "Microsoft.OperationalInsights/clusters",
"location": "region-name"
}
PUT https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/<resource-group-
name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>?api-version=2015-11-01-preview
Authorization: Bearer <token>
Content-type: application/json
{
"properties": {
"source": "Azure",
"customerId": "<workspace-id>", //Available in Azure portal under Log Analytics workspace Overview
section
"features": {
"clusterDefinitionId": "<cluster-id>" //It's the "clusterId" value provided in the respond from the
previous step
}
},
"id": "/subscriptions/<subscription-id>/resourcegroups/<resource-group-
name>/providers/microsoft.operationalinsights/workspaces/<workspace-name>",
"name": "<workspace-name>",
"type": "Microsoft.OperationalInsights/workspaces",
"location": "<region-name>"
}
Response
{
"properties": {
"source": "Azure",
"customerId": "workspace-id",
"retentionInDays": value,
"features": {
"legacy": value,
"searchVersion": value,
"clusterDefinitionId": "cluster-id" //The id of the Cluster resource
},
"workspaceCapping": {
"dailyQuotaGb": value,
"quotaNextResetTime": "timeStamp",
"dataIngestionStatus": "RespectQuota"
}
},
"id": "/subscriptions/subscription-id/resourcegroups/resource-group-
name/providers/microsoft.operationalinsights/workspaces/workspace-name",
"name": "workspace-name",
"type": "Microsoft.OperationalInsights/workspaces",
"location": "region-name"
}
After the association, data that is sent to your workspaces is stored encrypted with your managed key.
GET https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-
name>/providers/Microsoft.OperationalInsights/clusters?api-version=2019-08-01-preview
Authorization: Bearer <token>
Response
{
"value": [
{
"identity": {
"type": "SystemAssigned",
"tenantId": "tenant-id",
"principalId": "principal-Id"
},
"properties": {
"KeyVaultProperties": { // Key Vault key identifier
KeyVaultUri: "https://{key-vault-name}.vault.azure.net",
KeyName: "key-name",
KeyVersion: "current-version"
},
"provisioningState": "Succeeded",
"clusterType": "LogAnalytics",
"clusterId": "cluster-id"
},
"id": "/subscriptions/subscription-id/resourcegroups/resource-group-
name/providers/microsoft.operationalinsights/workspaces/workspace-name",
"name": "cluster-name",
"type": "Microsoft.OperationalInsights/clusters",
"location": "region-name"
}
]
}
Use this API call to Get all Cluster resources for a subscription:
GET https://management.azure.com/subscriptions/<subscription-
id>/providers/Microsoft.OperationalInsights/clusters?api-version=2019-08-01-preview
Authorization: Bearer <token>
Response
The same response as for 'Cluster resources for a resource group', but in subscription scope.
Use this API call to delete a Cluster resource -- You need to delete all the associated workspaces before you
can delete your Cluster resource:
DELETE
https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-
name>/providers/Microsoft.OperationalInsights/clusters/<cluster-name>?api-version=2019-08-01-preview
Authorization: Bearer <token>
Response
200 OK
Appendix
Application Insights Customer Managed Key (CMK) is supported as well, though you should consider the
following change to help you plan the deployment of CMK for your Application Insight components.
Log Analytics and Application Insights are using the same data-store platform and query engine. We are bringing
these two stores together via integration of Application Insights into Log Analytics to provide a single unified logs
store under Azure Monitor by the second quarter of 2020. This change will bring your Application Insight data into
Log Analytics workspaces and make queries, insights, and other improvements possible while the configuration of
CMK on your workspace, will also apply to your Application Insights data.
NOTE
If you don’t have to deploy CMK for your Application Insight data before the integration, we recommend waiting with
Application Insights CMK since such deployments will be disrupted by the integration and you will have to re-configure CMK
after the migration to Log Analytics workspace. The 1TB per day minimum applies at the cluster level and until the
consolidation completes during second quarter, Application Insights and Log Analytics require separate clusters.
{
"location": "<region-name>",
"properties": {
"clusterType":"ApplicationInsights"
},
"identity": {
"type": "systemAssigned"
}
}
Response
Identity is assigned to the Cluster resource at creation time.
{
"identity": {
"type": "SystemAssigned",
"tenantId": "tenant-id",
"principalId": "principle-id" //A GUID that was generated by the managed identity service
},
"properties": {
"provisioningState": "Succeeded",
"clusterType": "ApplicationInsights", //The value is ‘ApplicationInsights’ for Application Insights CMK
"clusterId": "cluster-id" //A GUID that Log Analytics generates for the cluster - copy it since you need it
for Key Vault and components association
},
"id": "/subscriptions/subscription-id/resourceGroups/resource-group-
name/providers/Microsoft.OperationalInsights/clusters/cluster-name", //The cluster resource Id
"name": "cluster-name",
"type": "Microsoft.OperationalInsights/clusters",
"location": "region-name"
}
PUT https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<resource-group-
name>/providers/Microsoft.Insights/components/<component-name>?api-version=2015-05-01
Authorization: Bearer <token>
Content-type: application/json
{
"properties": {
"clusterDefinitionId": "cluster-id" //It's the "clusterId" value provided in the respond from the
previous step
},
"location": "<region-name>",
"kind": "<component-type>", //Example: web
}
Response
{
"id": "/subscriptions/subscription-id/resourcegroups/resource-group-
name/providers/microsoft.insights/components/component-name",
"name": "component-name",
"type": "Microsoft.Insights/components",
"location": "region-name",
"tags": "",
"kind": "",
"properties": {
"clusterDefinitionId": "cluster-id" //The Cluster resource ID that is associated to this component
"ApplicationId": "",
"AppId": "",
"Application_Type": "",
"Flow_Type": "",
"Request_Source": "",
"InstrumentationKey": "",
"CreationDate": "",
"TenantId": "",
"HockeyAppId": "",
"HockeyAppToken": "",
"provisioningState": "",
"SamplingPercentage":,
"RetentionInDays":,
"ConnectionString": "",
"DisableIpMasking":,
"ImmediatePurgeDataOn30Days":
}
}
After the association, data that is sent to your components is stored encrypted with your managed key.
Guidance for personal data stored in Log Analytics
and Application Insights
12/13/2019 • 8 minutes to read • Edit Online
Log Analytics is a data store where personal data is likely to be found. Application Insights stores its data in a Log
Analytics partition. This article will discuss where in Log Analytics and Application Insights such data is typically
found, as well as the capabilities available to you to handle such data.
NOTE
For the purposes of this article log data refers to data sent to a Log Analytics workspace, while application data refers to
data collected by Application Insights.
NOTE
For information about viewing or deleting personal data, see Azure Data Subject Requests for the GDPR. For more
information about GDPR, see the GDPR section of the Service Trust portal.
User IDs: User IDs are found in a large variety of solutions and tables. You can look for a particular username
across your entire dataset using the search command:
Remember to look not only for human-readable user names but also GUIDs that can directly be traced back to
a particular user!
Device IDs: Like user IDs, device IDs are sometimes considered "private". Use the same approach as listed
above for user IDs to identify tables where this might be a concern.
Custom data: Log Analytics allows the collection in a variety of methods: custom logs and custom fields, the
HTTP Data Collector API , and custom data collected as part of system event logs. All of these are susceptible to
containing private data, and should be examined to verify whether any such data exists.
Solution-captured data: Because the solution mechanism is an open-ended one, we recommend reviewing all
tables generated by solutions to ensure compliance.
Application data
IP addresses: While Application Insights will by default obfuscate all IP address fields to "0.0.0.0", it is a fairly
common pattern to override this value with the actual user IP to maintain session information. The Analytics
query below can be used to find any table that contains values in the IP address column other than "0.0.0.0"
over the last 24 hours:
User IDs: By default, Application Insights will use randomly generated IDs for user and session tracking.
However, it is common to see these fields overridden to store an ID more relevant to the application. For
example: usernames, AAD GUIDs, etc. These IDs are often considered to be in-scope as personal data, and
therefore, should be handled appropriately. Our recommendation is always to attempt to obfuscate or
anonymize these IDs. Fields where these values are commonly found include session_Id, user_Id,
user_AuthenticatedId, user_AccountId, as well as customDimensions.
Custom data: Application Insights allows you to append a set of custom dimensions to any data type. These
dimensions can be any data. Use the following query to identify any custom dimensions collected over the last
24 hours:
search *
| where isnotempty(customDimensions)
| where timestamp > ago(1d)
| project $table, timestamp, name, customDimensions
In-memory and in-transit data: Application Insights will track exceptions, requests, dependency calls, and traces.
Private data can often be collected at the code and HTTP call level. Review the exceptions, requests,
dependencies, and traces tables to identify any such data. Use telemetry initializers where possible to obfuscate
this data.
Snapshot Debugger captures: The Snapshot Debugger feature in Application Insights allows you to collect
debug snapshots whenever an exception is caught on the production instance of your application. Snapshots
will expose the full stack trace leading to the exceptions as well as the values for local variables at every step in
the stack. Unfortunately, this feature does not allow for selective deletion of snap points, or programmatic
access to data within the snapshot. Therefore, if the default snapshot retention rate does not satisfy your
compliance requirements, the recommendation is to turn off the feature.
NOTE
This article provides steps for how to delete personal data from the device or service and can be used to support your
obligations under the GDPR. If you’re looking for general info about GDPR, see the GDPR section of the Service Trust portal.
IMPORTANT
While the vast majority of purge operations may complete much quicker than the SLA, the formal SLA for the completion
of purge operations is set at 30 days due to their heavy impact on the data platform used. This is an automated process;
there is no way to request that an operation be handled faster.
Delete
WARNING
Deletes in Log Analytics are destructive and non-reversible! Please use extreme caution in their execution.
We have made available as part of a privacy handling a purge API path. This path should be used sparingly due to
the risk associated with doing so, the potential performance impact, and the potential to skew all-up aggregations,
measurements, and other aspects of your Log Analytics data. See the Strategy for personal data handling section
for alternative approaches to handle private data.
Purge is a highly privileged operation that no app or user in Azure (including even the resource owner) will have
permissions to execute without explicitly being granted a role in Azure Resource Manager. This role is Data Purger
and should be cautiously delegated due to the potential for data loss.
IMPORTANT
In order to manage system resources, purge requests are throttled at 50 requests per hour. You should batch the execution
of purge requests by sending a single command whose predicate includes all user identities that require purging. Use the in
operator to specify multiple identities. You should run the query before executing the purge request to verify that the results
are expected.
Once the Azure Resource Manager role has been assigned, two new API paths are available:
Log data
POST purge - takes an object specifying parameters of data to delete and returns a reference GUID
GET purge status - the POST purge call will return an 'x-ms-status-location' header that will include a URL
that you can call to determine the status of your purge API. For example:
x-ms-status-location:
https://management.azure.com/subscriptions/[SubscriptionId]/resourceGroups/[ResourceGroupName]/providers
/Microsoft.OperationalInsights/workspaces/[WorkspaceName]/operations/purge-[PurgeOperationId]?api-
version=2015-03-20
IMPORTANT
While we expect the vast majority of purge operations to complete much quicker than our SLA, due to their heavy impact on
the data platform used by Log Analytics, the formal SLA for the completion of purge operations is set at 30 days.
Application data
POST purge - takes an object specifying parameters of data to delete and returns a reference GUID
GET purge status - the POST purge call will return an 'x-ms-status-location' header that will include a URL
that you can call to determine the status of your purge API. For example:
x-ms-status-location:
https://management.azure.com/subscriptions/[SubscriptionId]/resourceGroups/[ResourceGroupName]/providers
/microsoft.insights/components/[ComponentName]/operations/purge-[PurgeOperationId]?api-version=2015-05-
01
IMPORTANT
While the vast majority of purge operations may complete much quicker than the SLA, due to their heavy impact on the data
platform used by Application Insights, the formal SLA for the completion of purge operations is set at 30 days.
Next steps
To learn more about how Log Analytics data is collected, processed, and secured, see Log Analytics data
security.
To learn more about how Application Insights data is collected, processed, and secured, see Application Insights
data security.
Data collection, retention, and storage in
Application Insights
1/22/2020 • 14 minutes to read • Edit Online
When you install Azure Application Insights SDK in your app, it sends telemetry about your app to the Cloud.
Naturally, responsible developers want to know exactly what data is sent, what happens to the data, and how
they can keep control of it. In particular, could sensitive data be sent, where is it stored, and how secure is it?
First, the short answer:
The standard telemetry modules that run "out of the box" are unlikely to send sensitive data to the service.
The telemetry is concerned with load, performance and usage metrics, exception reports, and other
diagnostic data. The main user data visible in the diagnostic reports are URLs; but your app shouldn't in any
case put sensitive data in plain text in a URL.
You can write code that sends additional custom telemetry to help you with diagnostics and monitoring
usage. (This extensibility is a great feature of Application Insights.) It would be possible, by mistake, to write
this code so that it includes personal and other sensitive data. If your application works with such data, you
should apply a thorough review process to all the code you write.
While developing and testing your app, it's easy to inspect what's being sent by the SDK. The data appears in
the debugging output windows of the IDE and browser.
The data is held in Microsoft Azure servers in the USA or Europe. (But your app can run anywhere.) Azure
has strong security processes and meets a broad range of compliance standards. Only you and your
designated team have access to your data. Microsoft staff can have restricted access to it only under specific
limited circumstances with your knowledge. It's encrypted in transit and at rest.
Review the collected data, as this may include data that is allowed in some circumstances but not others. A
good example of this is Device Name. The device name from a server has no privacy impact and is useful, but
a device name from a phone or laptop may have a privacy impact and be less useful. An SDK developed
primarily to target servers, would collect device name by default, and this may need to be overwritten in both
normal events and exceptions.
The rest of this article elaborates more fully on these answers. It's designed to be self-contained, so that you can
show it to colleagues who aren't part of your immediate team.
<TelemetryChannel Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel,
Microsoft.AI.ServerTelemetryChannel">
<StorageFolder>D:\NewTestFolder</StorageFolder>
</TelemetryChannel>
Via code:
Remove ServerTelemetryChannel from configuration file
Add this snippet to your configuration:
NetCore
By default ServerTelemetryChannel uses the current user’s local app data folder
%localAppData%\Microsoft\ApplicationInsights or temp folder %TMP% . ( See implementation here.) In a Linux
environment, local storage will be disabled unless a storage folder is specified.
The following code snippet shows how to set ServerTelemetryChannel.StorageFolder in the ConfigureServices()
method of your Startup.cs class:
Azure App Services Supported, configuration may be Support was announced in April 2018.
required. Read the announcement for
configuration details.
Azure Function Apps Supported, configuration may be Support was announced in April 2018.
required. Read the announcement for
configuration details.
.NET Supported, configuration varies by For detailed configuration info for .NET
version. 4.7 and earlier versions, refer to these
instructions.
Java Supported, JDK support for TLS 1.2 JDK 8 uses TLS 1.2 by default.
was added in JDK 6 update 121 and
JDK 7.
Windows 8.0 - 10 Supported, and enabled by default. To confirm that you are still using the
default settings.
Windows Server 2012 - 2016 Supported, and enabled by default. To confirm that you are still using the
default settings
Windows 7 SP1 and Windows Server Supported, but not enabled by default. See the Transport Layer Security (TLS)
2008 R2 SP1 registry settings page for details on
how to enable.
Windows Server 2008 SP2 Support for TLS 1.2 requires an update. See Update to add support for TLS 1.2
in Windows Server 2008 SP2.
openssl version -a
Session session id
ServerContext Machine name, locale, OS, device, user session, user context,
operation
Availability Web test response code, duration of each test step, test
name, timestamp, success, response time, test location
NOTE
Client IP is used to infer geographic location, but by default IP data is no longer stored and all zeroes are written to the
associated field. To understand more about personal data handling we recommend this article. If you need to store IP
address data our IP address collection article will walk you through your options.
Credits
This product includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com.
Overview of alerts in Microsoft Azure
1/9/2020 • 10 minutes to read • Edit Online
This article describes what alerts are, their benefits, and how to get started using them.
Overview
The diagram below represents the flow of alerts.
Alert Rule
Target Resource
Signal
Criteria /
Logic Test
Alert rules are separated from alerts and the actions taken when an alert fires. The alert rule captures the
target and criteria for alerting. The alert rule can be in an enabled or a disabled state. Alerts only fire when
enabled.
The following are key attributes of an alert rule:
Target Resource: Defines the scope and signals available for alerting. A target can be any Azure resource.
Example targets: a virtual machine, a storage account, a virtual machine scale set, a Log Analytics workspace,
or an Application Insights resource. For certain resources (like virtual machines), you can specify multiple
resources as the target of the alert rule.
Signal: Emitted by the target resource. Signals can be of the following types: metric, activity log, Application
Insights, and log.
Criteria: A combination of signal and logic applied on a target resource. Examples:
Percentage CPU > 70%
Server Response Time > 4 ms
Result count of a log query > 100
Alert Name: A specific name for the alert rule configured by the user.
Alert Description: A description for the alert rule configured by the user.
Severity: The severity of the alert after the criteria specified in the alert rule is met. Severity can range from
0 to 4.
Sev 0 = Critical
Sev 1 = Error
Sev 2 = Warning
Sev 3 = Informational
Sev 4 = Verbose
Action: A specific action taken when the alert is fired. For more information, see Action Groups.
Application Insights Web availability tests Not supported. See Web test alerts.
Available to any website that's
instrumented to send data to
Application Insights. Receive a
notification when availability or
responsiveness of a website is below
expectations.
Manage alerts
You can set the state of an alert to specify where it is in the resolution process. When the criteria specified in
the alert rule is met, an alert is created or fired, and it has a status of New. You can change the status when
you acknowledge an alert and when you close it. All state changes are stored in the history of the alert.
The following alert states are supported.
STATE DESCRIPTION
New The issue has just been detected and hasn't yet been
reviewed.
Closed The issue has been resolved. After an alert has been
closed, you can reopen it by changing it to another state.
Alert state is different and independent of the monitor condition. Alert state is set by the user. Monitor
condition is set by the system. When an alert fires, the alert's monitor condition is set to fired. When the
underlying condition that caused the alert to fire clears, the monitor condition is set to resolved. The alert
state isn't changed until the user changes it. Learn how to change the state of your alerts and smart groups.
Smart groups
Smart groups are aggregations of alerts based on machine learning algorithms, which can help reduce alert
noise and aid in troubleshooting. Learn more about Smart Groups and how to manage your smart groups.
Alerts experience
The default Alerts page provides a summary of alerts that are created within a particular time range. It
displays the total alerts for each severity, with columns that identify the total number of alerts in each state
for each severity. Select any of the severities to open the All Alerts page filtered by that severity.
Alternatively, you can programmatically enumerate the alert instances generated on your subscriptions by
using REST APIs.
NOTE
You can only access alerts generated in the last 30 days.
It doesn't show or track classic alerts. You can change the subscriptions or filter parameters to update the
page.
You can filter this view by selecting values in the drop-down menus at the top of the page.
COLUMN DESCRIPTION
Subscription Select the Azure subscriptions for which you want to view
the alerts. You can optionally choose to select all your
subscriptions. Only alerts that you have access to in the
selected subscriptions are included in the view.
Resource group Select a single resource group. Only alerts with targets in
the selected resource group are included in the view.
Time range Only alerts fired within the selected time range are
included in the view. Supported values are the past hour,
the past 24 hours, the past 7 days, and the past 30 days.
Select the following values at the top of the Alerts page to open another page:
VALUE DESCRIPTION
Total alerts The total number of alerts that match the selected criteria.
Select this value to open the All Alerts view with no filter.
Smart groups The total number of smart groups that were created from
the alerts that match the selected criteria. Select this value
to open the smart groups list in the All Alerts view.
Total alert rules The total number of alert rules in the selected subscription
and resource group. Select this value to open the Rules
view filtered on the selected subscription and resource
group.
You can filter the view by selecting the following values in the drop-down menus at the top of the page:
COLUMN DESCRIPTION
Subscription Select the Azure subscriptions for which you want to view
the alerts. You can optionally choose to select all your
subscriptions. Only alerts that you have access to in the
selected subscriptions are included in the view.
Resource group Select a single resource group. Only alerts with targets in
the selected resource group are included in the view.
Resource type Select one or more resource types. Only alerts with targets
of the selected type are included in the view. This column is
only available after a resource group has been specified.
COLUMN DESCRIPTION
Alert state Select an alert state, or select All to include alerts of all
states.
Monitor service Select a service, or select All to include all services. Only
alerts created by rules that use service as a target are
included.
Time range Only alerts fired within the selected time range are
included in the view. Supported values are the past hour,
the past 24 hours, the past 7 days, and the past 30 days.
Select Columns at the top of the page to select which columns to show.
SECTION DESCRIPTION
History Lists each action taken by the alert and any changes made
to the alert. Currently limited to state changes.
SECTION DESCRIPTION
{
"subscriptions": [
<subscriptionId>
],
"query": "AlertsManagementResources | where type =~ 'Microsoft.AlertsManagement/alerts' | summarize
count()"
}
You can also see the result of this Resource Graph query in the portal with Azure Resource Graph Explorer:
portal.azure.com
You can query the alerts for their essential fields.
Use the Alert Management REST API to get more information about specific alerts, including their alert
context fields.
Next steps
Learn more about Smart Groups
Learn about action groups
Managing your alert instances in Azure
Managing Smart Groups
Learn more about Azure alerts pricing
Understand how metric alerts work in Azure Monitor
12/5/2019 • 7 minutes to read • Edit Online
Metric alerts in Azure Monitor work on top of multi-dimensional metrics. These metrics could be platform metrics,
custom metrics, popular logs from Azure Monitor converted to metrics and Application Insights metrics. Metric
alerts evaluate at regular intervals to check if conditions on one or more metric time-series are true and notify you
when the evaluations are met. Metric alerts are stateful, that is, they only send out notifications when the state
changes.
Typical latency
For metric alerts, typically you will get notified in under 5 minutes if you set the alert rule frequency to be 1 min. In
cases of heavy load for notification systems, you might see a longer latency.
Next steps
Learn how to create, view, and manage metric alerts in Azure
Learn how to deploy metric alerts using Azure Resource Manager templates
Learn more about action groups
Learn more about Dynamic Thresholds condition type
Log alerts in Azure Monitor
12/23/2019 • 13 minutes to read • Edit Online
This article provides details of Log alerts are one of the types of alerts supported within the Azure Alerts and
allow users to use Azure's analytics platform as basis for alerting.
Log Alert consists of Log Search rules created for Azure Monitor Logs or Application Insights. To learn more
about its usage, see creating log alerts in Azure
NOTE
Popular log data from Azure Monitor Logs is now also available on the metric platform in Azure Monitor. For details view,
Metric Alert for Logs
IMPORTANT
cross-resource query support in log alerts for Application Insights and log alerts for Log Analytics configured using
scheduledQueryRules API only.
Some analytic commands and combinations are incompatible with use in log alerts; for more details view,
Log alert queries in Azure Monitor.
Time Period. Specifies the time range for the query. The query returns only records that were created
within this range of the current time. Time period restricts the data fetched for log query to prevent abuse
and circumvents any time command (like ago) used in log query.
For example, If the time period is set to 60 minutes, and the query is run at 1:15 PM, only records created
between 12:15 PM and 1:15 PM is returned to execute log query. Now if the log query uses time command
like ago (7d ), the log query would be run only for data between 12:15 PM and 1:15 PM - as if data exists
for only the past 60 minutes. And not for seven days of data as specified in log query.
Frequency. Specifies how often the query should be run. Can be any value between 5 minutes and 24
hours. Should be equal to or less than the time period. If the value is greater than the time period, then
you risk records being missed.
For example, consider a time period of 30 minutes and a frequency of 60 minutes. If the query is run at
1:00, it returns records between 12:30 and 1:00 PM. The next time the query would run is 2:00 when it
would return records between 1:30 and 2:00. Any records created between 1:00 and 1:30 would never be
evaluated.
Threshold. The results of the log search are evaluated to determine whether an alert should be created.
The threshold is different for the different types of log search alert rules.
Log search rules be it for Azure Monitor Logs or Application Insights, can be of two types. Each of these types is
described in detail in the sections that follow.
Number of results. Single alert created when the number records returned by the log search exceed a
specified number.
Metric measurement. Alert created for each object in the results of the log search with values that exceed
specified threshold.
The differences between alert rule types are as follows.
Number of results alert rules always creates a single alert, while Metric measurement alert rule creates an
alert for each object that exceeds the threshold.
Number of results alert rules create an alert when the threshold is exceeded a single time. Metric
measurement alert rules can create an alert when the threshold is exceeded a certain number of times over a
particular time interval.
Number of results alert rules
Number of results alert rules create a single alert when the number of records returned by the search query
exceed the specified threshold. This type of alert rule is ideal for working with events such as Windows event
logs, Syslog, WebApp Response, and Custom logs. You may want to create an alert when a particular error event
gets created, or when multiple error events are created within a particular time period.
Threshold: The threshold for a Number of results alert rules is greater than or less than a particular value. If the
number of records returned by the log search match this criteria, then an alert is created.
To alert on a single event, set the number of results to greater than 0 and check for the occurrence of a single
event that was created since the last time the query was run. Some applications may log an occasional error that
shouldn't necessarily raise an alert. For example, the application may retry the process that created the error
event and then succeed the next time. In this case, you may not want to create the alert unless multiple events are
created within a particular time period.
In some cases, you may want to create an alert in the absence of an event. For example, a process may log
regular events to indicate that it's working properly. If it doesn't log one of these events within a particular time
period, then an alert should be created. In this case, you would set the threshold to less than 1.
Example of Number of Records type log alert
Consider a scenario where you want to know when your web-based App gives a response to users with code 500
(that is) Internal Server Error. You would create an alert rule with the following details:
Query: requests | where resultCode == "500"
Time period: 30 minutes
Alert frequency: five minutes
Threshold value: Greater than 0
Then alert would run the query every 5 minutes, with 30 minutes of data - to look for any record where result
code was 500. If even one such record is found, it fires the alert and triggers the action configured.
Metric measurement alert rules
Metric measurement alert rules create an alert for each object in a query with a value that exceeds a specified
threshold and specified trigger condition. Unlike Number of results alert rules, Metric measurement alert
rules work when analytics result provides a time series. They have the following distinct differences from
Number of results alert rules.
Aggregate function: Determines the calculation that is performed and potentially a numeric field to
aggregate. For example, count() returns the number of records in the query, avg(CounterValue) returns
the average of the CounterValue field over the interval. Aggregate function in query must be
named/called: AggregatedValue and provide a numeric value.
Group Field: A record with an aggregated value is created for each instance of this field, and an alert can
be generated for each. For example, if you wanted to generate an alert for each computer, you would use
by Computer. In case, there are multiple group fields specified in alert query, user can specify which field
to be used to sort results by using the Aggregate On (metricColumn) parameter
NOTE
Aggregate On (metricColumn) option is available for Metric Measurement type log alerts for Application Insights
and log alerts for Log Analytics configured using scheduledQueryRules API only.
Interval: Defines the time interval over which the data is aggregated. For example, if you specified five
minutes, a record would be created for each instance of the group field aggregated at 5-minute intervals
over the time period specified for the alert.
NOTE
Bin function must be used in query to specify interval. As bin() can result in unequal time intervals - Alert will
automatically convert bin command to bin_at command with appropriate time at runtime, to ensure results with a
fixed point. Metric measurement type of log alert is designed to work with queries having up to three instances of
bin() command
Threshold: The threshold for Metric measurement alert rules is defined by an aggregate value and a
number of breaches. If any data point in the log search exceeds this value, it's considered a breach. If the
number of breaches in for any object in the results exceeds the specified value, then an alert is created for
that object.
Misconfiguration of the Aggregate On or metricColumn option can cause alert rules to misfire. For more
information, see troubleshooting when metric measurement alert rule is incorrect.
Example of Metric Measurement type log alert
Consider a scenario where you wanted an alert if any computer exceeded processor utilization of 90% three
times over 30 minutes. You would create an alert rule with the following details:
Query: Perf | where ObjectName == "Processor" and CounterName == "% Processor Time" | summarize
AggregatedValue = avg(CounterValue) by bin(TimeGenerated, 5m), Computer
Time period: 30 minutes
Alert frequency: five minutes
Alert Logic - Condition & Threshold: Greater than 90
Group Field (Aggregate-on): Computer
Trigger alert based on: Total breaches Greater than 2
The query would create an average value for each computer at 5-minute intervals. This query would be run
every 5 minutes for data collected over the previous 30 minutes. Since the Group Field (Aggregate-on) chosen is
columnar 'Computer' - the AggregatedValue is split for various values of 'Computer' and average processor
utilization for each computer is determined for a time bin of 5 minutes. Sample query result for (say) three
computers, would be as below.
TIMEGENERATED [UTC] COMPUTER AGGREGATEDVALUE
20xx-xx-xxT01:00:00Z srv01.contoso.com 72
20xx-xx-xxT01:00:00Z srv02.contoso.com 91
20xx-xx-xxT01:00:00Z srv03.contoso.com 83
20xx-xx-xxT01:30:00Z srv01.contoso.com 88
20xx-xx-xxT01:30:00Z srv02.contoso.com 84
20xx-xx-xxT01:30:00Z srv03.contoso.com 92
In this example, we see in bins of 5 mins for each of the three computers - average processor utilization as
computed for 5 mins. Threshold of 90 being breached by srv01 only once at 1:25 bin. In comparison, srv02
exceeds 90 threshold at 1:10, 1:15 and 1:25 bins; while srv03 exceeds 90 threshold at 1:10, 1:15, 1:20 and 1:30.
Since alert is configured to trigger based on total breaches are more than two, we see that srv02 and srv03 only
meet the criteria. Hence separate alerts would be created for srv02 and srv03 since they breached the 90%
threshold twice across multiple time bins. If the Trigger alert based on: parameter was instead configured for
Continuous breaches option, then an alert would be fired only for srv03 since it breached the threshold for three
consecutive time bins from 1:10 to 1:20. And not for srv02, as it breached the threshold for two consecutive time
bins from 1:10 to 1:15.
NOTE
If invalid characters such as <, >, %, &, \, ?, / are present, they will be replaced with _ in the hidden pseudo alert
rule name and hence also in the Azure bill.
To remove the hidden scheduleQueryRules resources created for billing of alert rules using legacy Log Analytics
API, user can do any of the following:
Either user can switch API preference for the alert rules on the Log Analytics workspace and with no loss of
their alert rules or monitoring move to Azure Resource Manager compliant scheduledQueryRules API.
Thereby eliminate the need to make pseudo hidden alert rules for billing.
Or if the user doesn't want to switch API preference, the user will need to delete the original schedule and
alert action using legacy Log Analytics API or delete in Azure portal the original log alert rule
Additionally for the hidden scheduleQueryRules resources created for billing of alert rules using legacy Log
Analytics API, any modification operation like PUT will fail. As the microsoft.insights/scheduledqueryrules type
pseudo rules are for purpose of billing the alert rules created using legacy Log Analytics API. Any alert rule
modification should be done using legacy Log Analytics API (or) user can switch API preference for the alert
rules to use scheduledQueryRules API instead.
Next steps
Learn about creating in log alerts in Azure.
Understand webhooks in log alerts in Azure.
Learn about Azure Alerts.
Learn more about Application Insights.
Learn more about Log Analytics.
Alerts on activity log
1/8/2020 • 3 minutes to read • Edit Online
Overview
Activity log alerts are alerts that activate when a new activity log event occurs that matches the conditions
specified in the alert. Based on the order and volume of the events recorded in Azure activity log, the alert rule will
fire. Activity log alert rules are Azure resources, so they can be created by using an Azure Resource Manager
template. They also can be created, updated, or deleted in the Azure portal. This article introduces the concepts
behind activity log alerts. For more information on creating or usage of activity log alert rules, see Create and
manage activity log alerts.
NOTE
Alerts cannot be created for events in Alert category of activity log.
NOTE
In a subscription up to 100 alert rules can be created for an activity of scope at either: a single resource, all resources in
resource group (or) entire subscription level.
When an activity log alert is activated, it uses an action group to generate actions or notifications. An action group
is a reusable set of notification receivers, such as email addresses, webhook URLs, or SMS phone numbers. The
receivers can be referenced from multiple alerts to centralize and group your notification channels. When you
define your activity log alert, you have two options. You can:
Use an existing action group in your activity log alert.
Create a new action group.
To learn more about action groups, see Create and manage action groups in the Azure portal.
Next steps
Get an overview of alerts.
Learn about create and modify activity log alerts.
Review the activity log alert webhook schema.
Learn about service health notifications.
Use Application Change Analysis (preview) in Azure
Monitor
12/1/2019 • 5 minutes to read • Edit Online
When a live site issue or outage occurs, quickly determining the root cause is critical. Standard monitoring
solutions might alert you to a problem. They might even indicate which component is failing. But this alert won't
always immediately explain the failure's cause. You know your site worked five minutes ago, and now it's broken.
What changed in the last five minutes? This is the question that Application Change Analysis is designed to answer
in Azure Monitor.
Building on the power of Azure Resource Graph, Change Analysis provides insights into your Azure application
changes to increase observability and reduce MTTR (mean time to repair).
IMPORTANT
Change Analysis is currently in preview. This preview version is provided without a service-level agreement. This version is not
recommended for production workloads. Some features might not be supported or might have constrained capabilities. For
more information, see Supplemental terms of use for Microsoft Azure previews.
Overview
Change Analysis detects various types of changes, from the infrastructure layer all the way to application
deployment. It's a subscription-level Azure resource provider that checks resource changes in the subscription.
Change Analysis provides data for various diagnostic tools to help users understand what changes might have
caused issues.
The following diagram illustrates the architecture of Change Analysis:
Currently Change Analysis is integrated into the Diagnose and solve problems experience in the App Service
web app, as well as available as a standalone blade in Azure portal. See the Viewing changes for all resources in
Azure section to access Change Analysis blade and the Change Analysis for the Web Apps feature section for using
it within Web App portal later in this article.
Azure Resource Manager tracked properties changes
Using Azure Resource Graph, Change Analysis provides a historical record of how the Azure resources that host
your application have changed over time. Tracked settings such as managed identities, Platform OS upgrade, and
hostnames can be detected.
Azure Resource Manager proxied setting changes
Settings such as IP Configuration rule, SSL settings and extension versions are not yet available in ARG, so Change
Analysis queries and computes these changes securely to provide more details in what changed in the app. These
information is not available yet in Azure Resource Graph but will be available soon.
Changes in web app deployment and configuration (in-guest changes)
Change Analysis captures the deployment and configuration state of an application every 4 hours. It can detect, for
example, changes in the application environment variables. The tool computes the differences and presents what
has changed. Unlike Resource Manager changes, code deployment change information might not be available
immediately in the tool. To view the latest changes in Change Analysis, select Scan changes now.
Dependency changes
Changes to resource dependencies can also cause issues in a web app. For example, if a web app calls into a Redis
cache, the Redis cache SKU could affect the web app performance. To detect changes in dependencies, Change
Analysis checks the web app's DNS record. In this way, it identifies changes in all app components that could cause
issues. Currently the following dependencies are supported:
Web Apps
Azure Storage
Azure SQL
Enablement
"Microsoft.ChangeAnalysis" resource provider needs to be registered with a subscription for the Azure Resource
Manager tracked properties and proxied settings change data to be available. As you enter the Web App diagnose
and solve problems or bring up the Change Analysis standalone blade, this resource provider is automatically
registered. It does not have any performance and cost implementations for your subscription. For web app in-guest
changes, separate enablement is required for scanning code files within a web app. See Enable Change Analysis in
the Diagnose and solve problems tool section later in this article for more details.
You can see Insights and related dependencies resources that host your application. This view is designed to be
application-centric for developers to troubleshoot issues.
Currently supported resources include:
Virtual Machines
Virtual Machine Scale Set
Azure Networking resources
Web app with in-guest file tracking and environment variables changes
For any feedback, please use the send feedback button in the blade or email changeanalysisteam@microsoft.com.
Change Analysis for the Web Apps feature
In Azure Monitor, Change Analysis is also built into the self-service Diagnose and solve problems experience.
Access this experience from the Overview page of your App Service application.
2. Select Application Changes. Not that the feature is also available in Application Crashes.
3. To enable Change Analysis, select Enable now.
4. Turn on Change Analysis and select Save. The tool displays all web apps under an App Service plan. You
can use the plan level switch to turn on Change Analysis for all web apps under a plan.
5. To access Change Analysis, select Diagnose and solve problems > Availability and Performance >
Application Crashes. You'll see a graph that summarizes the type of changes over time along with details
on those changes. By default, changes in the past 24 hours are displayed to help with immediate problems.
Enable Change Analysis at scale
If your subscription includes numerous web apps, enabling the service at the level of the web app would be
inefficient. Run the following script to enable all web apps in your subscription.
Pre-requisites:
PowerShell Az Module. Follow instructions at Install the Azure PowerShell module
Run the following script:
# Get subscription Id
$SubscriptionId = Read-Host -Prompt 'Input your subscription Id'
Next steps
Enable Application Insights for Azure App Services apps.
Enable Application Insights for Azure VM and Azure virtual machine scale set IIS -hosted apps.
Learn more about Azure Resource Graph, which helps power Change Analysis.
Visualizing data from Azure Monitor
10/18/2019 • 4 minutes to read • Edit Online
This article provides a summary of the available methods to visualize log and metric data stored in Azure Monitor.
Visualizations such as charts and graphs can help you analyze your monitoring data to drill-down on issues and
identify patterns. Depending on the tool you use, you may also have the option to share visualizations with other
users inside and outside of your organization.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log
Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology
to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Azure Dashboards
Azure dashboards are the primary dashboarding technology for Azure. They're particularly useful in providing
single pane of glass over your Azure infrastructure and services allowing you to quickly identify important issues.
Advantages
Deep integration into Azure. Visualizations can be pinned to dashboards from multiple Azure pages including
Metrics Explorer, Log Analytics, and Application Insights.
Supports both metrics and logs.
Combine data from multiple sources including output from metrics explorer, Log queries, and maps and
availability in Application Insights.
Option for personal or shared dashboards. Integrated with Azure role based authentication (RBAC ).
Automatic refresh. Metrics refresh depends on time range with minimum of five minutes. Logs refresh every
hour, with a manual refresh option on demand by clicking the "refresh" icon on a given visualization, or by
refreshing the full dashboard.
Parametrized metrics dashboards with timestamp and custom parameters.
Flexible layout options.
Full screen mode.
Limitations
Limited control over log visualizations with no support for data tables. Total number of data series is limited to
10 with further data series grouped under an other bucket.
No custom parameters support for log charts.
Log charts are limited to last 30 days.
Log charts can only be pinned to shared dashboards.
No interactivity with dashboard data.
Limited contextual drill-down.
Advantages
Rich visualizations for log data.
Export and import views to transfer them to other resource groups and subscriptions.
Integrates into Azure Monitor management model with workspaces and monitoring solutions.
Filters for custom parameters.
Interactive, supports multi-level drill-in (view that drills into another view )
Limitations
Supports logs but not metrics.
No personal views. Available to all users with access to the workspace.
No automatic refresh.
Limited layout options.
No support for querying across multiple workspaces or Application Insights applications.
Queries are limited in response size to 8MB and query execution time of 110 seconds.
Workbooks
Workbooks are interactive documents that provide deep insights into your data, investigation, and collaboration
inside the team. Specific examples where workbooks are useful are troubleshooting guides and incident
postmortem.
Advantages
Supports both metrics and logs.
Supports parameters enabling interactive reports where selecting an element in a table will dynamically update
associated charts and visualizations.
Document-like flow.
Option for personal or shared workbooks.
Easy, collaborative-friendly authoring experience.
Templates support public GitHub-based template gallery.
Limitations
No automatic refresh.
No dense layout like dashboards, which make workbooks less useful as a single pane of glass. Intended more
for providing deeper insights.
Power BI
Power BI is particularly useful for creating business-centric dashboards and reports, as well as reports analyzing
long-term KPI trends. You can import the results of a log query into a Power BI dataset so you can take advantage
of its features such as combining data from different sources and sharing reports on the web and mobile devices.
Advantages
Rich visualizations.
Extensive interactivity including zoom-in and cross-filtering.
Easy to share throughout your organization.
Integration with other data from multiple data sources.
Better performance with results cached in a cube.
Limitations
Supports logs but not metrics.
No Azure integration. Can't manage dashboards and models through Azure Resource Manager.
Query results need to be imported into Power BI model to configure. Limitation on result size and refresh.
Limited data refresh of eight times per day.
Grafana
Grafana is an open platform that excels in operational dashboards. It's particularly useful for detecting and
isolating and triaging operational incidents. You can add Grafana Azure Monitor data source plugin to your Azure
subscription to have it visualize your Azure metrics data.
Advantages
Rich visualizations.
Rich ecosystem of datasources.
Data interactivity including zoom in.
Supports parameters.
Limitations
No Azure integration. Can't manage dashboards and models through Azure Resource Manager.
Cost to support additional Grafana infrastructure or additional cost for Grafana Cloud.
Next steps
Learn about the data collected by Azure Monitor.
Learn about Azure dashboards.
Learn about Views in Azure Monitor.
Learn about Workbooks.
Learn about import log data into Power BI.
Learn about the Grafana Azure Monitor data source plugin.
Azure Monitor partner integrations
12/6/2019 • 9 minutes to read • Edit Online
Alert Logic Log Manager collects VM, application, and Azure platform logs for security analysis and retention. It
also collects the Azure Activity Log through the Azure Monitor API. This information is used to detect malfeasance
and meet compliance requirements.
Go to the documentation.
AppDynamics
AppDynamics Application Performance Management (APM ) enables application owners to rapidly troubleshoot
performance bottlenecks and optimize the performance of their applications running in Azure environment. It can
monitor Azure Cloud Services (PaaS ), web & worker roles, Virtual Machines (IaaS ), Remote Service Detection
(Microsoft Azure Service Bus), Microsoft Azure Queue, Microsoft Azure Remote Services (Azure Blob), Azure
Queue (Microsoft Service Bus), Data Storage, and Microsoft Azure Blob Storage. AppDynamics APM is available in
the Azure Marketplace.
Go to the documentation.
Atlassian JIRA
Botmetric
Learn more.
Circonus
Circonus is a microservices monitoring and analytics platform built for on premises or SaaS deployment. It is fully
automatable API-Centric platform is more scalable and reliable than systems it monitors. Developed for the
requirements of DevOps, Circonus delivers percentile-based alerts, graphs, dashboards, and machine-learning
intelligence that enable business optimization. Circonus monitors your Microsoft Azure cloud resources and their
applications in real time. You can use Circonus to collect and track metrics for the variables you want to measure
for your resources and applications. With Circonus, you gain system-wide visibility into Azure’s resource
utilization, application performance, and operational health.
Go to the documentation.
CloudHealth
Unite and automate your cloud with a platform built to save time and money. CloudHealth provides visibility,
intuitive optimization, and rock-solid governance practices for cloud management. The CloudHealth platform
enables enterprises and MSPs to maximize return on cloud investments. Make confident decisions around cost,
usage, performance, and security.
Learn more
CloudMonix
CloudMonix offers monitoring, automation, and self-healing services for Microsoft Azure platform.
Go to the documentation.
Datadog
Datadog is the world’s leading monitoring service for cloud-scale applications. It brings together data from servers,
databases, tools, and services to present a unified view of your entire stack. These capabilities are provided on a
SaaS -based data analytics platform. This service enables Dev and Ops teams to work collaboratively to avoid
downtime, resolve performance problems, and ensure that development and deployment cycles finish on time. By
integrating Datadog and Azure, you can collect and view metrics from across your infrastructure. Correlate VM
metrics with application-level metrics. Slice and dice your metrics using any combination of properties and custom
tags.
Go to the documentation.
Dynatrace
The Dynatrace OneAgent integrates with Azure VMs and App Services via the Azure extension mechanism. This
way Dynatrace OneAgent can gather performance metrics about hosts, network, and services. Besides just
displaying metrics, Dynatrace visualizes environments end-to-end. It shows transactions from the client side to the
database layer. Dynatrace provides AI-based correlation of problems and fully integrated root-cause-analysis to
give method level insights into code and database. This insight makes troubleshooting and performance
optimizations much easier.
Go to the documentation.
Elastic
Elastic is a search company. As the creators of the Elastic Stack (Elasticsearch, Kibana, Beats, and Logstash), Elastic
builds self-managed and SaaS offerings that make data usable in real time and at scale for search, logging, security,
and analytics use cases.
Go to the documentation.
Grafana
Grafana is an open-source application that enables you to visualize time series metric data.
Go to the documentation.
InfluxData
InfluxData, the creator of InfluxDB, delivers a modern Open Source Platform built from the ground up for
analyzing metrics and events (time series data) for DevOps and IoT applications. Whether the data comes from
humans, sensors, or machines, InfluxData empowers developers to build next-generation monitoring, analytics,
and IoT applications faster, easier, and to scale delivering real business value quickly. Based in San Francisco,
InfluxData’s more than 420 customers include Cisco, eBay, IBM, and Siemens.
Go to the documentation.
Logic Monitor
LogicMonitor® is the leading SaaS -based, performance monitoring platform for complex IT infrastructure. With
coverage for thousands of technologies, LogicMonitor provides granular visibility into infrastructure and
application performance. LM Cloud’s comprehensive Azure monitoring enables users to correlate the performance
of Azure cloud, on-premises, and hybrid cloud resources -- all from a single platform. Automated resource
discovery, built in monitoring templates, preconfigured alert thresholds, and customizable dashboards combine to
give IT the speed, flexibility, and visibility required to succeed.
Go to the documentation.
LogRhythm
LogRhythm, a leader in NextGen SIEM, empowers organizations on six continents to measurably reduce risk by
rapidly detecting, responding to, and neutralizing cyberthreats. LogRhythm’s Threat Lifecycle Management (TLM )
workflow is the foundation for security operations centers, helping customers secure their cloud, physical, and
virtual infrastructures for IT and OT environments. If you’re a LogRhythm customer and are ready to start your
Azure journey, you’ll need to install and configure the LogRhythm Open Collector and EventHub integration. More
details, including documentation of both configuring Azure Monitor and the Open Collector, can be found here.
Microfocus
Microfocus ArcSight has a smart connector for Azure Monitor event hubs.
Learn more
Microfocus Operations Bridge automatically monitors all Hybrid IT resources – any device, operating system,
database, application, or service, regardless of where it runs and applies AIOps to all data types – events, metrics,
logs and dependencies. It provides a unique combination of quality of service monitoring, coupled with deep
application health analytics, and includes comprehensive performance and availability monitoring of Microsoft
Azure services. Operations Bridge enables customers to provide a single pane of glass, available on any device
with a browser, in ways both business and IT stakeholders can understand.
Learn more
OB Suite overview
Download
SiteScope - SiteScope is a component in the Operations Bridge Suite.
Moogsoft
NewRelic
Learn more.
OpsGenie
OpsGenie acts as a dispatcher for the alerts generated by Azure. OpsGenie determines the right people to notify
based on on-call schedules and escalations. It can notify them using by email, text messages (SMS ), phone calls, or
push notifications. Azure generates alerts for detected problems. OpsGenie ensures the right people are working
on the problem.
Go to the documentation.
PagerDuty
PagerDuty, the leading incident management solution, has provided first-class support for Azure Alerts on metrics.
PagerDuty supports notifications on Azure Monitor alerts, autoscale notifications, activity log events, and platform-
level metrics for Azure services. These enhancements give you increased visibility into the core Azure Platform.
You can take full advantage of PagerDuty’s incident management capabilities for real-time response. The expanded
Azure integration is made possible through webhooks. Webhooks allow you to set up and customize the solution
quickly and easily.
Go to the documentation.
QRadar
The Microsoft Azure DSM and Microsoft Azure Event Hub Protocol are available for download from the IBM
support website. You can learn more about the integration with Azure here.
ScienceLogic
ScienceLogic delivers the next generation IT service assurance platform for managing any technology, anywhere.
ScienceLogic delivers the scale, security, automation, and resiliency necessary to simplify the tasks of managing IT
resources, services, and applications. The ScienceLogic platform uses Azure APIs to interface with Microsoft Azure.
ScienceLogic gives you real-time visibility into your Azure services and resources. So you know when something’s
not working and you can fix it faster. You can also manage Azure alongside your other clouds and data center
systems and services.
Learn more.
Serverless360
Serverless360 is a one platform tool to operate, manage, and monitor Azure serverless components.
Manageability is one of the key challenges with serverless implementations. Hundreds of small, discrete serverless
services are scattered in various places – managing and operating such solutions is complex. Serverless360 solves
these challenges with rich set of sophisticated tools. It can monitor serverless services like Azure Functions, Logic
Apps, Event Grids, Service Bus Queues, Topics, Relays, Event Hubs, Storage queues, files, blob, and tables.
Serverless360 is available in the Azure Marketplace. These capabilities are available on both SaaS and private
hosting (hosted on your own environment).
Learn more.
SignalFx
SignalFx is the leader in real-time operational intelligence for data-driven DevOps. The service discovers and
collects metrics across every component in the cloud. It replaces traditional point tools and provides real-time
visibility into today’s dynamic environments. Leveraging the massively scalable SignalFx platform, the SaaS
platform is optimized for container and microservices based architectures and provides powerful visualization,
proactive alerting, and collaborative triage capabilities across organizations of all sizes. SignalFx integrates directly
with Azure Monitor as well as through open-source connectors such as Telegraf, statsD, and collectd to provide
best in class dashboards, analytics, and alerts for Azure.
Go to the documentation.
SIGNL4
SIGNL4 - the mobile alerting app for operations teams - is the fastest way to route critical alerts from Azure
Monitor to the right people at the right time – anywhere by push, text, and voice calls. SIGNL4 manages on-call
duties and shifts of your team, tracks delivery and ownership of alerts and escalates if necessary. Full transparency
across your team is provided. Using the super-easy REST web-hook of SIGNL4 any Azure service can be
connected with no effort. With SIGNL4, you will see up to 10x faster response over email notifications and manual
alerting.
Go to the documentation.
SolarWinds
Learn more.
Splunk
The Azure Monitor Add-on for Splunk is available in the Splunkbase here.
Go to the documentation.
Sumo Logic
Sumo Logic is a secure, cloud-native, machine data analytics service, delivering real-time, continuous intelligence
from structured, semi-structured, and unstructured data across the entire application lifecycle and stack. More than
1,000 customers around the globe rely on Sumo Logic for the analytics and insights to build, run, and secure their
applications and cloud infrastructures. With Sumo Logic, customers gain a multi-tenant, service-model advantage
to help increase competitive advantage, business value, and growth.
Learn more.
Turbonomic
Turbonomic delivers workload automation for hybrid clouds by simultaneously optimizing performance, cost, and
compliance in real time. Turbonomic helps organizations be elastic in their Azure estate by continuously optimizing
the estate to ensure applications constantly get the resources they require to deliver their SLA and nothing more
across compute, storage, and network for the IaaS and PaaS layer. Organizations can simulate migrations, properly
scale workloads, and retire datacenter resources to responsibly migrate to Azure on-time, within budget, while
assuring both performance and compliance. Turbonomic is API driven and runs as an agentless VM in Azure and
on-premises.
Learn more.
Next steps
Learn more about Azure Monitor
Access metrics using the REST API
Stream the Activity Log to a non-Microsoft service
Stream resource logs to a non-Microsoft service
Designing your Azure Monitor Logs deployment
12/17/2019 • 11 minutes to read • Edit Online
Azure Monitor stores log data in a Log Analytics workspace, which is an Azure resource and a container where
data is collected, aggregated, and serves as an administrative boundary. While you can deploy one or more
workspaces in your Azure subscription, there are several considerations you should understand in order to
ensure your initial deployment is following our guidelines to provide you with a cost effective, manageable, and
scalable deployment meeting your organizations needs.
Data in a workspace is organized into tables, each of which stores different kinds of data and has its own unique
set of properties based on the resource generating the data. Most data sources will write to their own tables in a
Log Analytics workspace.
Access mode Method the user uses to access the workspace. Defines the
scope of the data available and the access control mode that's
applied.
Access control mode Setting on the workspace that defines whether permissions
are applied at the workspace or resource level.
Table level RBAC Optional granular permissions that apply to all users
regardless of their access mode or access control mode.
Defines which data types a user can access.
Access mode
The access mode refers to how a user accesses a Log Analytics workspace and defines the scope of data they can
access.
Users have two options for accessing the data:
Workspace-context: You can view all logs in the workspace you have permission to. Queries in this mode
are scoped to all data in all tables in the workspace. This is the access mode used when logs are accessed
with the workspace as the scope, such as when you select Logs from the Azure Monitor menu in the
Azure portal.
Resource-context: When you access the workspace for a particular resource, resource group, or
subscription, such as when you select Logs from a resource menu in the Azure portal, you can view logs
for only resources in all tables that you have access to. Queries in this mode are scoped to only data
associated with that resource. This mode also enables granular RBAC.
NOTE
Logs are available for resource-context queries only if they were properly associated with the relevant resource.
Currently, the following resources have limitations:
Computers outside of Azure
Service Fabric
Application Insights
You can test if logs are properly associated with their resource by running a query and inspecting the records you're
interested in. If the correct resource ID is in the _ResourceId property, then data is available to resource-centric
queries.
Azure Monitor automatically determines the right mode depending on the context you perform the log search
from. The scope is always presented in the top-left section of Log Analytics.
Comparing access modes
The following table summarizes the access modes:
WORKSPACE-CONTEX T RESOURCE-CONTEX T
Who is each model intended for? Central administration. Administrators Application teams. Administrators of
who need to configure data collection Azure resources being monitored.
and users who need access to a wide
variety of resources. Also currently
required for users who need to access
logs for resources outside of Azure.
WORKSPACE-CONTEX T RESOURCE-CONTEX T
What does a user require to view logs? Permissions to the workspace. See Read access to the resource. See
Workspace permissions in Manage Resource permissions in Manage
access using workspace permissions. access using Azure permissions.
Permissions can be inherited (such as
from the containing resource group) or
directly assigned to the resource.
Permission to the logs for the resource
will be automatically assigned.
What is the scope of permissions? Workspace. Users with access to the Azure resource. User can query logs for
workspace can query all logs in the specific resources, resource groups, or
workspace from tables that they have subscription they have access to from
permissions to. See Table access control any workspace but can't query logs for
other resources.
How can user access logs? Start Logs from Azure Start Logs from the menu for
Monitor menu. the Azure resource
Start Logs from Log Analytics Start Logs from Azure Monitor
workspaces. menu.
From Azure Monitor Start Logs from Log Analytics
Workbooks. workspaces.
From Azure Monitor
Workbooks.
NOTE
If a user has only resource permissions to the workspace, they are only able to access the workspace using
resource-context mode assuming the workspace access mode is set to Use resource or workspace permissions.
To learn how to change the access control mode in the portal, with PowerShell, or using a Resource Manager
template, see Configure access control mode.
Operation
|where OperationCategory == "Ingestion"
|where Detail startswith "The rate of data crossed the threshold"
Recommendations
This scenario covers a single workspace design in your IT organizations subscription that is not constrained by
data sovereignty or regulatory compliance, or needs to map to the regions your resources are deployed within. It
allows your organizations security and IT admin teams the ability to leverage the improved integration with
Azure access management and more secure access control.
All resources, monitoring solutions, and Insights such as Application Insights and Azure Monitor for VMs,
supporting infrastructure and applications maintained by the different teams are configured to forward their
collected log data to the IT organizations centralized shared workspace. Users on each team are granted access to
logs for resources they have been given access to.
Once you have deployed your workspace architecture, you can enforce this on Azure resources with Azure Policy.
It provides a way to define policies and ensure compliance with your Azure resources so they send all their
resource logs to a particular workspace. For example, with Azure virtual machines or virtual machine scale sets,
you can use existing policies that evaluate workspace compliance and report results, or customize to remediate if
non-compliant.
Next steps
To implement the security permissions and controls recommended in this guide, review manage access to logs.
Roles, permissions, and security in Azure Monitor
12/6/2019 • 9 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Many teams need to strictly regulate access to monitoring data and settings. For example, if you have team
members who work exclusively on monitoring (support engineers, DevOps engineers) or if you use a managed
service provider, you may want to grant them access to only monitoring data while restricting their ability to create,
modify, or delete resources. This article shows how to quickly apply a built-in monitoring RBAC role to a user in
Azure or build your own custom role for a user who needs limited monitoring permissions. It then discusses
security considerations for your Azure Monitor-related resources and how you can limit access to the data they
contain.
Monitoring Contributor
People assigned the Monitoring Contributor role can view all monitoring data in a subscription and create or
modify monitoring settings, but cannot modify any other resources. This role is a superset of the Monitoring
Reader role, and is appropriate for members of an organization’s monitoring team or managed service providers
who, in addition to the permissions above, also need to be able to:
Publish monitoring dashboards as a shared dashboard.
Set diagnostic settings for a resource.*
Set the log profile for a subscription.*
Set alert rules activity and settings via Azure Alerts.
Create Application Insights web tests and components.
List Log Analytics workspace shared keys.
Enable or disable monitoring packs in Log Analytics workspace.
Create and delete and execute saved searches in Log Analytics workspace.
Create and delete the Log Analytics workspace storage configuration.
*user must also separately be granted ListKeys permission on the target resource (storage account or event hub
namespace) to set a log profile or diagnostic setting.
NOTE
This role does not give read access to log data that has been streamed to an event hub or stored in a storage account. See
below for information on configuring access to these resources.
OPERATION DESCRIPTION
Microsoft.Insights/AlertRules/Incidents/Read List incidents (history of the alert rule being triggered) for
alert rules. This only applies to the portal.
Microsoft.Insights/LogProfiles/[Read, Write, Delete] Read/write/delete log profiles (streaming Activity Log to event
hub or storage account).
NOTE
Access to alerts, diagnostic settings, and metrics for a resource requires that the user has Read access to the resource type
and scope of that resource. Creating (“write”) a diagnostic setting or log profile that archives to a storage account or streams
to event hubs requires the user to also have ListKeys permission on the target resource.
For example, using the above table you could create a custom RBAC role for an “Activity Log Reader” like this:
$context = New-AzStorageContext -ConnectionString "[connection string for your monitoring Storage Account]"
$token = New-AzStorageAccountSASToken -ResourceType Service -Service Blob -Permission "rl" -Context $context
You can then give the token to the entity that needs to read from that storage account, and it can list and read from
all blobs in that storage account.
Alternatively, if you need to control this permission with RBAC, you can grant that entity the
Microsoft.Storage/storageAccounts/listkeys/action permission on that particular storage account. This is necessary
for users who need to be able to set a diagnostic setting or log profile to archive to a storage account. For example,
you could create the following custom RBAC role for a user or application that only needs to read from one
storage account:
WARNING
The ListKeys permission enables the user to list the primary and secondary storage account keys. These keys grant the user
all signed permissions (read, write, create blobs, delete blobs, etc.) across all signed services (blob, queue, table, file) in that
storage account. We recommend using an Account SAS described above when possible.
Next steps
Read about RBAC and permissions in Resource Manager
Read the overview of monitoring in Azure
IP addresses used by Application Insights and Log
Analytics
1/14/2020 • 2 minutes to read • Edit Online
The Azure Application Insights service uses a number of IP addresses. You might need to know these
addresses if the app that you are monitoring is hosted behind a firewall.
NOTE
Although these addresses are static, it's possible that we will need to change them from time to time. All Application
Insights traffic represents outbound traffic with the exception of availability monitoring and webhooks which require
inbound firewall rules.
TIP
Subscribe to this page as a RSS feed by adding https://github.com/MicrosoftDocs/azure-
docs/commits/master/articles/azure-monitor/app/ip-addresses.md.atom to your favorite RSS/ATOM reader to get
notified of the latest changes.
Outgoing ports
You need to open some outgoing ports in your server's firewall to allow the Application Insights SDK and/or
Status Monitor to send data to the portal:
Status Monitor
Status Monitor Configuration - needed only when making changes.
Availability tests
This is the list of addresses from which availability web tests are run. If you want to run web tests on your app,
but your web server is restricted to serving specific clients, then you will have to permit incoming traffic from
our availability test servers.
Open ports 80 (http) and 443 (https) for incoming traffic from these addresses (IP addresses are grouped by
location):
Australia East
20.40.124.176/28
20.40.124.240/28
20.40.125.80/28
Brazil South
191.233.26.176/28
191.233.26.128/28
191.233.26.64/28
France Central
20.40.129.32/28
20.40.129.48/28
20.40.129.64/28
20.40.129.80/28
East Asia
52.229.216.48/28
52.229.216.64/28
52.229.216.80/28
North Europe
52.158.28.64/28
52.158.28.80/28
52.158.28.96/28
52.158.28.112/28
Japan East
52.140.232.160/28
52.140.232.176/28
52.140.232.192/28
West Europe
51.144.56.96/28
51.144.56.112/28
51.144.56.128/28
51.144.56.144/28
51.144.56.160/28
51.144.56.176/28
UK South
51.105.9.128/28
51.105.9.144/28
51.105.9.160/28
UK West
20.40.104.96/28
20.40.104.112/28
20.40.104.128/28
20.40.104.144/28
Southeast Asia
52.139.250.96/28
52.139.250.112/28
52.139.250.128/28
52.139.250.144/28
West US
40.91.82.48/28
40.91.82.64/28
40.91.82.80/28
40.91.82.96/28
40.91.82.112/28
40.91.82.128/28
Central US
13.86.97.224/28
13.86.97.240/28
13.86.98.48/28
13.86.98.0/28
13.86.98.16/28
13.86.98.64/28
North Central US
23.100.224.16/28
23.100.224.32/28
23.100.224.48/28
23.100.224.64/28
23.100.224.80/28
23.100.224.96/28
23.100.224.112/28
23.100.225.0/28
South Central US
20.45.5.160/28
20.45.5.176/28
20.45.5.192/28
20.45.5.208/28
20.45.5.224/28
20.45.5.240/28
East US
20.42.35.32/28
20.42.35.64/28
20.42.35.80/28
20.42.35.96/28
20.42.35.112/28
20.42.35.128/28
Alert webhooks
PURPOSE IP PORTS
Profiler
PURPOSE URI IP PORTS
PURPOSE URI IP PORTS
Snapshot Debugger
NOTE
Profiler and Snapshot Debugger share the same set of IP addresses.
Use the Log Analytics workspaces menu to create a Log Analytics workspace using the Azure portal. A Log
Analytics workspace is a unique environment for Azure Monitor log data. Each workspace has its own data
repository and configuration, and data sources and solutions are configured to store their data in a particular
workspace. You require a Log Analytics workspace if you intend on collecting data from the following sources:
Azure resources in your subscription
On-premises computers monitored by System Center Operations Manager
Device collections from Configuration Manager
Diagnostics or log data from Azure storage
For other sources, such as Azure VMs and Windows or Linux VMs in your environment, see the following topics:
Collect data from Azure virtual machines
Collect data from hybrid Linux computer
Collect data from hybrid Windows computer
If you don't have an Azure subscription, create a free account before you begin.
Create a workspace
1. In the Azure portal, click All services. In the list of resources, type Log Analytics. As you begin typing,
the list filters based on your input. Select Log Analytics workspaces.
2. Click Add, and then select choices for the following items:
Provide a name for the new Log Analytics workspace, such as DefaultLAWorkspace. This name
must be globally unique across all Azure Monitor subscriptions.
Select a Subscription to link to by selecting from the drop-down list if the default selected is not
appropriate.
For Resource Group, choose to use an existing resource group already setup or create a new one.
Select an available Location. For more information, see which regions Log Analytics is available in
and search for Azure Monitor from the Search for a product field.
If you are creating a workspace in a new subscription created after April 2, 2018, it will
automatically use the Per GB pricing plan and the option to select a pricing tier will not be available.
If you are creating a workspace for an existing subscription created before April 2, or to
subscription that was tied to an existing Enterprise Agreement (EA) enrollment, select your
preferred pricing tier. For more information about the particular tiers, see Log Analytics Pricing
Details.
3. After providing the required information on the Log Analytics Workspace pane, click OK.
While the information is verified and the workspace is created, you can track its progress under Notifications
from the menu.
Next steps
Now that you have a workspace available, you can configure collection of monitoring telemetry, run log searches
to analyze that data, and add a management solution to provide additional data and analytic insights.
To enable data collection from Azure resources with Azure Diagnostics or Azure storage, see Collect Azure
service logs and metrics for use in Log Analytics.
Add System Center Operations Manager as a data source to collect data from agents reporting your
Operations Manager management group and store it in your Log Analytics workspace.
Connect Configuration Manager to import computers that are members of collections in the hierarchy.
Review the monitoring solutions available and how to add or remove a solution from your workspace.
Create a Log Analytics workspace with Azure CLI 2.0
1/22/2020 • 4 minutes to read • Edit Online
The Azure CLI 2.0 is used to create and manage Azure resources from the command line or in scripts. This
quickstart shows you how to use Azure CLI 2.0 to deploy a Log Analytics workspace in Azure Monitor. A Log
Analytics workspace is a unique environment for Azure Monitor log data. Each workspace has its own data
repository and configuration, and data sources and solutions are configured to store their data in a particular
workspace. You require a Log Analytics workspace if you intend on collecting data from the following sources:
Azure resources in your subscription
On-premises computers monitored by System Center Operations Manager
Device collections from Configuration Manager
Diagnostic or log data from Azure storage
For other sources, such as Azure VMs and Windows or Linux VMs in your environment, see the following topics:
Collect data from Azure virtual machines
Collect data from hybrid Linux computer
Collect data from hybrid Windows computer
If you don't have an Azure subscription, create a free account before you begin.
OPTION EXAMPLE/LINK
Create a workspace
Create a workspace with az group deployment create. The following example creates a workspace in the eastus
location using a Resource Manager template from your local machine. The JSON template is configured to only
prompt you for the name of the workspace, and specifies a default value for the other parameters that would likely
be used as a standard configuration in your environment. Or you can store the template in an Azure storage
account for shared access in your organization. For further information about working with templates, see Deploy
resources with Resource Manager templates and Azure CLI
For information about regions supported, see regions Log Analytics is available in and search for Azure Monitor
from the Search for a product field.
The following parameters set a default value:
location - defaults to East US
sku - defaults to the new Per-GB pricing tier released in the April 2018 pricing model
WARNING
If creating or configuring a Log Analytics workspace in a subscription that has opted into the new April 2018 pricing model,
the only valid Log Analytics pricing tier is PerGB2018.
Next steps
Now that you have a workspace available, you can configure collection of monitoring telemetry, run log searches
to analyze that data, and add a management solution to provide additional data and analytic insights.
To enable data collection from Azure resources with Azure Diagnostics or Azure storage, see Collect Azure
service logs and metrics for use in Log Analytics.
Add System Center Operations Manager as a data source to collect data from agents reporting your
Operations Manager management group and store it in your Log Analytics workspace.
Connect Configuration Manager to import computers that are members of collections in the hierarchy.
Review the monitoring solutions available and how to add or remove a solution from your workspace.
Create a Log Analytics workspace with Azure
PowerShell
1/22/2020 • 4 minutes to read • Edit Online
The Azure PowerShell module is used to create and manage Azure resources from the PowerShell command line
or in scripts. This quickstart shows you how to use the Azure PowerShell module to deploy a Log Analytics
workspace in Azure Monitor. A Log Analytics workspace is a unique environment for Azure Monitor log data. Each
workspace has its own data repository and configuration, and data sources and solutions are configured to store
their data in a particular workspace. You require a Log Analytics workspace if you intend on collecting data from
the following sources:
Azure resources in your subscription
On-premises computers monitored by System Center Operations Manager
Device collections from Configuration Manager
Diagnostic or log data from Azure storage
For other sources, such as Azure VMs and Windows or Linux VMs in your environment, see the following topics:
Collect data from Azure virtual machines
Collect data from hybrid Linux computer
Collect data from hybrid Windows computer
If you don't have an Azure subscription, create a free account before you begin.
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
OPTION EXAMPLE/LINK
Create a workspace
Create a workspace with New -AzResourceGroupDeployment. The following example creates a workspace in the
eastus location using a Resource Manager template from your local machine. The JSON template is configured to
only prompt you for the name of the workspace, and specifies a default value for the other parameters that would
likely be used as a standard configuration in your environment.
For information about regions supported, see regions Log Analytics is available in and search for Azure Monitor
from the Search for a product field.
The following parameters set a default value:
location - defaults to East US
sku - defaults to the new Per-GB pricing tier released in the April 2018 pricing model
WARNING
If creating or configuring a Log Analytics workspace in a subscription that has opted into the new April 2018 pricing model,
the only valid Log Analytics pricing tier is PerGB2018.
Next steps
Now that you have a workspace available, you can configure collection of monitoring telemetry, run log searches
to analyze that data, and add a management solution to provide additional data and analytic insights.
To enable data collection from Azure resources with Azure Diagnostics or Azure storage, see Collect Azure
service logs and metrics for use in Azure Monitor.
Add System Center Operations Manager as a data source to collect data from agents reporting your
Operations Manager management group and store it in your Log Analytics workspace.
Connect Configuration Manager to import computers that are members of collections in the hierarchy.
Review the monitoring solutions available and how to add or remove a solution from your workspace.
Delete and restore Azure Log Analytics workspace
1/22/2020 • 4 minutes to read • Edit Online
This article explains the concept of Azure Log Analytics workspace soft-delete and how to recover deleted
workspace.
NOTE
The soft-delete behavior cannot be turned off. We will shortly add an option to override the soft-delete when using a ‘force’
tag in the delete operation.
You want to exercise caution when you delete a workspace because there might be important data and
configuration that may negatively impact your service operation. Review what agents, solutions, and other Azure
services and sources that store their data in Log Analytics, such as:
Management solutions
Azure Automation
Agents running on Windows and Linux virtual machines
Agents running on Windows and Linux computers in your environment
System Center Operations Manager
The soft-delete operation deletes the workspace resource and any associated users’ permission is broken. If users
are associated with other workspaces, then they can continue using Log Analytics with those other workspaces.
Soft-delete behavior
The workspace delete operation removes the workspace Resource Manager resource, but its configuration and
data are kept for 14 days, while giving the appearance that the workspace is deleted. Any agents and System
Center Operations Manager management groups configured to report to the workspace remain in an orphaned
state during the soft-delete period. The service further provides a mechanism for recovering the deleted workspace
including its data and connected resources, essentially undoing the deletion.
NOTE
Installed solutions and linked services like your Azure Automation account are permanently removed from the workspace at
deletion time and can’t be recovered. These should be reconfigured after the recovery operation to bring the workspace to its
previously configured state.
You can delete a workspace using PowerShell, REST API, or in the Azure portal.
Azure portal
1. To sign in, go to the Azure portal.
2. In the Azure portal, select All services. In the list of resources, type Log Analytics. As you begin typing, the list
filters based on your input. Select Log Analytics workspaces.
3. In the list of Log Analytics workspaces, select a workspace and then click Delete from the top of the middle
pane.
4. When the confirmation message window appears asking you to confirm deletion of the workspace, click Yes.
PowerShell
IMPORTANT
Use caution when permanently deleting your workspace since the operation is irreversible, and your workspace and its data
won’t be recoverable.
The permanent workspace delete can currently be performed via REST API.
NOTE
Any API request must include a Bearer authorization token in the request header.
You can acquire the token using:
App registrations
Navigate to Azure portal using the developer's console (F12) in the browser. Look in one of the batch? instances for the
authentication string under Request Headers. This will be in the pattern authorization: Bearer . Copy and add this to
your API call as shown in the examples.
Navigate to the Azure REST documentation site. press Try it on any API, copy the Bearer token, and add it to your API
call. To permanently delete your workspace, use the Workspaces - Delete REST API call with a force tag:
DELETE https://management.azure.com/subscriptions/<subscription-id>/resourcegroups/<resource-group-
name>/providers/Microsoft.OperationalInsights/workspaces/<workspace-name>?api-version=2015-11-01-
preview&force=true
Authorization: Bearer eyJ0eXAiOiJKV1Qi….
Recover workspace
If you have Contributor permissions to the subscription and resource group where the workspace was associated
before the soft-delete operation, you can recover it during its soft-delete period including its data, configuration
and connected agents. After the soft-delete period, the workspace is non-recoverable and assigned for permanent
deletion. Names of deleted workspaces are preserved during the soft-delete period and can't be used when
attempting to create a new workspace.
You can recover a workspace by re-creating it using the following workspace create methods: PowerShell or REST
API as long as the following properties are populated with the deleted workspace details:
Subscription ID
Resource Group name
Workspace name
Region
PowerShell
PS C:\>Select-AzSubscription "subscription-name-the-workspace-was-in"
PS C:\>New-AzOperationalInsightsWorkspace -ResourceGroupName "resource-group-name-the-workspace-was-in" -Name
"deleted-workspace-name" -Location "region-name-the-workspace-was-in"
The workspace and all its data are brought back after the recovery operation. Solutions and linked services were
permanently removed from the workspace when it was deleted and these should be reconfigured to bring the
workspace to its previously configured state. Some of the data may not be available for query after the workspace
recovery until the associated solutions are re-installed and their schemas are added to the workspace.
NOTE
Workspace recovery isn't supported in the Azure portal.
Re-creating a workspace during the soft-delete period gives an indication that this workspace name is already in use.
Move a Log Analytics workspace to different
subscription or resource group
1/14/2020 • 2 minutes to read • Edit Online
In this article, you'll learn the steps to move Log Analytics workspace to another resource group or subscription in
the same region. You can learn more about moving Azure resources through the Azure portal, PowerShell, the
Azure CLI, or the REST API. at Move resources to a new resource group or subscription.
IMPORTANT
You can't move a workspace to a different region.
IMPORTANT
After the move operation, removed solutions and Automation account link should be reconfigured to bring the workspace
back to its previous state.
Next steps
For a list of which resources support move, see Move operation support for resources.
Azure Monitor for Service Providers
10/25/2019 • 4 minutes to read • Edit Online
Log Analytics workspaces in Azure Monitor can help managed service providers (MSPs), large enterprises,
independent software vendors (ISVs), and hosting service providers manage and monitor servers in customer's
on-premises or cloud infrastructure.
Large enterprises share many similarities with service providers, particularly when there is a centralized IT team
that is responsible for managing IT for many different business units. For simplicity, this document uses the term
service provider but the same functionality is also available for enterprises and other customers.
For partners and service providers who are part of the Cloud Solution Provider (CSP ) program, Log Analytics in
Azure Monitor is one of the Azure services available in Azure CSP subscriptions.
Next steps
Automate creation and configuration of workspaces using Resource Manager templates
Automate creation of workspaces using PowerShell
Use Alerts to integrate with existing systems
Generate summary reports using Power BI
Review the process of configuring Log Analytics and Power BI to monitor multiple CSP customers
Manage access to log data and workspaces in Azure
Monitor
1/21/2020 • 10 minutes to read • Edit Online
Azure Monitor stores log data in a Log Analytics workspace. A workspace is a container that includes data and
configuration information. To manage access to log data, you perform various administrative tasks related to
your workspace.
This article explains how to manage access to logs and to administer the workspaces that contain them, including
how to grant access to:
The workspace using workspace permissions.
Users who need access to log data from specific resources using Azure role-based access control (RBAC ).
Users who need access to log data in a specific table in the workspace using Azure RBAC.
DefaultWorkspace38917: True
DefaultWorkspace21532: False
A value of False means the workspace is configured with the workspace-context access mode. A value of True
means the workspace is configured with the resource-context access mode.
NOTE
If a workspace is returned without a boolean value and is blank, this also matches the results of a False value.
Use the following script to set the access control mode for a specific workspace to the resource-context
permission:
$WSName = "my-workspace"
$Workspace = Get-AzResource -Name $WSName -ExpandProperties
if ($Workspace.Properties.features.enableLogAccessUsingOnlyResourcePermissions -eq $null)
{ $Workspace.Properties.features | Add-Member enableLogAccessUsingOnlyResourcePermissions $true -Force }
else
{ $Workspace.Properties.features.enableLogAccessUsingOnlyResourcePermissions = $true }
Set-AzResource -ResourceId $Workspace.ResourceId -Properties $Workspace.Properties -Force
Use the following script to set the access control mode for all workspaces in the subscription to the resource-
context permission:
Viewing data in the Backup and Site Administrator / Co-administrator Accesses resources deployed using the
Recovery solution tiles classic deployment model
NOTE
In order to successfully perform the last two actions, this permission needs to be granted at the resource group or
subscription level.
NOTE
You can use the ability to add a virtual machine extension to a virtual machine to gain full control over a virtual machine.
The Log Analytics Contributor role includes the following Azure actions:
PERMISSION DESCRIPTION
To add and remove users to a user role, it is necessary to have Microsoft.Authorization/*/Delete and
Microsoft.Authorization/*/Write permission.
Examples:
Microsoft.Insights/logs/*/read
Microsoft.Insights/logs/Heartbeat/read
/read permission is usually granted from a role that includes */read or * permissions such as the built-in Reader
and Contributor roles. Custom roles that include specific actions or dedicated built-in roles might not include this
permission.
See Defining per-table access control below if you want to create different access control for different tables.
"Actions": [
"Microsoft.OperationalInsights/workspaces/read",
"Microsoft.OperationalInsights/workspaces/query/read",
"Microsoft.OperationalInsights/workspaces/query/Heartbeat/read",
"Microsoft.OperationalInsights/workspaces/query/AzureActivity/read"
],
To create a role with access to only the SecurityBaseline table, create a custom role using the following actions:
"Actions": [
"Microsoft.OperationalInsights/workspaces/read",
"Microsoft.OperationalInsights/workspaces/query/read",
"Microsoft.OperationalInsights/workspaces/query/SecurityBaseline/read"
],
Custom logs
Custom logs are created from data sources such as custom logs and HTTP Data Collector API. The easiest way to
identify the type of log is by checking the tables listed under Custom Logs in the log schema.
You can't currently grant access to individual custom logs, but you can grant access to all custom logs. To create a
role with access to all custom logs, create a custom role using the following actions:
"Actions": [
"Microsoft.OperationalInsights/workspaces/read",
"Microsoft.OperationalInsights/workspaces/query/read",
"Microsoft.OperationalInsights/workspaces/query/Tables.Custom/read"
],
Considerations
If a user is granted global read permission with the standard Reader or Contributor roles that include the
*/read action, it will override the per-table access control and give them access to all log data.
If a user is granted per-table access but no other permissions, they would be able to access log data from the
API but not from the Azure portal. To provide access from the Azure portal, use Log Analytics Reader as its
base role.
Administrators of the subscription will have access to all data types regardless of any other permission
settings.
Workspace owners are treated like any other user for per-table access control.
We recommend assigning roles to security groups instead of individual users to reduce the number of
assignments. This will also help you use existing group management tools to configure and verify access.
Next steps
See Log Analytics agent overview to gather data from computers in your datacenter or other cloud
environment.
See Collect data about Azure virtual machines to configure data collection from Azure VMs.
Monitoring usage and estimated costs in Azure
Monitor
1/23/2020 • 5 minutes to read • Edit Online
NOTE
This article, describes how to view usage and estimated costs across multiple Azure monitoring features. Related articles for
specific components of Azure Monitor include:
Manage usage and costs with Azure Monitor Logs describes how to control your costs by changing your data retention
period, and how to analyze and alert on your data usage.
Manage usage and costs for Application Insights describes how to analyze data usage in Application Insights.
From here, you can drill in from this accumulated cost summary to get the finer details in the "Cost by resource"
view. In the current pricing tiers, Azure Log data is charged on the same set of meters whether it originates from
Log Analytics or Application Insights. To separate costs from your Log Analytics or Application Insights usage, you
can add a filter on Resource type. To see all Application Insights costs, filter the Resource type to
"microsoft.insights/components", and for Log Analytics costs, filter Resource type to
"microsoft.operationalinsights/workspaces".
More detail of your usage is available by downloading your usage from the Azure portal. In the downloaded
spreadsheet, you can see usage per Azure resource per day. In this Excel spreadsheet, usage from your Application
Insights resources can be found by first filtering on the "Meter Category" column to show "Application Insights"
and "Log Analytics", and then adding a filter on the "Instance ID" column which is "contains
microsoft.insights/components". Most Application Insights usage is reported on meters with the Meter Category
of Log Analytics, since there is a single logs backend for all Azure Monitor components. Only Application Insights
resources on legacy pricing tiers and multi-step web tests are reported with a Meter Category of Application
Insights. The usage is shown in the "Consumed Quantity" column and the unit for each entry is shown in the "Unit
of Measure" column. More details are available to help you understand your Microsoft Azure bill.
NOTE
Using Cost Management in the Azure Cost Management + Billing hub is the preferred approach to broadly
understanding monitoring costs. The Usage and Estimated Costs experiences for Log Analytics and Application Insights
provide deeper insights for each of those parts of Azure Monitor.
Another option for viewing your Azure Monitor usage is the Usage and estimated costs page in the Monitor
hub. This shows the usage of core monitoring features such as alerting, metrics, notifications, Azure Log Analytics,
and Azure Application Insights. For customers on the pricing plans available before April 2018, this also includes
Log Analytics usage purchased through the Insights and Analytics offer.
On this page, users can view their resource usage for the past 31 days, aggregated per subscription. Drill-ins
show usage trends over the 31-day period. A lot of data needs to come together for this estimate, so please be
patient as the page loads.
This example shows monitoring usage and an estimate of the resulting costs:
Select the link in the monthly usage column to open a chart that shows usage trends over the last 31-day period:
Operations Management Suite subscription entitlements
Customers who purchased Microsoft Operations Management Suite E1 and E2 are eligible for per-node data
ingestion entitlements for Log Analytics and Application Insights. To receive these entitlements for Log Analytics
workspaces or Application Insights resources in a given subscription:
Log Analytics workspaces should use the "Per-node (OMS )" pricing tier.
Application Insights resources should use the "Enterprise" pricing tier.
Depending on the number of nodes of the suite that your organization purchased, moving some subscriptions into
a Pay-As-You-Go (Per GB ) pricing tier could be advantageous, but this requires careful consideration.
WARNING
If your organization has current Microsoft Operations Management Suite E1 and E2, it is usually best to keep your Log
Analytics workspaces in the "Per-node (OMS)" pricing tier, and your Application Insights resources in the "Enterprise" pricing
tier.
Manage usage and costs for Application Insights
12/13/2019 • 20 minutes to read • Edit Online
NOTE
This article describes how to understand and control your costs for Application Insights. A related article, Monitoring usage
and estimated costs describes how to view usage and estimated costs across multiple Azure monitoring features for
different pricing models.
Application Insights is designed to get everything you need to monitor the availability, performance, and usage
of your web applications, whether they're hosted on Azure or on-premises. Application Insights supports popular
languages and frameworks, such as .NET, Java, and Node.js, and integrates with DevOps processes and tools like
Azure DevOps, Jira, and PagerDuty. It's important to understand what determines the costs of monitoring your
applications. In this article, we review what drives your application monitoring costs and how you can proactively
monitor and control them.
If you have questions about how pricing works for Application Insights, you can post a question in our forum.
Pricing model
The pricing for Azure Application Insights is a Pay-As-You-Go model based on data volume ingested and
optionally for longer data retention. Each Application Insights resource is charged as a separate service and
contributes to the bill for your Azure subscription. Data volume is measured as the size of the uncompressed
JSON data package that's received by Application Insights from your application. There is no data volume
charge for using the Live Metrics Stream.
Multi-step web tests incur an additional charge. Multi-step web tests are web tests that perform a sequence of
actions. There's no separate charge for ping tests of a single page. Telemetry from ping tests and multi-step tests
is charged the same as other telemetry from your app.
A. Review your data volume for the month. This includes all the data that's received and retained (after any
sampling) from your server and client apps, and from availability tests.
B. A separate charge is made for multi-step web tests. (This doesn't include simple availability tests, which are
included in the data volume charge.)
C. View data volume trends for the past month.
D. Enable data ingestion sampling.
E. Set the daily data volume cap.
(Note that all prices displayed in screenshots in this article are for example purposes only. For current prices in
your currency and region, see Application Insights pricing.)
To investigate your Application Insights usage more deeply, open the Metrics page, add the metric named "Data
point volume", and then select the Apply splitting option to split the data by "Telemetry item type".
Application Insights charges are added to your Azure bill. You can see details of your Azure bill in the Billing
section of the Azure portal, or in the Azure billing portal.
Using data volume metrics
To learn more about your data volumes, selecting Metrics for your Application Insights resource, add a new
chart. For the chart metric, under Log-based metrics, select Data point volume. Click Apply splitting, and
select group by Telemetryitem type.
systemEvents
| where timestamp >= ago(24h)
| where type == "Billing"
| extend BillingTelemetryType = tostring(dimensions["BillingTelemetryType"])
| extend BillingTelemetrySizeInBytes = todouble(measurements["BillingTelemetrySize"])
| summarize sum(BillingTelemetrySizeInBytes)
Or to see a chart of data volume (in bytes) by data type for the last 30 days, you can use:
systemEvents
| where timestamp >= startofday(ago(30d))
| where type == "Billing"
| extend BillingTelemetryType = tostring(dimensions["BillingTelemetryType"])
| extend BillingTelemetrySizeInBytes = todouble(measurements["BillingTelemetrySize"])
| summarize sum(BillingTelemetrySizeInBytes) by BillingTelemetryType, bin(timestamp, 1d) | render barchart
Note that this query can be used in an Azure Log Alert to set up alerting on data volumes.
To learn more about your telemetry data changes, let's check the count of events by type using the query:
systemEvents
| where timestamp >= startofday(ago(30d))
| where type == "Billing"
| extend BillingTelemetryType = tostring(dimensions["BillingTelemetryType"])
| summarize count() by BillingTelemetryType, bin(timestamp, 1d) | render barchart
If a similar changes are seen in the counts as is seen in the volume in bytes, then we can focus on the data types
of events, which show increased counts. For instance, if it is observed that the number of dependencies
increased, here's a query to understand which operations are responsible for the increase:
dependencies
| where timestamp >= startofday(ago(30d))
| summarize count() by operation_Name, bin(timestamp, 1d)
| render barchart
Sampling
Sampling is a method of reducing the rate at which telemetry is sent to your app, while retaining the ability to
find related events during diagnostic searches. You also retain correct event counts.
Sampling is an effective way to reduce charges and stay within your monthly quota. The sampling algorithm
retains related items of telemetry so, for example, when you use Search, you can find the request related to a
particular exception. The algorithm also retains correct counts so you see the correct values in Metric Explorer for
request rates, exception rates, and other counts.
There are several forms of sampling.
Adaptive sampling is the default for the ASP.NET SDK. Adaptive sampling automatically adjusts to the
volume of telemetry that your app sends. It operates automatically in the SDK in your web app so that
telemetry traffic on the network is reduced.
Ingestion sampling is an alternative that operates at the point where telemetry from your app enters the
Application Insights service. Ingestion sampling doesn't affect the volume of telemetry sent from your app,
but it reduces the volume that's retained by the service. You can use ingestion sampling to reduce the quota
that's used up by telemetry from browsers and other SDKs.
To set ingestion sampling, go to the Pricing pane:
WARNING
The Data sampling pane controls only the value of ingestion sampling. It doesn't reflect the sampling rate that's applied
by the Application Insights SDK in your app. If the incoming telemetry has already been sampled in the SDK, ingestion
sampling isn't applied.
To discover the actual sampling rate, no matter where it's been applied, use an Analytics query. The query looks
like this:
requests | where timestamp > ago(1d)
| summarize 100/avg(itemCount) by bin(timestamp, 1h)
| render areachart
In each retained record, itemCount indicates the number of original records that it represents. It's equal to 1 +
the number of previous discarded records.
The retention can also be set programatically using Powershell using the retentionInDays parameter.
Additionally, if you set the data retention to 30 days, you can trigger an immediate purge of older data using the
immediatePurgeDataOn30Days parameter, which may be useful for compliance-related scenarios. This purge
functionality is only exposed via Azure Resource Manager and should be used with extreme care. The daily reset
time for the data volume cap can be configured using Azure Resource Manager to set the dailyQuotaResetTime
parameter.
Limits summary
There are some limits on the number of metrics and events per application, that is, per instrumentation key.
Limits depend on the pricing plan that you choose.
Total data per day 100 GB You can reduce data by setting a cap. If
you need more data, you can increase
the limit in the portal, up to 1,000 GB.
For capacities greater than 1,000 GB,
send email to
AIDataCap@microsoft.com.
Availability multi-step test detailed 90 days This resource provides detailed results
results retention of each step.
For more information, see About pricing and quotas in Application Insights.
NOTE
These legacy pricing tiers have been renamed. The Enterprise pricing tier is now called Per Node and the Basic pricing tier
is now called Per GB. These new names are used below and in the Azure portal.
The Per Node (formerly Enterprise) tier has a per-node charge, and each node receives a daily data allowance. In
the Per Node pricing tier, you are charged for data ingested above the included allowance. If you are using
Operations Management Suite, you should choose the Per Node tier.
For current prices in your currency and region, see Application Insights pricing.
NOTE
In April 2018, we introduced a new pricing model for Azure monitoring. This model adopts a simple "pay-as-you-go"
model across the complete portfolio of monitoring services. Learn more about the new pricing model, how to assess the
impact of moving to this model based on your usage patterns, and how to opt into the new model
NOTE
To ensure that you get this entitlement, your Application Insights resources must be in the Per Node pricing tier. This
entitlement applies only as nodes. Application Insights resources in the Per GB tier don't realize any benefit. This
entitlement isn't visible in the estimated costs shown in the Usage and estimated cost pane. Also, if you move a
subscription to the new Azure monitoring pricing model in April 2018, the Per GB tier is the only tier available. Moving a
subscription to the new Azure monitoring pricing model isn't advisable if you have an Operations Management Suite
subscription.
The precise node counting depends on which Application Insights SDK your application is using.
In SDK versions 2.2 and later, both the Application Insights Core SDK and the Web SDK report each
application host as a node. Examples are the computer name for physical server and VM hosts or the
instance name for cloud services. The only exception is an application that uses only the .NET Core and
the Application Insights Core SDK. In that case, only one node is reported for all hosts because the
host name isn't available.
For earlier versions of the SDK, the Web SDK behaves like the newer SDK versions, but the Core SDK
reports only one node, regardless of the number of application hosts.
If your application uses the SDK to set roleInstance to a custom value, by default, that same value is
used to determine node count.
If you're using a new SDK version with an app that runs from client machines or mobile devices, the
node count might return a number that's large (because of the large number of client machines or
mobile devices).
Automation
You can write a script to set the pricing tier by using Azure Resource Management. Learn how.
Next steps
Sampling
Manage usage and costs with Azure Monitor Logs
1/23/2020 • 23 minutes to read • Edit Online
NOTE
This article describes how to understand and control your costs for Azure Monitor Logs. A related article, Monitoring usage
and estimated costs describes how to view usage and estimated costs across multiple Azure monitoring features for
different pricing models.
NOTE
All prices and costs shown in this article are for example purposes only.
Azure Monitor Logs is designed to scale and support collecting, indexing, and storing massive amounts of data
per day from any source in your enterprise or deployed in Azure. While this may be a primary driver for your
organization, cost-efficiency is ultimately the underlying driver. To that end, it's important to understand that the
cost of a Log Analytics workspace isn't based only on the volume of data collected, it is also dependent on the
plan selected, and how long you chose to store data generated from your connected sources.
In this article we review how you can proactively monitor ingested data volume and storage growth, and define
limits to control those associated costs.
Pricing model
The default pricing for Log Analytics is a Pay-As-You-Go model based on data volume ingested and optionally
for longer data retention. Data volume is measured as the size of the data that will be stored. Each Log Analytics
workspace is charged as a separate service and contributes to the bill for your Azure subscription. The amount of
data ingestion can be considerable depending on the following factors:
Number of management solutions enabled and their configuration (e.g.
Number of VMs monitored
Type of data collected from each monitored VM
In addition to the Pay-As-You-Go model, Log Analytics has Capacity Reservation tiers which enable you to
save as much as 25% compared to the Pay-As-You-Go price. The capacity reservation pricing enables you to buy
a reservation starting at 100 GB/day. Any usage above the reservation level will be billed at the Pay-As-You-Go
rate. The Capacity Reservation tiers have a 31-day commitment period. During the commitment period, you can
change to a higher level Capacity Reservation tier (which will restart the 31-day commitment period), but you
cannot move back to Pay-As-You-Go or to a lower Capacity Reservation tier until after the commitment period is
finished. Learn more about Log Analytics Pay-As-You-Go and Capacity Reservation pricing.
In all pricing tiers, the data volume is calculated from a string representation of the data as it is prepared to be
stored. Several properties common to all data types are not included in the calculation of the event size, including
_ResourceId , _ItemId , _IsBillable and _BilledSize .
Also, note that Some solutions, such as Azure Security Center and Azure Sentinel, have their own pricing model.
To explore your data in more detail, click on the icon at the top right of either of the charts on the Usage and
Estimated Costs page. Now you can work with this query to explore more details of your usage.
From the Usage and Estimated Costs page you can review your data volume for the month. This includes all
the data received and retained in your Log Analytics workspace. Click Usage details from the top of the page, to
view the usage dashboard with information on data volume trends by source, computers, and offering. To view
and set a daily cap or to modify the retention period, click Data volume management.
Log Analytics charges are added to your Azure bill. You can see details of your Azure bill under the Billing section
of the Azure portal or in the Azure Billing Portal.
NOTE
To use the entitlements that come from purchasing OMS E1 Suite, OMS E2 Suite or OMS Add-On for System Center,
choose the Log Analytics Per Node pricing tier.
The retention can also be set via Azure Resource Manager using the retentionInDays parameter. Additionally, if
you set the data retention to 30 days, you can trigger an immediate purge of older data using the
immediatePurgeDataOn30Days parameter, which may be useful for compliance-related scenarios. This functionality
is only exposed via Azure Resource Manager.
Two data types -- Usage and AzureActivity -- are retained for 90 days by default, and there is no charge for for
this 90 day retention. These data types are also free from data ingestion charges.
Retention by data type
It is also possible to specify different retention settings for individual data types. Each data type is a sub-resource
of the workspace. For instance the SecurityEvent table can be addressed in Azure Resource Manager as:
/subscriptions/00000000-0000-0000-0000-
00000000000/resourceGroups/MyResourceGroupName/providers/Microsoft.OperationalInsights/workspaces/MyWorkspace
Name/Tables/SecurityEvent
Note that the data type (table) is case sensitive. To get the current per data type retention settings of a particular
data type (in this example SecurityEvent), use:
GET /subscriptions/00000000-0000-0000-0000-
00000000000/resourceGroups/MyResourceGroupName/providers/Microsoft.OperationalInsights/workspaces/MyWorkspace
Name/Tables/SecurityEvent?api-version=2017-04-26-preview
To get the current per data type retention settings for all data types in your workspace, just omit the specific data
type, for example:
GET /subscriptions/00000000-0000-0000-0000-
00000000000/resourceGroups/MyResourceGroupName/providers/Microsoft.OperationalInsights/workspaces/MyWorkspace
Name/Tables?api-version=2017-04-26-preview
To set the retention of a particular data type (in this example SecurityEvent) to 730 days, do
PUT /subscriptions/00000000-0000-0000-0000-
00000000000/resourceGroups/MyResourceGroupName/providers/Microsoft.OperationalInsights/workspaces/MyWorkspace
Name/Tables/SecurityEvent?api-version=2017-04-26-preview
{
"properties":
{
"retentionInDays": 730
}
}
The Usage and AzureActivity data types cannot be set with custom retention. They will take on the maximum of
the default workspace retention or 90 days.
A great tool to connect directly to Azure Resource Manager to set retention by data type is the OSS tool
ARMclient. Learn more about ARMclient from articles by David Ebbo and Daniel Bowbyes. Here's an example
using ARMClient, setting SecurityEvent data to a 730 day retention:
NOTE
Setting retention on individual data types can be used to reduce your costs for data retention. For data collected starting in
October 2019 (when this feature was released), reducing the retention for some data types can reduce your retention cost
over time. For data collected earlier, setting a lower retention for an individual type will not affect your retention costs.
NOTE
The daily cap does not stop the collection of data from Azure Security Center, except for workspaces in which Azure
Security Center was installed before June 19, 2017.
NOTE
Latency inherent in applying the daily cap can mean that the cap is not applied as precisely the specified daily cap level.
To get a list of computers which will be billed as nodes if the workspace is in the legacy Per Node pricing tier, look
for nodes which are sending billed data types (some data types are free). To do this, use the _IsBillable
property and use the leftmost field of the fully qualified domain name. This returns the list of computers with
billed data:
union withsource = tt *
| where _IsBillable == true
| extend computerName = tolower(tostring(split(Computer, '.')[0]))
| where computerName != ""
| summarize TotalVolumeBytes=sum(_BilledSize) by computerName
union withsource = tt *
| where _IsBillable == true
| extend computerName = tolower(tostring(split(Computer, '.')[0]))
| where computerName != ""
| summarize billableNodes=dcount(computerName)
NOTE
Use these union withsource = tt * queries sparingly as scans across data types are expensive to execute. This query
replaces the old way of querying per-computer information with the Usage data type.
A more accurate calculation of what will actually be billed is to get the count of computers per hour that are
sending billed data types. (For workspaces in the legacy Per Node pricing tier, Log Analytics calculates the
number of nodes which need to be billed on an hourly basis.)
union withsource = tt *
| where _IsBillable == true
| extend computerName = tolower(tostring(split(Computer, '.')[0]))
| where computerName != ""
| summarize billableNodes=dcount(computerName) by bin(TimeGenerated, 1h) | sort by TimeGenerated asc
Understanding ingested data volume
On the Usage and Estimated Costs page, the Data ingestion per solution chart shows the total volume of data
sent and how much is being sent by each solution. This allows you to determine trends such as whether the
overall data usage (or usage by a particular solution) is growing, remaining steady or decreasing. The query used
to generate this is
Note that the clause "where IsBillable = true" filters out data types from certain solutions for which there is no
ingestion charge.
You can drill in further to see data trends for specific data types, for example if you want to study the data due to
IIS logs:
union withsource = tt *
| where _IsBillable == true
| extend computerName = tolower(tostring(split(Computer, '.')[0]))
| summarize Bytes=sum(_BilledSize) by computerName | sort by Bytes nulls last
The _IsBillable property specifies whether the ingested data will incur charges.
To see the count of billable events ingested per computer, use
union withsource = tt *
| where _IsBillable == true
| extend computerName = tolower(tostring(split(Computer, '.')[0]))
| summarize eventCount=count() by computerName | sort by eventCount nulls last
If you want to see counts for billable data types are sending data to a specific computer, use:
union withsource = tt *
| where Computer == "computer name"
| where _IsBillable == true
| summarize count() by tt | sort by count_ nulls last
union withsource = tt *
| where _IsBillable == true
| summarize Bytes=sum(_BilledSize) by _ResourceId | sort by Bytes nulls last
For data from nodes hosted in Azure you can get the size of billable events ingested per Azure subscription,
parse the _ResourceId property as:
union withsource = tt *
| where _IsBillable == true
| parse tolower(_ResourceId) with "/subscriptions/" subscriptionId "/resourcegroups/"
resourceGroup "/providers/" provider "/" resourceType "/" resourceName
| summarize Bytes=sum(_BilledSize) by subscriptionId | sort by Bytes nulls last
Changing subscriptionId to resourceGroup will show the billable ingested data volume by Azure resource
group.
NOTE
Some of the fields of the Usage data type, while still in the schema, have been deprecated and will their values are no
longer populated. These are Computer as well as fields related to ingestion (TotalBatches, BatchesWithinSla,
BatchesOutsideSla, BatchesCapped and AverageProcessingTimeMs.
Solution data from computers that don't need the solution Use solution targeting to collect data from only required
groups of computers.
union
(
Heartbeat
| where (Solutions has 'security' or Solutions has 'antimalware' or Solutions has 'securitycenter')
| project Computer
),
(
ProtectionStatus
| where Computer !in~
(
(
Heartbeat
| project Computer
)
)
| project Computer
)
| distinct Computer
| project lowComputer = tolower(Computer)
| distinct lowComputer
| count
The following query uses a simple formula to predict when more than 100 GB of data will be sent in a day:
To alert on a different data volume, change the 100 in the queries to the number of GB you want to alert on.
Use the steps described in create a new log alert to be notified when data collection is higher than expected.
When creating the alert for the first query -- when there is more than 100 GB of data in 24 hours, set the:
Define alert condition specify your Log Analytics workspace as the resource target.
Alert criteria specify the following:
Signal Name select Custom log search
Search query to
union withsource = $table Usage | where QuantityUnit == "MBytes" and
iff(isnotnull(toint(IsBillable)), IsBillable == true, IsBillable == "true") == true | extend Type =
$table | summarize DataGB = sum((Quantity / 1000.)) by Type | where DataGB > 100
Alert logic is Based on number of results and Condition is Greater than a Threshold of 0
Time period of 1440 minutes and Alert frequency to every 60 minutes since the usage data only
updates once per hour.
Define alert details specify the following:
Name to Data volume greater than 100 GB in 24 hours
Severity to Warning
Specify an existing or create a new Action Group so that when the log alert matches criteria, you are notified.
When creating the alert for the second query -- when it is predicted that there will be more than 100 GB of data
in 24 hours, set the:
Define alert condition specify your Log Analytics workspace as the resource target.
Alert criteria specify the following:
Signal Name select Custom log search
Search query to
union withsource = $table Usage | where QuantityUnit == "MBytes" and
iff(isnotnull(toint(IsBillable)), IsBillable == true, IsBillable == "true") == true | extend Type =
$table | summarize EstimatedGB = sum(((Quantity * 8) / 1000.)) by Type | where EstimatedGB > 100
Alert logic is Based on number of results and Condition is Greater than a Threshold of 0
Time period of 180 minutes and Alert frequency to every 60 minutes since the usage data only
updates once per hour.
Define alert details specify the following:
Name to Data volume expected to greater than 100 GB in 24 hours
Severity to Warning
Specify an existing or create a new Action Group so that when the log alert matches criteria, you are notified.
When you receive an alert, use the steps in the following section to troubleshoot why usage is higher than
expected.
When data collection stops, the OperationStatus is Warning. When data collection starts, the OperationStatus is
Succeeded. The following table describes reasons that data collection stops and a suggested action to resume
data collection:
Daily limit of legacy Free pricing tier reached Wait until the following day for collection to automatically
restart, or change to a paid pricing tier.
Daily cap of your workspace was reached Wait for collection to automatically restart, or increase the
daily data volume limit described in manage the maximum
daily data volume. The daily cap reset time is shows on the
Data volume management page.
REASON COLLECTION STOPS SOLUTION
To be notified when data collection stops, use the steps described in Create daily data cap alert to be notified
when data collection stops. Use the steps described in create an action group to configure an e-mail, webhook, or
runbook action for the alert rule.
Limits summary
There are some additional Log Analytics limits, some of which depend on the Log Analytics pricing tier. These are
documented here.
Next steps
See Log searches in Azure Monitor Logs to learn how to use the search language. You can use search queries
to perform additional analysis on the usage data.
Use the steps described in create a new log alert to be notified when a search criteria is met.
Use solution targeting to collect data from only required groups of computers.
To configure an effective event collection policy, review Azure Security Center filtering policy.
Change performance counter configuration.
To modify your event collection settings, review event log configuration.
To modify your syslog collection settings, review syslog configuration.
What is Application Insights?
12/16/2019 • 5 minutes to read • Edit Online
In addition, you can pull in telemetry from the host environments such as performance counters,
Azure diagnostics, or Docker logs. You can also set up web tests that periodically send synthetic
requests to your web service.
All these telemetry streams are integrated into Azure Monitor. In the Azure portal, you can apply
powerful analytic and search tools to the raw data.
What's the overhead?
The impact on your app's performance is very small. Tracking calls are non-blocking, and are batched
and sent in a separate thread.
What does Application Insights monitor?
Application Insights is aimed at the development team, to help you understand how your app is
performing and how it's being used. It monitors:
Request rates, response times, and failure rates - Find out which pages are most popular, at
what times of day, and where your users are. See which pages perform best. If your response times
and failure rates go high when there are more requests, then perhaps you have a resourcing
problem.
Dependency rates, response times, and failure rates - Find out whether external services are
slowing you down.
Exceptions - Analyze the aggregated statistics, or pick specific instances and drill into the stack
trace and related requests. Both server and browser exceptions are reported.
Page views and load performance - reported by your users' browsers.
AJAX calls from web pages - rates, response times, and failure rates.
User and session counts.
Performance counters from your Windows or Linux server machines, such as CPU, memory, and
network usage.
Host diagnostics from Docker or Azure.
Diagnostic trace logs from your app - so that you can correlate trace events with requests.
Custom events and metrics that you write yourself in the client or server code, to track business
events such as items sold or games won.
Application map
Explore the components of your app, with key metrics
and alerts.
Profiler
Inspect the execution profiles of sampled requests.
Usage analysis
Analyze user segmentation and retention.
Diagnostic search for instance data
Search and filter events such as requests, exceptions,
dependency calls, log traces, and page views.
Dashboards
Mash up data from multiple resources and share with
others. Great for multi-component applications, and for
continuous display in the team room.
Analytics
Answer tough questions about your app's performance
and usage by using this powerful query language.
Visual Studio
See performance data in the code. Go to code from
stack traces.
Snapshot debugger
Debug snapshots sampled from live operations, with
parameter values.
Power BI
Integrate usage metrics with other business
intelligence.
REST API
Write code to run queries over your metrics and raw
data.
Continuous export
Bulk export of raw data to storage as soon as it arrives.
Get started
Application Insights is one of the many services hosted within Microsoft Azure, and telemetry is sent
there for analysis and presentation. So before you do anything else, you'll need a subscription to
Microsoft Azure. It's free to sign up, and if you choose the basic pricing plan of Application Insights,
there's no charge until your application has grown to have substantial usage. If your organization
already has a subscription, they could add your Microsoft account to it.
There are several ways to get started. Begin with whichever works best for you. You can add the others
later.
At run time: instrument your web app on the server. Ideal for applications already deployed.
Avoids any update to the code.
ASP.NET or ASP.NET Core applications hosted on Azure Web Apps
ASP.NET applications hosted in IIS on Azure VM or Azure virtual machine scale set
ASP.NET applications hosted in IIS on-premises VM
At development time: add Application Insights to your code. Allows you to customize
telemetry collection and send additional telemetry.
ASP.NET Applications
ASP.NET Core Applications
.NET Console Applications
Java
Node.js
Python (preview )
Other platforms
Instrument your web pages for page view, AJAX, and other client-side telemetry.
Analyze mobile app usage by integrating with Visual Studio App Center.
Availability tests - ping your website regularly from our servers.
Next steps
Get started at runtime with:
Azure VM and Azure virtual machine scale set IIS -hosted apps
IIS server
Azure Web Apps
Get started at development time with:
ASP.NET
ASP.NET Core
Java
Node.js
Python (preview )
This article describes how to enable Application Insights for an ASP.NET Core application. When you complete
the instructions in this article, Application Insights will collect requests, dependencies, exceptions, performance
counters, heartbeats, and logs from your ASP.NET Core application.
The example we'll use here is an MVC application that targets netcoreapp2.2 . You can apply these instructions to
all ASP.NET Core applications.
Supported scenarios
The Application Insights SDK for ASP.NET Core can monitor your applications no matter where or how they
run. If your application is running and has network connectivity to Azure, telemetry can be collected. Application
Insights monitoring is supported everywhere .NET Core is supported. Support covers:
Operating system: Windows, Linux, or Mac.
Hosting method: In process or out of process.
Deployment method: Framework dependent or self-contained.
Web server: IIS (Internet Information Server) or Kestrel.
Hosting platform: The Web Apps feature of Azure App Service, Azure VM, Docker, Azure Kubernetes
Service (AKS ), and so on.
.NET Core Runtime version: 1.XX, 2.XX, or 3.XX
IDE: Visual Studio, VS Code, or command line.
NOTE
If you are using ASP.NET Core 3.0 along with Application Insights, please use the 2.8.0 version or higher. This is the only
version that supports ASP.NET Core 3.0.
Prerequisites
A functioning ASP.NET Core application. If you need to create an ASP.NET Core application, follow this
ASP.NET Core tutorial.
A valid Application Insights instrumentation key. This key is required to send any telemetry to Application
Insights. If you need to create a new Application Insights resource to get an instrumentation key, see Create
an Application Insights resource.
TIP
If you want to, you can set up source control for your project so you can track all the changes that Application
Insights makes. To enable source control, select File > Add to Source Control.
6. If you followed the optional tip and added your project to source control, go to View > Team Explorer >
Changes. Then select each file to see a diff view of the changes made by Application Insights telemetry.
<ItemGroup>
<PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.12.0" />
</ItemGroup>
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
// The following line enables Application Insights telemetry collection.
services.AddApplicationInsightsTelemetry();
Alternatively, specify the instrumentation key in either of the following environment variables:
APPINSIGHTS_INSTRUMENTATIONKEY
ApplicationInsights:InstrumentationKey
For example:
SET ApplicationInsights:InstrumentationKey=putinstrumentationkeyhere
SET APPINSIGHTS_INSTRUMENTATIONKEY=putinstrumentationkeyhere
NOTE
An instrumentation key specified in code wins over the environment variable APPINSIGHTS_INSTRUMENTATIONKEY ,
which wins over other options.
2. In _Layout.cshtml , insert HtmlHelper at the end of the <head> section but before any other script. If you
want to report any custom JavaScript telemetry from the page, inject it after this snippet:
@Html.Raw(JavaScriptSnippet.FullScript)
</head>
The .cshtml file names referenced earlier are from a default MVC application template. Ultimately, if you want
to properly enable client-side monitoring for your application, the JavaScript snippet must appear in the <head>
section of each page of your application that you want to monitor. You can accomplish this goal for this
application template by adding the JavaScript snippet to _Layout.cshtml .
If your project doesn't include _Layout.cshtml , you can still add client-side monitoring. You can do this by adding
the JavaScript snippet to an equivalent file that controls the <head> of all pages within your app. Or you can add
the snippet to multiple pages, but this solution is difficult to maintain and we generally don't recommend it.
NOTE
In ASP.NET Core applications, changing configuration by modifying TelemetryConfiguration.Active isn't supported.
Using ApplicationInsightsServiceOptions
You can modify a few common settings by passing ApplicationInsightsServiceOptions to
AddApplicationInsightsTelemetry , as in this example:
public void ConfigureServices(IServiceCollection services)
{
Microsoft.ApplicationInsights.AspNetCore.Extensions.ApplicationInsightsServiceOptions aiOptions
= new
Microsoft.ApplicationInsights.AspNetCore.Extensions.ApplicationInsightsServiceOptions();
// Disables adaptive sampling.
aiOptions.EnableAdaptiveSampling = false;
See the configurable settings in ApplicationInsightsServiceOptions for the most up-to-date list.
Sampling
The Application Insights SDK for ASP.NET Core supports both fixed-rate and adaptive sampling. Adaptive
sampling is enabled by default.
For more information, see Configure adaptive sampling for ASP.NET Core applications.
Adding TelemetryInitializers
Use telemetry initializers when you want to define global properties that are sent with all telemetry.
Add any new TelemetryInitializer to the DependencyInjection container as shown in the following code. The
SDK automatically picks up any TelemetryInitializer that's added to the DependencyInjection container.
// The following removes all default counters from EventCounterCollectionModule, and adds a single one.
services.ConfigureTelemetryModule<EventCounterCollectionModule>(
(module, o) =>
{
module.Counters.Clear();
module.Counters.Add(new EventCounterCollectionRequest("System.Runtime", "gen-0-size"));
}
);
using Microsoft.ApplicationInsights.Channel;
services.AddApplicationInsightsTelemetry();
}
The above does not prevent any auto collection modules from collecting telemetry. Only the sending of
telemetry to Application Insights gets disabled with the above approach. If a particular auto collection module is
not desired, it is best to remove the telemetry module
using Microsoft.ApplicationInsights;
For more information about custom data reporting in Application Insights, see Application Insights custom
metrics API reference.
Some Visual Studio templates used the UseApplicationInsights() extension method on IWebHostBuilder to
enable Application Insights. Is this usage still valid?
While the extension method UseApplicationInsights() is still supported, it is marked obsolete in Application
Insights SDK version 2.8.0 onwards. It will be removed in the next major version of the SDK. The recommended
way to enable Application Insights telemetry is by using AddApplicationInsightsTelemetry() because it provides
overloads to control some configuration. Also, in ASP.NET Core 3.0 apps,
services.AddApplicationInsightsTelemetry() is the only way to enable application insights.
I'm deploying my ASP.NET Core application to Web Apps. Should I still enable the Application Insights
extension from Web Apps?
If the SDK is installed at build time as shown in this article, you don't need to enable the Application Insights
extension from the App Service portal. Even if the extension is installed, it will back off when it detects that the
SDK is already added to the application. If you enable Application Insights from the extension, you don't have to
install and update the SDK. But if you enable Application Insights by following instructions in this article, you
have more flexibility because:
Application Insights telemetry will continue to work in:
All operating systems, including Windows, Linux, and Mac.
All publish modes, including self-contained or framework dependent.
All target frameworks, including the full .NET Framework.
All hosting options, including Web Apps, VMs, Linux, containers, Azure Kubernetes Service, and non-
Azure hosting.
All .NET Core versions including preview versions.
You can see telemetry locally when you're debugging from Visual Studio.
You can track additional custom telemetry by using the TrackXXX() API.
You have full control over the configuration.
Can I enable Application Insights monitoring by using tools like Status Monitor?
No. Status Monitor and Status Monitor v2 currently support ASP.NET 4.x only.
Is Application Insights automatically enabled for my ASP.NET Core 2.0 application?
The Microsoft.AspNetCore.All 2.0 metapackage included the Application Insights SDK (version 2.1.0). If you run
the application under Visual Studio debugger, Visual Studio enables Application Insights and shows telemetry
locally in the IDE itself. Telemetry wasn't sent to the Application Insights service unless an instrumentation key
was specified. We recommend following the instructions in this article to enable Application Insights, even for 2.0
apps.
If I run my application in Linux, are all features supported?
Yes. Feature support for the SDK is the same in all platforms, with the following exceptions:
Performance counters are supported only in Windows.
Even though ServerTelemetryChannel is enabled by default, if the application is running in Linux or MacOS,
the channel doesn't automatically create a local storage folder to keep telemetry temporarily if there are
network issues. Because of this limitation, telemetry is lost when there are temporary network or server
issues. To work around this issue, configure a local folder for the channel:
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;
Is this SDK supported for the new .NET Core 3.0 Worker Service template applications?
This SDK requires HttpContext , and hence does not work in any non-HTTP applications, including the .NET
Core 3.0 Worker Service applications. Refer to this document for enabling application insights in such
applications, using the newly released Microsoft.ApplicationInsights.WorkerService SDK.
Open-source SDK
Read and contribute to the code.
Video
Check out this external step-by-step video to configure Application Insights with .NET Core and Visual Studio
from scratch.
Check out this external step-by-step video to configure Application Insights with .NET Core and Visual Studio
Code from scratch.
Next steps
Explore user flows to understand how users navigate through your app.
Configure a snapshot collection to see the state of source code and variables at the moment an exception is
thrown.
Use the API to send your own events and metrics for a detailed view of your app's performance and usage.
Use availability tests to check your app constantly from around the world.
Dependency Injection in ASP.NET Core
Set up Application Insights for your ASP.NET
website
10/20/2019 • 5 minutes to read • Edit Online
This procedure configures your ASP.NET web app to send telemetry to the Azure Application Insights
service. It works for ASP.NET apps that are hosted either in your own IIS server on-premises or in the
Cloud. You get charts and a powerful query language that help you understand the performance of your
app and how people are using it, plus automatic alerts on failures or performance issues. Many
developers find these features great as they are, but you can also extend and customize the telemetry if
you need to.
Setup takes just a few clicks in Visual Studio. You have the option to avoid charges by limiting the volume
of telemetry. This functionality allows you to experiment and debug, or to monitor a site with not many
users. When you decide you want to go ahead and monitor your production site, it's easy to raise the limit
later.
Prerequisites
To add Application Insights to your ASP.NET website, you need to:
Install Visual Studio 2019 for Windows with the following workloads:
ASP.NET and web development (Do not uncheck the optional components)
Azure development
If you don't have an Azure subscription, create a free account before you begin.
Right-click your web app name in the Solution Explorer, and choose Add > Application Insights
Telemetry
(Depending on your Application Insights SDK version you may be prompted to upgrade to the latest SDK
release. If prompted, select Update SDK.)
NOTE
If you don't want to send telemetry to the portal while you're debugging, just add the Application Insights SDK to
your app but don't configure a resource in the portal. You are able to see telemetry in Visual Studio while you are
debugging. Later, you can return to this configuration page, or you could wait until after you have deployed your
app and switch on telemetry at run time.
NOTE
If your app sends enough telemetry to approach the throttling limits, automatic sampling switches on. Sampling
reduces the quantity of telemetry sent from your app, while preserving correlated data for diagnostic purposes.
Video
External step-by-step video about configuring Application Insights with a .NET application from
scratch.
Next steps
There are alternative topics to look at if you are interested in:
Instrumenting a web app at runtime
Azure Cloud Services
More telemetry
Browser and page load data - Insert a code snippet in your web pages.
Get more detailed dependency and exception monitoring - Install Status Monitor on your server.
Code custom events to count, time, or measure user actions.
Get log data - Correlate log data with your telemetry.
Analysis
Working with Application Insights in Visual Studio
Includes information about debugging with telemetry, diagnostic search, and drill through to code.
Analytics - The powerful query language.
Alerts
Availability tests: Create tests to make sure your site is visible on the web.
Smart diagnostics: These tests run automatically, so you don't have to do anything to set them up. They
tell you if your app has an unusual rate of failed requests.
Metric alerts: Set alerts to warn you if a metric crosses a threshold. You can set them on custom
metrics that you code into your app.
Automation
Automate creating an Application Insights resource
Application Insights for Worker Service applications
(non-HTTP applications)
12/17/2019 • 13 minutes to read • Edit Online
Application Insights is releasing a new SDK, called Microsoft.ApplicationInsights.WorkerService , which is best suited
for non-HTTP workloads like messaging, background tasks, console applications etc. These types of applications don't
have the notion of an incoming HTTP request like a traditional ASP.NET/ASP.NET Core Web Application, and hence
using Application Insights packages for ASP.NET or ASP.NET Core applications is not supported.
The new SDK does not do any telemetry collection by itself. Instead, it brings in other well known Application Insights
auto collectors like DependencyCollector, PerfCounterCollector, ApplicationInsightsLoggingProvider etc. This SDK
exposes extension methods on IServiceCollection to enable and configure telemetry collection.
Supported scenarios
The Application Insights SDK for Worker Service is best suited for non-HTTP applications no matter where or how
they run. If your application is running and has network connectivity to Azure, telemetry can be collected. Application
Insights monitoring is supported everywhere .NET Core is supported. This package can be used in the newly
introduced .NET Core 3.0 Worker Service, background tasks in Asp.Net Core 2.1/2.2, Console apps (.NET Core/ .NET
Framework), etc.
Prerequisites
A valid Application Insights instrumentation key. This key is required to send any telemetry to Application Insights. If
you need to create a new Application Insights resource to get an instrumentation key, see Create an Application
Insights resource.
<ItemGroup>
<PackageReference Include="Microsoft.ApplicationInsights.WorkerService" Version="2.12.0" />
</ItemGroup>
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
using (_telemetryClient.StartOperation<RequestTelemetry>("operation"))
{
_logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher
is captured by Application Insights");
_logger.LogInformation("Calling bing.com");
var res = await _httpClient.GetAsync("https://bing.com");
_logger.LogInformation("Calling bing completed with status:" + res.StatusCode);
_telemetryClient.TrackEvent("Bing call event completed");
}
Alternatively, specify the instrumentation key in either of the following environment variables.
APPINSIGHTS_INSTRUMENTATIONKEY or ApplicationInsights:InstrumentationKey
Typically, APPINSIGHTS_INSTRUMENTATIONKEY specifies the instrumentation key for applications deployed to Web Apps as
Web Jobs.
NOTE
An instrumentation key specified in code wins over the environment variable APPINSIGHTS_INSTRUMENTATIONKEY , which wins
over other options.
using (host)
{
// Start the host
await host.StartAsync();
Following is the code for TimedHostedService where the background task logic resides.
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
return Task.CompletedTask;
}
using (_telemetryClient.StartOperation<RequestTelemetry>("operation"))
{
_logger.LogWarning("A sample warning message. By default, logs with severity Warning or higher is
captured by Application Insights");
_logger.LogInformation("Calling bing.com");
var res = await httpClient.GetAsync("https://bing.com");
_logger.LogInformation("Calling bing completed with status:" + res.StatusCode);
telemetryClient.TrackEvent("Bing call event completed");
}
}
}
3. Set up the instrumentation key. Use the same appsettings.json from the .NET Core 3.0 Worker Service example
above.
namespace WorkerSDKOnConsole
{
class Program
{
static async Task Main(string[] args)
{
// Create the DI container.
IServiceCollection services = new ServiceCollection();
// Build ServiceProvider.
IServiceProvider serviceProvider = services.BuildServiceProvider();
// Obtain TelemetryClient instance from DI, for additional manual tracking or to flush.
var telemetryClient = serviceProvider.GetRequiredService<TelemetryClient>();
while (true) // This app runs indefinitely. replace with actual application termination logic.
{
logger.LogInformation("Worker running at: {time}", DateTimeOffset.Now);
await Task.Delay(1000);
}
This console application also uses the same default TelemetryConfiguration , and it can be customized in the same way
as the examples in earlier section.
NOTE
While using this SDK, changing configuration by modifying TelemetryConfiguration.Active isn't supported, and changes will
not be reflected.
Using ApplicationInsightsServiceOptions
You can modify a few common settings by passing ApplicationInsightsServiceOptions to
AddApplicationInsightsTelemetryWorkerService , as in this example:
using Microsoft.ApplicationInsights.WorkerService;
See the configurable settings in ApplicationInsightsServiceOptions for the most up-to-date list.
Sampling
The Application Insights SDK for Worker Service supports both fixed-rate and adaptive sampling. Adaptive sampling
is enabled by default. Configuring sampling for Worker Service is done the same way as for ASP.NET Core
Applications.
Adding TelemetryInitializers
Use telemetry initializers when you want to define properties that are sent with all telemetry.
Add any new TelemetryInitializer to the DependencyInjection container and SDK will automatically add them to the
TelemetryConfiguration .
using Microsoft.ApplicationInsights.Extensibility;
Removing TelemetryInitializers
Telemetry initializers are present by default. To remove all or specific telemetry initializers, use the following sample
code after you call AddApplicationInsightsTelemetryWorkerService() .
using Microsoft.ApplicationInsights.Channel;
services.AddApplicationInsightsTelemetryWorkerService();
}
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;
Sample applications
.NET Core Console Application Use this sample if you are using a Console Application written in either .NET Core (2.0
or higher) or .NET Framework (4.7.2 or higher)
ASP .NET Core background tasks with HostedServices Use this sample if you are in Asp.Net Core 2.1/2.2, and
creating background tasks as per official guidance here
.NET Core 3.0 Worker Service Use this sample if you have a .NET Core 3.0 Worker Service application as per official
guidance here
Open-source SDK
Read and contribute to the code.
Next steps
Use the API to send your own events and metrics for a detailed view of your app's performance and usage.
Track additional dependencies not automatically tracked.
Enrich or Filter auto collected telemetry.
Dependency Injection in ASP.NET Core.
Application Insights for .NET console applications
12/5/2019 • 4 minutes to read • Edit Online
Application Insights lets you monitor your web application for availability, performance, and usage.
You need a subscription with Microsoft Azure. Sign in with a Microsoft account, which you might have for
Windows, Xbox Live, or other Microsoft cloud services. Your team might have an organizational subscription to
Azure: ask the owner to add you to it using your Microsoft account.
NOTE
There is a new Application Insights SDK called Microsoft.ApplicationInsights.WorkerService which can used to enable
Application Insights for any Console Applications. It is recommended to use this package and associated instructions from
here. This package targets NetStandard2.0 , and hence can be used in .NET Core 2.0 or higher, and .NET Framework 4.7.2
or higher.
Getting started
In the Azure portal, create an Application Insights resource. For application type, choose General.
Take a copy of the Instrumentation Key. Find the key in the Essentials drop-down of the new resource you
created.
Install latest Microsoft.ApplicationInsights package.
Set the instrumentation key in your code before tracking any telemetry (or set
APPINSIGHTS_INSTRUMENTATIONKEY environment variable). After that, you should be able to manually
track telemetry and see it on the Azure portal
// you may use different options to create configuration as shown later in this article
TelemetryConfiguration configuration = TelemetryConfiguration.CreateDefault();
configuration.InstrumentationKey = " *your key* ";
var telemetryClient = new TelemetryClient(configuration);
telemetryClient.TrackTrace("Hello World!");
NOTE
Telemetry is not sent instantly. Telemetry items are batched and sent by the ApplicationInsights SDK. In Console apps, which
exits right after calling Track() methods, telemetry may not be sent unless Flush() and Sleep is done before the app
exits as shown in full example later in this article.
NOTE
Instructions referring to ApplicationInsights.config are only applicable to apps that are targeting the .NET Framework, and
do not apply to .NET Core applications.
Using config file
By default, Application Insights SDK looks for ApplicationInsights.config file in the working directory when
TelemetryConfiguration is being created
using System.IO;
TelemetryConfiguration configuration =
TelemetryConfiguration.CreateFromConfiguration(File.ReadAllText("C:\\ApplicationInsights.config"));
var telemetryClient = new TelemetryClient(configuration);
NOTE
Reading config file is not supported on .NET Core. You may consider using Application Insights SDK for ASP.NET Core
// prevent Correlation Id to be sent to certain endpoints. You may add other domains as needed.
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.windows.net");
//...
// enable known dependency tracking, note that in future versions, we will extend this list.
// please check default settings in https://github.com/Microsoft/ApplicationInsights-dotnet-
server/blob/develop/Src/DependencyCollector/DependencyCollector/ApplicationInsights.config.install.xdt
module.IncludeDiagnosticSourceActivities.Add("Microsoft.Azure.ServiceBus");
module.IncludeDiagnosticSourceActivities.Add("Microsoft.Azure.EventHubs");
//....
If you created configuration with plain TelemetryConfiguration() constructor, you need to enable correlation
support additionally. It is not needed if you read configuration from file, used
TelemetryConfiguration.CreateDefault() or TelemetryConfiguration.Active .
configuration.TelemetryInitializers.Add(new OperationCorrelationTelemetryInitializer());
You may also want to install and initialize Performance Counter collector module as described here
Full example
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DependencyCollector;
using Microsoft.ApplicationInsights.Extensibility;
using System.Net.Http;
using System.Threading.Tasks;
namespace ConsoleApp
{
class Program
{
static void Main(string[] args)
{
TelemetryConfiguration configuration = TelemetryConfiguration.CreateDefault();
configuration.InstrumentationKey = "removed";
configuration.TelemetryInitializers.Add(new HttpDependenciesParsingTelemetryInitializer());
telemetryClient.TrackTrace("Hello World!");
// prevent Correlation Id to be sent to certain endpoints. You may add other domains as needed.
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.windows.net");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.chinacloudapi.cn");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.cloudapi.de");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("core.usgovcloudapi.net");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("localhost");
module.ExcludeComponentCorrelationHttpHeadersOnDomains.Add("127.0.0.1");
// enable known dependency tracking, note that in future versions, we will extend this list.
// please check default settings in https://github.com/microsoft/ApplicationInsights-dotnet-
server/blob/develop/WEB/Src/DependencyCollector/DependencyCollector/ApplicationInsights.config.install.xdt
module.IncludeDiagnosticSourceActivities.Add("Microsoft.Azure.ServiceBus");
module.IncludeDiagnosticSourceActivities.Add("Microsoft.Azure.EventHubs");
return module;
}
}
}
Next steps
Monitor dependencies to see if REST, SQL, or other external resources are slowing you down.
Use the API to send your own events and metrics for a more detailed view of your app's performance and
usage.
ApplicationInsightsLoggerProvider for .NET Core
ILogger logs
1/23/2020 • 12 minutes to read • Edit Online
ASP.NET Core supports a logging API that works with different kinds of built-in and third-party logging providers.
Logging is done by calling Log() or a variant of it on ILogger instances. This article demonstrates how to use
ApplicationInsightsLoggerProvider to capture ILogger logs in console and ASP.NET Core applications. This article
also describes how ApplicationInsightsLoggerProvider integrates with other Application Insights telemetry. To
learn more, see Logging in ASP.NET Core.
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Logging.ApplicationInsights" Version="2.9.1" />
</ItemGroup>
// Optional: Apply filters to control what logs are sent to Application Insights.
// The following configures LogLevel Information or above to be sent to
// Application Insights for all categories.
builder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>
("", LogLevel.Information);
}
);
}
The code in step 2 configures ApplicationInsightsLoggerProvider . The following code shows an example Controller
class, which uses ILogger to send logs. The logs are captured by Application Insights.
// GET api/values
[HttpGet]
public ActionResult<IEnumerable<string>> Get()
{
// All the following logs will be picked up by Application Insights.
// and all of them will have ("MyKey", "MyValue") in Properties.
using (_logger.BeginScope(new Dictionary<string, object> { { "MyKey", "MyValue" } }))
{
_logger.LogWarning("An example of a Warning trace..");
_logger.LogError("An example of an Error level message");
}
return new string[] { "value1", "value2" };
}
}
Capture ILogger logs from Startup.cs and Program.cs in ASP.NET Core apps
NOTE
In ASP.NET Core 3.0 and later, it is no longer possible to inject ILogger in Startup.cs and Program.cs. See
https://github.com/aspnet/Announcements/issues/353 for more details.
The new ApplicationInsightsLoggerProvider can capture logs from early in the application-startup pipeline.
Although ApplicationInsightsLoggerProvider is automatically enabled in Application Insights (starting with version
2.7.1), it doesn't have an instrumentation key set up until later in the pipeline. So, only logs from Controller/other
classes will be captured. To capture every log starting with Program.cs and Startup.cs itself, you must explicitly
enable an instrumentation key for ApplicationInsightsLoggerProvider. Also, TelemetryConfiguration is not fully set
up when you log from Program.cs or Startup.cs itself. So those logs will have a minimum configuration that uses
InMemoryChannel, no sampling, and no standard telemetry initializers or processors.
The following examples demonstrate this capability with Program.cs and Startup.cs.
Example Program.cs
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Logging;
// Adding the filter below to ensure logs of all severity from Program.cs
// is sent to ApplicationInsights.
builder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>
(typeof(Program).FullName, LogLevel.Trace);
// Adding the filter below to ensure logs of all severity from Startup.cs
// is sent to ApplicationInsights.
builder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>
(typeof(Startup).FullName, LogLevel.Trace);
}
);
}
Example Startup.cs
public class Startup
{
private readonly `ILogger` _logger;
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.AddApplicationInsightsTelemetry();
// This method gets called by the runtime. Use this method to configure the HTTP request pipeline.
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
// The following will be picked up by Application Insights.
_logger.LogInformation("Configuring for Development environment");
app.UseDeveloperExceptionPage();
}
else
{
// The following will be picked up by Application Insights.
_logger.LogInformation("Configuring for Production environment");
}
app.UseMvc();
}
}
NOTE
The new provider is available for applications that target NETSTANDARD2.0 or later. If your application targets older .NET
Core versions, such as .NET Core 1.1, or if it targets the .NET Framework, continue to use the old provider.
Console application
NOTE
There is a new Application Insights SDK called Microsoft.ApplicationInsights.WorkerService which can used to enable
Application Insights (ILogger and other Application Insights telemetry) for any Console Applications. It is recommended to
use this package and associated instructions from here.
The following code shows a sample console application that's configured to send ILogger traces to Application
Insights.
Packages installed:
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="2.1.0" />
<PackageReference Include="Microsoft.Extensions.Logging.ApplicationInsights" Version="2.9.1" />
<PackageReference Include="Microsoft.Extensions.Logging.Console" Version="2.1.0" />
</ItemGroup>
class Program
{
static void Main(string[] args)
{
// Create the DI container.
IServiceCollection services = new ServiceCollection();
// Add the logging pipelines to use. We are using Application Insights only here.
services.AddLogging(builder =>
{
// Optional: Apply filters to configure LogLevel Trace or above is sent to
// Application Insights for all categories.
builder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>
("", LogLevel.Trace);
builder.AddApplicationInsights("--YourAIKeyHere--");
});
// Build ServiceProvider.
IServiceProvider serviceProvider = services.BuildServiceProvider();
The following section shows how to override the default TelemetryConfiguration by using the
services.Configure<TelemetryConfiguration>() method. This example sets up ServerTelemetryChannel and
sampling. It adds a custom ITelemetryInitializer to the TelemetryConfiguration.
// Create the DI container.
IServiceCollection services = new ServiceCollection();
var serverChannel = new ServerTelemetryChannel();
services.Configure<TelemetryConfiguration>(
(config) =>
{
config.TelemetryChannel = serverChannel;
config.TelemetryInitializers.Add(new MyTelemetryInitalizer());
config.DefaultTelemetrySink.TelemetryProcessorChainBuilder.UseSampling(5);
serverChannel.Initialize(config);
}
);
// Add the logging pipelines to use. We are adding Application Insights only.
services.AddLogging(loggingBuilder =>
{
loggingBuilder.AddApplicationInsights();
});
........
........
{
"Logging": {
"ApplicationInsights": {
"LogLevel": {
"Default": "Warning",
"Microsoft": "Error"
}
},
"LogLevel": {
"Default": "Warning"
}
},
"AllowedHosts": "*"
}
If you experience double logging when you debug from Visual Studio, set EnableDebugLogger to false in the code
that enables Application Insights, as follows. This duplication and fix is only relevant when you're debugging the
application.
I updated to Microsoft.ApplicationInsights.AspNet SDK version 2.7.1, and logs from ILogger are captured
automatically. How do I turn off this feature completely?
See the Control logging level section to see how to filter logs in general. To turn-off
ApplicationInsightsLoggerProvider, use LogLevel.None :
In code:
builder.AddFilter<Microsoft.Extensions.Logging.ApplicationInsights.ApplicationInsightsLoggerProvider>
("", LogLevel.None);
In config:
{
"Logging": {
"ApplicationInsights": {
"LogLevel": {
"Default": "None"
}
}
Why do some ILogger logs not have the same properties as others?
Application Insights captures and sends ILogger logs by using the same TelemetryConfiguration that's used for
every other telemetry. But there's an exception. By default, TelemetryConfiguration is not fully set up when you log
from Program.cs or Startup.cs. Logs from these places won't have the default configuration, so they won't be
running all TelemetryInitializers and TelemetryProcessors.
I'm using the standalone package Microsoft.Extensions.Logging.ApplicationInsights, and I want to log some
additional custom telemetry manually. How should I do that?
When you use the standalone package, TelemetryClient is not injected to the DI container, so you need to create a
new instance of TelemetryClient and use the same configuration as the logger provider uses, as the following
code shows. This ensures that the same configuration is used for all custom telemetry as well as telemetry from
ILogger.
NOTE
If you use the Microsoft.ApplicationInsights.AspNetCore package to enable Application Insights, modify this code to get
TelemetryClient directly in the constructor. For an example, see this FAQ.
What Application Insights telemetry type is produced from ILogger logs? Or where can I see ILogger logs in
Application Insights?
ApplicationInsightsLoggerProvider captures ILogger logs and creates TraceTelemetry from them. If an Exception
object is passed to the Log() method on ILogger, ExceptionTelemetry is created instead of TraceTelemetry. These
telemetry items can be found in same places as any other TraceTelemetry or ExceptionTelemetry for Application
Insights, including portal, analytics, or Visual Studio local debugger.
If you prefer to always send TraceTelemetry, use this snippet:
builder.AddApplicationInsights((opt) => opt.TrackExceptionsAsExceptionTelemetry = false);
I don't have the SDK installed, and I use the Azure Web Apps extension to enable Application Insights for my
ASP.NET Core applications. How do I use the new provider?
The Application Insights extension in Azure Web Apps uses the new provider. You can modify the filtering rules in
the appsettings.json file for your application.
I'm using the standalone package Microsoft.Extensions.Logging.ApplicationInsights and enabling Application
Insights provider by calling builder.AddApplicationInsights("ikey"). Is there an option to get an instrumentation
key from configuration?
Modify Program.cs and appsettings.json as follows:
{
"myikeyfromconfig": "putrealikeyhere"
}
This code is required only when you use a standalone logging provider. For regular Application Insights
monitoring, the instrumentation key is loaded automatically from the configuration path ApplicationInsights:
Instrumentationkey. Appsettings.json should look like this:
{
"ApplicationInsights":
{
"Instrumentationkey":"putrealikeyhere"
}
}
Next steps
Learn more about:
Logging in ASP.NET Core
.NET trace logs in Application Insights
Get started with Application Insights in a Java web
project
12/8/2019 • 7 minutes to read • Edit Online
Application Insights is an extensible analytics service for web developers that helps you understand the
performance and usage of your live application. Use it to automatically instrument request, track
dependencies, and collect performance counters, diagnose performance issues and exceptions, and write
code to track what users do with your app.
dependencies {
compile group: 'com.microsoft.azure', name: 'applicationinsights-web-auto', version: '2.5.0'
// or applicationinsights-web for manual web filter registration
// or applicationinsights-core for bare API
}
<!-- HTTP request component (not required for bare API) -->
<TelemetryModules>
<Add
type="com.microsoft.applicationinsights.web.extensibility.modules.WebRequestTrackingTelemetryModule"/>
<Add
type="com.microsoft.applicationinsights.web.extensibility.modules.WebSessionTrackingTelemetryModule"/>
<Add
type="com.microsoft.applicationinsights.web.extensibility.modules.WebUserTrackingTelemetryModule"/>
</TelemetryModules>
</ApplicationInsights>
Optionally, the configuration file can reside in any location accessible to your application. The system
property -Dapplicationinsights.configurationDirectory specifies the directory that contains
ApplicationInsights.xml. For example, a configuration file located at
E:\myconfigs\appinsights\ApplicationInsights.xml would be configured with the property
-Dapplicationinsights.configurationDirectory="E:\myconfigs\appinsights" .
The instrumentation key is sent along with every item of telemetry and tells Application Insights to
display it in your resource.
The HTTP Request component is optional. It automatically sends telemetry about requests and response
times to the portal.
Event correlation is an addition to the HTTP request component. It assigns an identifier to each request
received by the server, and adds this identifier as a property to every item of telemetry as the property
'Operation.Id'. It allows you to correlate the telemetry associated with each request by setting a filter in
diagnostic search.
Alternative ways to set the instrumentation key
Application Insights SDK looks for the key in this order:
1. System property: -DAPPINSIGHTS_INSTRUMENTATIONKEY=your_ikey
2. Environment variable: APPINSIGHTS_INSTRUMENTATIONKEY
3. Configuration file: ApplicationInsights.xml
You can also set it in code:
String instrumentationKey = "00000000-0000-0000-0000-000000000000";
if (instrumentationKey != null)
{
TelemetryConfiguration.getActive().setInstrumentationKey(instrumentationKey);
}
4. Add agent
Install the Java Agent to capture outgoing HTTP calls, JDBC queries, application logging, and better
operation naming.
Instance data
Click through a specific request type to see individual instances.
Analytics: Powerful query language
As you accumulate more data, you can run queries both to aggregate data and to find individual instances.
Analytics is a powerful tool for both for understanding performance and usage, and for diagnostic purposes.
Performance counters
Open Investigate, Metrics, to see a range of performance counters.
Customize performance counter collection
To disable collection of the standard set of performance counters, add the following code under the root node
of the ApplicationInsights.xml file:
<PerformanceCounters>
<UseBuiltIn>False</UseBuiltIn>
</PerformanceCounters>
<PerformanceCounters>
<Jmx>
<Add objectName="java.lang:type=ClassLoading" attribute="TotalLoadedClassCount"
displayName="Loaded Class Count"/>
<Add objectName="java.lang:type=Memory" attribute="HeapMemoryUsage.used" displayName="Heap Memory
Usage-used" type="composite"/>
</Jmx>
</PerformanceCounters>
<PerformanceCounters>
<Windows>
<Add displayName="Process User Time" categoryName="Process" counterName="%User Time"
instanceName="__SELF__" />
<Add displayName="Bytes Printed per Second" categoryName="Print Queue" counterName="Bytes
Printed/sec" instanceName="Fax" />
</Windows>
</PerformanceCounters>
Questions? Problems?
Troubleshooting Java
Next steps
Monitor dependency calls
Monitor Unix performance counters
Add monitoring to your web pages to monitor page load times, AJAX calls, browser exceptions.
Write custom telemetry to track usage in the browser or at the server.
Use Analytics for powerful queries over telemetry from your app
For more information, visit Azure for Java developers.
Monitor Docker applications in Application Insights
(Deprecated)
10/20/2019 • 3 minutes to read • Edit Online
NOTE
This solution has been deprecated. To learn more about our current investments in container monitoring we recommend
checking out Azure Monitor for containers.
Lifecycle events and performance counters from Docker containers can be charted on Application Insights. Install
the Application Insights image in a container in your host, and it will display performance counters for the host, as
well as for the other images.
With Docker, you distribute your apps in lightweight containers complete with all dependencies. They'll run on any
host machine that runs a Docker Engine.
When you run the Application Insights image on your Docker host, you get these benefits:
Lifecycle telemetry about all the containers running on the host - start, stop, and so on.
Performance counters for all the containers. CPU, memory, network usage, and more.
If you installed Application Insights SDK for Java in the apps running in the containers, all the telemetry of those
apps will have additional properties identifying the container and host machine. So for example, if you have
instances of an app running in more than one host, you can easily filter your app telemetry by host.
Only one Application Insights image is required per Docker host. If your application is deployed on multiple Docker
hosts, then repeat the command on every host.
<Add type="com.microsoft.applicationinsights.extensibility.initializer.docker.DockerContextInitializer"/>
This adds Docker information such as container and host id to every telemetry item sent from your app.
Q&A
What does Application Insights give me that I can't get from Docker?
Detailed breakdown of performance counters by container and image.
Integrate container and app data in one dashboard.
Export telemetry for further analysis to a database, Power BI or other dashboard.
How do I get telemetry from the app itself?
Install the Application Insights SDK in the app. Learn how for: Java web apps, Windows web apps.
Video
Next steps
Application Insights for Java
Application Insights for Node.js
Application Insights for ASP.NET
Explore Java trace logs in Application Insights
12/8/2019 • 2 minutes to read • Edit Online
If you're using Logback or Log4J (v1.2 or v2.0) for tracing, you can have your trace logs sent automatically to
Application Insights where you can explore and search on them.
TIP
You only need to set your Application Insights Instrumentation Key once for your application. If you are using a framework
like Java Spring, you may have already registered the key elsewhere in your app's configuration.
You can disable the Java agent's logging capture using the AI-Agent.xml file:
Alternatively (as opposed to using the Java agent), you can follow the
instructions below
Install the Java SDK
Follow the instructions to install Application Insights SDK for Java, if you haven't already done that.
Add logging libraries to your project
Choose the appropriate way for your project.
If you're using Maven...
If your project is already set up to use Maven for build, merge one of the following snippets of code into your
pom.xml file.
Then refresh the project dependencies, to get the binaries downloaded.
Logback
<dependencies>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>applicationinsights-logging-logback</artifactId>
<version>[2.0,)</version>
</dependency>
</dependencies>
Log4J v2.0
<dependencies>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>applicationinsights-logging-log4j2</artifactId>
<version>[2.0,)</version>
</dependency>
</dependencies>
Log4J v1.2
<dependencies>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>applicationinsights-logging-log4j1_2</artifactId>
<version>[2.0,)</version>
</dependency>
</dependencies>
Log4J v2.0
Log4J v1.2
Otherwise ...
Follow the guidelines to manually install Application Insights Java SDK, download the jar (After arriving at Maven
Central Page click on 'jar' link in download section) for appropriate appender and add the downloaded appender
jar to the project.
LOGGER DOWNLOAD LIBRARY
<appender name="aiAppender"
class="com.microsoft.applicationinsights.logback.ApplicationInsightsAppender">
<instrumentationKey>[APPLICATION_INSIGHTS_KEY]</instrumentationKey>
</appender>
<root level="trace">
<appender-ref ref="aiAppender" />
</root>
Log4J v2.0
<Configuration packages="com.microsoft.applicationinsights.log4j.v2">
<Appenders>
<ApplicationInsightsAppender name="aiAppender" instrumentationKey="[APPLICATION_INSIGHTS_KEY]" />
</Appenders>
<Loggers>
<Root level="trace">
<AppenderRef ref="aiAppender"/>
</Root>
</Loggers>
</Configuration>
Log4J v1.2
<appender name="aiAppender"
class="com.microsoft.applicationinsights.log4j.v1_2.ApplicationInsightsAppender">
<param name="instrumentationKey" value="[APPLICATION_INSIGHTS_KEY]" />
</appender>
<root>
<priority value ="trace" />
<appender-ref ref="aiAppender" />
</root>
The Application Insights appenders can be referenced by any configured logger, and not necessarily by the root
logger (as shown in the code samples above).
To explore Linux system performance metrics in Application Insights, install collectd, together with its Application
Insights plug-in. This open-source solution gathers various system and network statistics.
Typically you'll use collectd if you have already instrumented your Java web service with Application Insights. It
gives you more data to help you to enhance your app's performance or diagnose problems.
LoadPlugin "com.microsoft.applicationinsights.collectd.ApplicationInsightsWriter"
<Plugin ApplicationInsightsWriter>
InstrumentationKey "Your key"
</Plugin>
Configure other collectd plugins, which can collect various data from different sources.
Restart collectd according to its manual.
DIRECTIVE EFFECT
Exclude disk:read,write Exclude the sources named read and write from the
disk plugin.
Known issue
The Application Insights Write plugin is incompatible with certain Read plugins. Some plugins sometimes send
"NaN" where the Application Insights plugin expects a floating-point number.
Symptom: The collectd log shows errors that include "AI: ... SyntaxError: Unexpected token N".
Workaround: Exclude data collected by the problem Write plugins.
Monitor dependencies, caught exceptions, and
method execution times in Java web apps
12/8/2019 • 3 minutes to read • Edit Online
If you have instrumented your Java web app with Application Insights, you can use the Java Agent to get
deeper insights, without any code changes:
Dependencies: Data about calls that your application makes to other components, including:
Outgoing HTTP calls made via Apache HttpClient, OkHttp, and java.net.HttpURLConnection are
captured.
Redis calls made via the Jedis client are captured.
JDBC queries - For MySQL and PostgreSQL, if the call takes longer than 10 seconds, the agent
reports the query plan.
Application logging: Capture and correlate your application logs with HTTP requests and other
telemetry
Log4j 1.2
Log4j2
Logback
Better operation naming: (used for aggregation of requests in the portal)
Spring - based on @RequestMapping .
JAX-RS - based on @Path .
To use the Java agent, you install it on your server. Your web apps must be instrumented with the Application
Insights Java SDK.
<!-- capture logging via Log4j 1.2, Log4j2, and Logback, default is true -->
<Logging enabled="true" />
<!-- capture outgoing HTTP calls performed through Apache HttpClient, OkHttp,
and java.net.HttpURLConnection, default is true -->
<HTTP enabled="true" />
<!-- capture query plans for JDBC queries that exceed this value (MySQL, PostgreSQL),
default is 10000 milliseconds -->
<MaxStatementQueryLimitInMS>1000</MaxStatementQueryLimitInMS>
</BuiltIn>
</Instrumentation>
</ApplicationInsightsAgent>
For the latest version of the Java agent, check the releases here.
The agent must be packaged as a resource in your project such that it ends up in the D:/home/site/wwwroot/
directory. You can confirm that your agent is in the correct App Service directory by going to Development
Tools > Advanced Tools > Debug Console and examining the contents of the site directory.
Save the settings and Restart your app. (These steps only apply to App Services running on Windows.)
NOTE
AI-Agent.xml and the agent jar file should be in the same folder. They are often placed together in the /resources
folder of the project.
<Instrumentation>
<BuiltIn enabled="true">
<HTTP enabled="true" W3C="true" enableW3CBackCompat="true"/>
</BuiltIn>
</Instrumentation>
NOTE
Backward compatibility mode is enabled by default and the enableW3CBackCompat parameter is optional and should be
used only when you want to turn it off.
Ideally this would be the case when all your services have been updated to newer version of SDKs supporting
W3C protocol. It is highly recommended to move to newer version of SDKs with W3C support as soon as
possible.
Make sure that both incoming and outgoing (agent) configurations are exactly same.
Questions? Problems?
No data? Set firewall exceptions
Troubleshooting Java
Filter telemetry in your Java web app
12/8/2019 • 3 minutes to read • Edit Online
Filters provide a way to select the telemetry that your Java web app sends to Application Insights. There are some
out-of-the-box filters that you can use, and you can also write your own custom filters.
The out-of-the-box filters include:
Trace severity level
Specific URLs, keywords or response codes
Fast responses - that is, requests to which your app responded to quickly
Specific event names
NOTE
Filters skew the metrics of your app. For example, you might decide that, in order to diagnose slow responses, you will set a
filter to discard fast response times. But you must be aware that the average response times reported by Application
Insights will then be slower than the true speed, and the count of requests will be smaller than the real count. If this is a
concern, use Sampling instead.
Setting filters
In ApplicationInsights.xml, add a TelemetryProcessors section like this example:
<ApplicationInsights>
<TelemetryProcessors>
<BuiltInProcessors>
<Processor type="TraceTelemetryFilter">
<Add name="FromSeverityLevel" value="ERROR"/>
</Processor>
<Processor type="RequestTelemetryFilter">
<Add name="MinimumDurationInMS" value="100"/>
<Add name="NotNeededResponseCodes" value="200-400"/>
</Processor>
<Processor type="PageViewTelemetryFilter">
<Add name="DurationThresholdInMS" value="100"/>
<Add name="NotNeededNames" value="home,index"/>
<Add name="NotNeededUrls" value=".jpg,.css"/>
</Processor>
<Processor type="TelemetryEventFilter">
<!-- Names of events we don't want to see -->
<Add name="NotNeededNames" value="Start,Stop,Pause"/>
</Processor>
</BuiltInProcessors>
<CustomProcessors>
<Processor type="com.fabrikam.MyFilter">
<Add name="Successful" value="false"/>
</Processor>
</CustomProcessors>
</TelemetryProcessors>
</ApplicationInsights>
Built-in filters
Metric Telemetry filter
<Processor type="MetricTelemetryFilter">
<Add name="NotNeeded" value="metric1,metric2"/>
</Processor>
DurationThresholdInMS - Duration refers to the time taken to load the page. If this is set, pages that loaded
faster than this time are not reported.
NotNeededNames - Comma-separated list of page names.
NotNeededUrls - Comma-separated list of URL fragments. For example, "home" filters out all pages that have
"home" in the URL.
Request Telemetry Filter
<Processor type="RequestTelemetryFilter">
<Add name="MinimumDurationInMS" value="500"/>
<Add name="NotNeededResponseCodes" value="page1,page2"/>
<Add name="NotNeededUrls" value="url1,url2"/>
</Processor>
Custom filters
1. Code your filter
In your code, create a class that implements TelemetryProcessor :
package com.fabrikam.MyFilter;
import com.microsoft.applicationinsights.extensibility.TelemetryProcessor;
import com.microsoft.applicationinsights.telemetry.Telemetry;
@Bean
public TelemetryProcessor successFilter() {
return new SuccessFilter();
}
You will need to create your own filter parameters in application.properties and leverage Spring Boot's
externalized configuration framework to pass those parameters into your custom filter.
Troubleshooting
My filter isn't working.
Check that you have provided valid parameter values. For example, durations should be integers. Invalid values
will cause the filter to be ignored. If your custom filter throws an exception from a constructor or set method, it
will be ignored.
Next steps
Sampling - Consider sampling as an alternative that does not skew your metrics.
How to use Micrometer with Azure Application
Insights Java SDK
12/5/2019 • 4 minutes to read • Edit Online
Micrometer application monitoring measures metrics for JVM -based application code and lets you export the data
to your favorite monitoring systems. This article will teach you how to use Micrometer with Application Insights for
both Spring Boot and non-Spring Boot applications.
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>applicationinsights-spring-boot-starter</artifactId>
<version>2.5.0</version>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-spring-legacy</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-azure-monitor</artifactId>
<version>1.1.0</version>
</dependency>
2. Update the application.properties or yml file with the Application Insights Instrumentation key using the
following property:
azure.application-insights.instrumentation-key=<your-instrumentation-key-here>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>azure-spring-boot-metrics-starter</artifactId>
<version>2.0.7</version>
</dependency>
2. Update the application.properties or yml file with the Application Insights Instrumentation key using the
following property:
azure.application-insights.instrumentation-key=<your-instrumentation-key-here>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-azure-monitor</artifactId>
<version>1.1.0</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>applicationinsights-web-auto</artifactId>
<version>2.5.0</version>
</dependency>
<!-- HTTP request component (not required for bare API) -->
<TelemetryModules>
<Add
type="com.microsoft.applicationinsights.web.extensibility.modules.WebRequestTrackingTelemetryModule"/>
<Add
type="com.microsoft.applicationinsights.web.extensibility.modules.WebSessionTrackingTelemetryModule"/>
<Add
type="com.microsoft.applicationinsights.web.extensibility.modules.WebUserTrackingTelemetryModule"/>
</TelemetryModules>
</ApplicationInsights>
@Override
@Timed(value = "hello.world")
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
response.getWriter().println("Hello World!");
MeterRegistry registry = (MeterRegistry)
getServletContext().getAttribute("AzureMonitorMeterRegistry");
@Override
public void contextInitialized(ServletContextEvent servletContextEvent) {
// Create AzureMonitorMeterRegistry
private final AzureMonitorConfig config = new AzureMonitorConfig() {
@Override
public String get(String key) {
return null;
}
@Override
public Duration step() {
return Duration.ofSeconds(60);}
@Override
public boolean enabled() {
return false;
}
};
@Override
public void contextDestroyed(ServletContextEvent servletContextEvent) {
}
}
@Bean
GuavaCacheMetrics guavaCacheMetrics() {
Return new GuavaCacheMetrics();
}
There are several metrics that are not enabled by default but can be bound in the above fashion. For a complete list,
refer to the official Micrometer GitHub repo.
Non-Spring apps
Add the following binding code to the configuration file:
New GuavaCacheMetrics().bind(registry);
Next steps
To learn more about Micrometer, see the official Micrometer documentation.
To learn about Spring on Azure, see the official Spring on Azure documentation.
Troubleshooting and Q and A for Application
Insights for Java
12/12/2019 • 7 minutes to read • Edit Online
Questions or problems with Azure Application Insights in Java? Here are some tips.
Build errors
In Eclipse or Intellij Idea, when adding the Application Insights SDK via Maven or Gradle, I get build or
checksum validation errors.
If the dependency element is using a pattern with wildcard characters (e.g. (Maven)
<version>
<version>[2.0,)</version> or ( Gradle) version:'2.0.+' ), try specifying a specific version instead like 2.0.1 .
See the release notes for the latest version.
No data
I added Application Insights successfully and ran my app, but I've never seen data in the portal.
Wait a minute and click Refresh. The charts refresh themselves periodically, but you can also refresh manually.
The refresh interval depends on the time range of the chart.
Check that you have an instrumentation key defined in the ApplicationInsights.xml file (in the resources folder
in your project) or configured as Environment variable.
Verify that there is no <DisableTelemetry>true</DisableTelemetry> node in the xml file.
In your firewall, you might have to open TCP ports 80 and 443 for outgoing traffic to
dc.services.visualstudio.com. See the full list of firewall exceptions
In the Microsoft Azure start board, look at the service status map. If there are some alert indications, wait until
they have returned to OK and then close and re-open your Application Insights application blade.
Turn on logging by adding an <SDKLogger /> element under the root node in the ApplicationInsights.xml file
(in the resources folder in your project), and check for entries prefaced with AI: INFO/WARN/ERROR for any
suspicious logs.
Make sure that the correct ApplicationInsights.xml file has been successfully loaded by the Java SDK, by
looking at the console's output messages for a "Configuration file has been successfully found" statement.
If the config file is not found, check the output messages to see where the config file is being searched for, and
make sure that the ApplicationInsights.xml is located in one of those search locations. As a rule of thumb, you
can place the config file near the Application Insights SDK JARs. For example: in Tomcat, this would mean the
WEB -INF/classes folder. During development you can place ApplicationInsights.xml in resources folder of your
web project.
Please also look at GitHub issues page for known issues with the SDK.
Please ensure to use same version of Application Insights core, web, agent and logging appenders to avoid any
version conflict issues.
I used to see data, but it has stopped
Check the status blog.
Have you hit your monthly quota of data points? Open Settings/Quota and Pricing to find out. If so, you can
upgrade your plan, or pay for additional capacity. See the pricing scheme.
Have you recently upgraded your SDK? Please ensure that only Unique SDK jars are present inside the project
directory. There should not be two different versions of SDK present.
Are you looking at the correct AI resource? Please match the iKey of your application to the resource where
you are expecting telemetry. They should be the same.
I don't see all the data I'm expecting
Open the Usage and estimated cost page and check whether sampling is in operation. (100% transmission
means that sampling isn't in operation.) The Application Insights service can be set to accept only a fraction of
the telemetry that arrives from your app. This helps you keep within your monthly quota of telemetry.
Do you have SDK Sampling turned on? If yes, data would be sampled at the rate specified for all the applicable
types.
Are you running an older version of Java SDK? Starting with version 2.0.1, we have introduced fault tolerance
mechanism to handle intermittent network and backend failures as well as data persistence on local drives.
Are you getting throttled due to excessive telemetry? If you turn on INFO logging, you will see a log message
"App is throttled". Our current limit is 32k telemetry items/second.
Java Agent cannot capture dependency data
Have you configured Java agent by following Configure Java Agent ?
Make sure both the java agent jar and the AI-Agent.xml file are placed in the same folder.
Make sure that the dependency you are trying to auto-collect is supported for auto collection. Currently we
only support MySQL, MsSQL, Oracle DB and Azure Cache for Redis dependency collection.
No usage data
I see data about requests and response times, but no page view, browser, or user data.
You successfully set up your app to send telemetry from the server. Now your next step is to set up your web
pages to send telemetry from the web browser.
Alternatively, if your client is an app in a phone or other device, you can send telemetry from there.
Use the same instrumentation key to set up both your client and server telemetry. The data will appear in the
same Application Insights resource, and you'll be able to correlate events from client and server.
Disabling telemetry
How can I disable telemetry collection?
In code:
Or
Update ApplicationInsights.xml (in the resources folder in your project). Add the following under the root node:
<DisableTelemetry>true</DisableTelemetry>
Using the XML method, you have to restart the application when you change the value.
azure.application-insights.logger.type=file
azure.application-insights.logger.base-folder-path=C:/agent/AISDK
azure.application-insights.logger.level=trace
azure.application-insights.logger.type=console
azure.application-insights.logger.level=trace
Java Agent
To enable JVM Agent Logging update the AI-Agent.xml file:
Intranet servers
Can I monitor a server on my intranet?
Yes, provided your server can send telemetry to the Application Insights portal through the public internet.
In your firewall, you might have to open TCP ports 80 and 443 for outgoing traffic to dc.services.visualstudio.com
and f5.services.visualstudio.com.
Data retention
How long is data retained in the portal? Is it secure?
See Data retention and privacy.
Debug logging
Application Insights uses org.apache.http . This is relocated within Application Insights core jars under the
namespace com.microsoft.applicationinsights.core.dependencies.http . This enables Application Insights to
handle scenarios where different versions of the same org.apache.http exist in one code base.
NOTE
If you enable DEBUG level logging for all namespaces in the app, it will be honored by all executing modules including
org.apache.http renamed as com.microsoft.applicationinsights.core.dependencies.http . Application Insights will
not be able to apply filtering for these calls because the log call is being made by the Apache library. DEBUG level logging
produce a considerable amount of log data and is not recommended for live production instances.
Next steps
I set up Application Insights for my Java server app. What else can I do?
Monitor availability of your web pages
Monitor web page usage
Track usage and diagnose issues in your device apps
Write code to track usage of your app
Capture diagnostic logs
Get help
Stack Overflow
File an issue on GitHub
Monitor your Node.js services and apps with
Application Insights
10/23/2019 • 5 minutes to read • Edit Online
Azure Application Insights monitors your backend services and components after deployment, to help you
discover and rapidly diagnose performance and other issues. You can use Application Insights for Node.js
services that are hosted in your datacenter, in Azure VMs and web apps, and even in other public clouds.
To receive, store, and explore your monitoring data, include the SDK in your code, and then set up a
corresponding Application Insights resource in Azure. The SDK sends data to that resource for further analysis
and exploration.
The Node.js SDK can automatically monitor incoming and outgoing HTTP requests, exceptions, and some
system metrics. Beginning in version 0.20, the SDK also can monitor some common third-party packages, like
MongoDB, MySQL, and Redis. All events related to an incoming HTTP request are correlated for faster
troubleshooting.
You can use the TelemetryClient API to manually instrument and monitor additional aspects of your app and
system. We describe the TelemetryClient API in more detail later in this article.
Get started
Complete the following tasks to set up monitoring for an app or service.
Prerequisites
Before you begin, make sure that you have an Azure subscription, or get a new one for free. If your organization
already has an Azure subscription, an administrator can follow these instructions to add you to it.
Set up an Application Insights resource
1. Sign in to the Azure portal.
2. Select Create a resource > Developer tools > Application Insights. The resource includes an
endpoint for receiving telemetry data, storage for this data, saved reports and dashboards, rule and alert
configuration, and more.
3. On the resource creation page, in the Application Type box, select Node.js Application. The app type
determines the default dashboards and reports that are created. (Any Application Insights resource can
collect data from any language and platform.)
Set up the Node.js SDK
Include the SDK in your app, so it can gather data.
1. Copy your resource's Instrumentation Key (also called an ikey) from the Azure portal. Application Insights
uses the ikey to map data to your Azure resource. Before the SDK can use your ikey, you must specify the
ikey in an environment variable or in your code.
2. Add the Node.js SDK library to your app's dependencies via package.json. From the root folder of your
app, run:
3. Explicitly load the library in your code. Because the SDK injects instrumentation into many other libraries,
load the library as early as possible, even before other require statements.
At the top of your first .js file, add the following code. The setup method configures the ikey (and thus,
the Azure resource) to be used by default for all tracked items.
You also can provide an ikey via the environment variable APPINSIGHTS_INSTRUMENTATIONKEY,
instead of passing it manually to setup() or new appInsights.TelemetryClient() . This practice lets you
keep ikeys out of committed source code, and you can specify different ikeys for different environments.
For additional configuration options, see the following sections.
You can try the SDK without sending telemetry by setting
appInsights.defaultClient.config.disableAppInsights = true .
Monitor your app
The SDK automatically gathers telemetry about the Node.js runtime and about some common third-party
modules. Use your application to generate some of this data.
Then, in the Azure portal go to the Application Insights resource that you created earlier. In the Overview
timeline, look for your first few data points. To see more detailed data, select different components in the charts.
To view the topology that is discovered for your app, select the Application map button. Select components in
the map to see more details.
To learn more about your app, and to troubleshoot problems, in the INVESTIGATE section, select the other
views that are available.
No data?
Because the SDK batches data for submission, there might be a delay before items are displayed in the portal. If
you don't see data in your resource, try some of the following fixes:
Continue to use the application. Take more actions to generate more telemetry.
Click Refresh in the portal resource view. Charts periodically refresh on their own, but manually refreshing
forces them to refresh immediately.
Verify that required outgoing ports are open.
Use Search to look for specific events.
Check the FAQ.
SDK configuration
The SDK's configuration methods and default values are listed in the following code example.
To fully correlate events in a service, be sure to set .setAutoDependencyCorrelation(true) . With this option set, the
SDK can track context across asynchronous callbacks in Node.js.
TelemetryClient API
For a full description of the TelemetryClient API, see Application Insights API for custom events and metrics.
You can track any request, event, metric, or exception by using the Application Insights Node.js SDK. The
following code example demonstrates some of the APIs that you can use:
Next steps
Monitor your telemetry in the portal
Write Analytics queries over your telemetry
Set up Azure Monitor for your Python application
(preview)
12/19/2019 • 8 minutes to read • Edit Online
Azure Monitor supports distributed tracing, metric collection, and logging of Python applications through
integration with OpenCensus. This article will walk you through the process of setting up OpenCensus for
Python and sending your monitoring data to Azure Monitor.
Prerequisites
An Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
Python installation. This article uses Python 3.7.0, though earlier versions will likely work with minor changes.
Name Globally unique value Name that identifies the app you're
monitoring
3. Select Create.
The SDK uses three Azure Monitor exporters to send different types of telemetry to Azure Monitor: trace,
metrics, and logs. For more information on these telemetry types, see the data platform overview. Use the
following instructions to send these telemetry types via the three exporters.
Trace
NOTE
Trace in OpenCensus refers to distributed tracing. The AzureExporter sends requests and dependency telemetry to
Azure Monitor.
1. First, let's generate some trace data locally. In Python IDLE, or your editor of choice, enter the following
code.
tracer = Tracer(sampler=ProbabilitySampler(1.0))
def valuePrompt():
with tracer.span(name="test") as span:
line = input("Enter a value: ")
print(line)
def main():
while True:
valuePrompt()
if __name__ == "__main__":
main()
2. Running the code will repeatedly prompt you to enter a value. With each entry, the value will be printed to
the shell, and the OpenCensus Python Module will generate a corresponding piece of SpanData . The
OpenCensus project defines a trace as a tree of spans.
Enter a value: 4
4
[SpanData(name='test', context=SpanContext(trace_id=8aa41bc469f1a705aed1bdb20c342603, span_id=None,
trace_options=TraceOptions(enabled=True), tracestate=None), span_id='15ac5123ac1f6847',
parent_span_id=None, attributes=BoundedDict({}, maxlen=32), start_time='2019-06-27T18:21:22.805429Z',
end_time='2019-06-27T18:21:44.933405Z', child_span_count=0, stack_trace=None,
annotations=BoundedList([], maxlen=32), message_events=BoundedList([], maxlen=128),
links=BoundedList([], maxlen=32), status=None, same_process_as_parent_span=None, span_kind=0)]
Enter a value: 25
25
[SpanData(name='test', context=SpanContext(trace_id=8aa41bc469f1a705aed1bdb20c342603, span_id=None,
trace_options=TraceOptions(enabled=True), tracestate=None), span_id='2e512f846ba342de',
parent_span_id=None, attributes=BoundedDict({}, maxlen=32), start_time='2019-06-27T18:21:44.933405Z',
end_time='2019-06-27T18:21:46.156787Z', child_span_count=0, stack_trace=None,
annotations=BoundedList([], maxlen=32), message_events=BoundedList([], maxlen=128),
links=BoundedList([], maxlen=32), status=None, same_process_as_parent_span=None, span_kind=0)]
Enter a value: 100
100
[SpanData(name='test', context=SpanContext(trace_id=8aa41bc469f1a705aed1bdb20c342603, span_id=None,
trace_options=TraceOptions(enabled=True), tracestate=None), span_id='f3f9f9ee6db4740a',
parent_span_id=None, attributes=BoundedDict({}, maxlen=32), start_time='2019-06-27T18:21:46.157732Z',
end_time='2019-06-27T18:21:47.269583Z', child_span_count=0, stack_trace=None,
annotations=BoundedList([], maxlen=32), message_events=BoundedList([], maxlen=128),
links=BoundedList([], maxlen=32), status=None, same_process_as_parent_span=None, span_kind=0)]
3. Although entering values is helpful for demonstration purposes, ultimately we want to emit the SpanData
to Azure Monitor. Modify your code from the previous step based on the following code sample:
def valuePrompt():
with tracer.span(name="test") as span:
line = input("Enter a value: ")
print(line)
def main():
while True:
valuePrompt()
if __name__ == "__main__":
main()
4. Now when you run the Python script, you should still be prompted to enter values, but only the value is
being printed in the shell. The created SpanData will be sent to Azure Monitor. You can find the emitted
span data under dependencies .
5. For information on sampling in OpenCensus, take a look at sampling in OpenCensus.
6. For details on telemetry correlation in your trace data, take a look at OpenCensus telemetry correlation.
Metrics
1. First, let's generate some local metric data. We'll create a simple metric to track the number of times the
user presses Enter.
stats = stats_module.stats
view_manager = stats.view_manager
stats_recorder = stats.stats_recorder
prompt_measure = measure_module.MeasureInt("prompts",
"number of prompts",
"prompts")
prompt_view = view_module.View("prompt view",
"number of prompts",
[],
prompt_measure,
aggregation_module.CountAggregation())
view_manager.register_view(prompt_view)
mmap = stats_recorder.new_measurement_map()
tmap = tag_map_module.TagMap()
def prompt():
input("Press enter.")
mmap.measure_int_put(prompt_measure, 1)
mmap.record(tmap)
metrics = list(mmap.measure_to_view_map.get_metrics(datetime.utcnow()))
print(metrics[0].time_series[0].points[0])
def main():
while True:
prompt()
if __name__ == "__main__":
main()
2. Running the code will repeatedly prompt you to press Enter. A metric is created to track the number of
times Enter is pressed. With each entry, the value will be incremented and the metric information will be
displayed in the console. The information includes the current value and the current time stamp when the
metric was updated.
Press enter.
Point(value=ValueLong(5), timestamp=2019-10-09 20:58:04.930426)
Press enter.
Point(value=ValueLong(6), timestamp=2019-10-09 20:58:06.570167)
Press enter.
Point(value=ValueLong(7), timestamp=2019-10-09 20:58:07.138614)
3. Although entering values is helpful for demonstration purposes, ultimately we want to emit the metric data
to Azure Monitor. Modify your code from the previous step based on the following code sample:
from datetime import datetime
from opencensus.ext.azure import metrics_exporter
from opencensus.stats import aggregation as aggregation_module
from opencensus.stats import measure as measure_module
from opencensus.stats import stats as stats_module
from opencensus.stats import view as view_module
from opencensus.tags import tag_map as tag_map_module
stats = stats_module.stats
view_manager = stats.view_manager
stats_recorder = stats.stats_recorder
prompt_measure = measure_module.MeasureInt("prompts",
"number of prompts",
"prompts")
prompt_view = view_module.View("prompt view",
"number of prompts",
[],
prompt_measure,
aggregation_module.CountAggregation())
view_manager.register_view(prompt_view)
mmap = stats_recorder.new_measurement_map()
tmap = tag_map_module.TagMap()
view_manager.register_exporter(exporter)
def prompt():
input("Press enter.")
mmap.measure_int_put(prompt_measure, 1)
mmap.record(tmap)
metrics = list(mmap.measure_to_view_map.get_metrics(datetime.utcnow()))
print(metrics[0].time_series[0].points[0])
def main():
while True:
prompt()
if __name__ == "__main__":
main()
4. The exporter will send metric data to Azure Monitor at a fixed interval. The default is every 15 seconds.
We're tracking a single metric, so this metric data, with whatever value and time stamp it contains, will be
sent every interval. You can find the data under customMetrics .
Logs
1. First, let's generate some local log data.
import logging
logger = logging.getLogger(__name__)
def valuePrompt():
line = input("Enter a value: ")
logger.warning(line)
def main():
while True:
valuePrompt()
if __name__ == "__main__":
main()
2. The code will continuously ask for a value to be entered. A log entry is emitted for every entered value.
Enter a value: 24
24
Enter a value: 55
55
Enter a value: 123
123
Enter a value: 90
90
3. Although entering values is helpful for demonstration purposes, ultimately we want to emit the log data to
Azure Monitor. Modify your code from the previous step based on the following code sample:
import logging
from opencensus.ext.azure.log_exporter import AzureLogHandler
logger = logging.getLogger(__name__)
def valuePrompt():
line = input("Enter a value: ")
logger.warning(line)
def main():
while True:
valuePrompt()
if __name__ == "__main__":
main()
4. The exporter will send log data to Azure Monitor. You can find the data under traces .
NOTE
traces in this context is not the same as Tracing . traces refers to the type of telemetry that you will see in Azure
Monitor when utilizing the AzureLogHandler . Tracing refers to a concept in OpenCensus and relates to distributed
tracing.
5. To format your log messages, you can use formatters in the built-in Python logging API.
import logging
from opencensus.ext.azure.log_exporter import AzureLogHandler
logger = logging.getLogger(__name__)
def valuePrompt():
line = input("Enter a value: ")
logger.warning(line)
def main():
while True:
valuePrompt()
if __name__ == "__main__":
main()
6. You can also add custom dimensions to your logs. These will appear as key-value pairs in
customDimensions in Azure Monitor.
NOTE
For this feature to work, you need to pass a dictionary as an argument to your logs, any other data structure will be
ignored. To maintain string formatting, store them in a dictionary and pass them as arguments.
```python
import logging
logger = logging.getLogger(__name__)
# TODO: replace the all-zero GUID with your instrumentation key.
logger.addHandler(AzureLogHandler(
connection_string='InstrumentationKey=00000000-0000-0000-0000-000000000000')
)
logger.warning('action', {'key-1': 'value-1', 'key-2': 'value2'})
```
7. For details on how to enrich your logs with trace context data, see OpenCensus Python logs integration.
For more detailed information about how to use queries and logs, see Logs in Azure Monitor.
Next steps
Application map
End-to-end performance monitoring
Alerts
Availability tests: Create tests to make sure your site is visible on the web.
Smart diagnostics: These tests run automatically, so you don't have to do anything to set them up. They tell
you if your app has an unusual rate of failed requests.
Metric alerts: Set alerts to warn you if a metric crosses a threshold. You can set them on custom metrics that
you code into your app.
Track dependencies with OpenCensus Python
12/13/2019 • 2 minutes to read • Edit Online
A dependency is an external component that is called by your application. Dependency data is collected using
OpenCensus Python and its various integrations. The data is then sent to Application Insights under Azure Monitor
as dependencies telemetry.
First, instrument your Python application with latest OpenCensus Python SDK.
In-process dependencies
OpenCensus Python SDK for Azure Monitor allows you to send "in-process" dependency telemetry (information
and logic that occurs within your application). In-process dependencies will have the type field as INPROC in
analytics.
tracer = Tracer(exporter=AzureExporter(connection_string="InstrumentationKey=<your-ikey-here>"),
sampler=ProbabilitySampler(1.0))
with tracer.span(name='foo'): # <-- A dependency telemetry item will be sent for this span "foo"
print('Hello, World!')
import requests
from opencensus.ext.azure.trace_exporter import AzureExporter
from opencensus.trace import config_integration
from opencensus.trace.samplers import ProbabilitySampler
from opencensus.trace.tracer import Tracer
tracer = Tracer(exporter=AzureExporter(connection_string="InstrumentationKey=<your-ikey-here>"),
sampler=ProbabilitySampler(1.0))
with tracer.span(name='parent'):
response = requests.get(url='https://www.wikipedia.org/wiki/Rabbit') # <-- this request will be tracked
config_integration.trace_integrations(['httplib'])
conn = httplib.HTTPConnection("www.python.org")
tracer = Tracer(
exporter=AzureExporter(),
sampler=ProbabilitySampler(1.0)
)
MIDDLEWARE = [
...
'opencensus.ext.django.middleware.OpencensusMiddleware',
]
OPENCENSUS = {
'TRACE': {
'SAMPLER': 'opencensus.trace.samplers.ProbabilitySampler(rate=1)',
'EXPORTER': '''opencensus.ext.ocagent.trace_exporter.TraceExporter(
service_name='foobar',
)''',
}
}
config_integration.trace_integrations(['mysql'])
config_integration.trace_integrations(['pymysql'])
config_integration.trace_integrations(['postgresql'])
config_integration.trace_integrations(['pymongo'])
config_integration.trace_integrations(['sqlalchemy'])
Next steps
Application Map
Availability
Search
Log (Analytics) query
Transaction diagnostics
Track incoming requests with OpenCensus Python
11/4/2019 • 2 minutes to read • Edit Online
Incoming request data is collected using OpenCensus Python and its various integrations. Track incoming request
data sent to your web applications built on top of the popular web frameworks django , flask and pyramid . The
data is then sent to Application Insights under Azure Monitor as requests telemetry.
First, instrument your Python application with latest OpenCensus Python SDK.
MIDDLEWARE = (
...
'opencensus.ext.django.middleware.OpencensusMiddleware',
...
)
OPENCENSUS = {
'TRACE': {
'SAMPLER': 'opencensus.trace.samplers.ProbabilitySampler(rate=0.5)',
'EXPORTER': '''opencensus.ext.azure.trace_exporter.AzureExporter(
service_name='foobar',
)''',
}
}
4. You can also add urls to settings.py under BLACKLIST_PATHS for requests that you do not want to track.
OPENCENSUS = {
'TRACE': {
'SAMPLER': 'opencensus.trace.samplers.ProbabilitySampler(rate=0.5)',
'EXPORTER': '''opencensus.ext.azure.trace_exporter.AzureExporter(
service_name='foobar',
)''',
'BLACKLIST_PATHS': 'https://example.com', <--- This site will not be traced if a request is
sent from it.
}
}
app = Flask(__name__)
middleware = FlaskMiddleware(
app,
exporter=AzureExporter(connection_string="InstrumentationKey=<your-ikey-here>"),
sampler=ProbabilitySampler(rate=1.0),
)
@app.route('/')
def hello():
return 'Hello World!'
if __name__ == '__main__':
app.run(host='localhost', port=8080, threaded=True)
2. You can configure your flask middleware directly in the code. For requests from urls that you do not wish
to track, add them to BLACKLIST_PATHS .
app.config['OPENCENSUS'] = {
'TRACE': {
'SAMPLER': 'opencensus.trace.samplers.ProbabilitySampler(rate=1.0)',
'EXPORTER': '''opencensus.ext.azure.trace_exporter.AzureExporter(
service_name='foobar',
)''',
'BLACKLIST_PATHS': 'https://example.com', <--- This site will not be traced if a request is
sent to it.
}
}
config.add_tween('opencensus.ext.pyramid'
'.pyramid_middleware.OpenCensusTweenFactory')
2. You can configure your pyramid tween directly in the code. For requests from urls that you do not wish to
track, add them to BLACKLIST_PATHS .
settings = {
'OPENCENSUS': {
'TRACE': {
'SAMPLER': 'opencensus.trace.samplers.ProbabilitySampler(rate=1.0)',
'EXPORTER': '''opencensus.ext.azure.trace_exporter.AzureExporter(
service_name='foobar',
)''',
'BLACKLIST_PATHS': 'https://example.com', <--- This site will not be traced if a request is
sent to it.
}
}
}
config = Configurator(settings=settings)
Next steps
Application Map
Availability
Search
Log (Analytics) query
Transaction diagnostics
Application Insights for web pages
1/19/2020 • 12 minutes to read • Edit Online
Find out about the performance and usage of your web page or app. If you add Application Insights to
your page script, you get timings of page loads and AJAX calls, counts, and details of browser exceptions
and AJAX failures, as well as users and session counts. All these can be segmented by page, client OS and
browser version, geo location, and other dimensions. You can set alerts on failure counts or slow page
loading. And by inserting trace calls in your JavaScript code, you can track how the different features of
your web page application are used.
Application Insights can be used with any web pages - you just add a short piece of JavaScript. If your
web service is Java or ASP.NET, you can use the server-side SDKs in conjunction with the client-side
JavaScript SDK to get an end-to-end understanding of your app's performance.
IMPORTANT
Only use one method to add the JavaScript SDK to your application. If you use the NPM Setup, don't use the
Snippet and vice versa.
NOTE
NPM Setup installs the JavaScript SDK as a dependency to your project, enabling IntelliSense, whereas the Snippet
fetches the SDK at runtime. Both support the same features. However, developers who desire more custom events
and configuration generally opt for NPM Setup whereas users looking for quick enablement of out-of-the-box web
analytics opt for the Snippet.
<script type="text/javascript">
var sdkInstance="appInsightsSDK";window[sdkInstance]="appInsights";var
aiName=window[sdkInstance],aisdk=window[aiName]||function(e){function n(e){t[e]=function(){var
n=arguments;t.queue.push(function(){t[e].apply(t,n)})}}var t={config:e};t.initialize=!0;var
i=document,a=window;setTimeout(function(){var
n=i.createElement("script");n.src=e.url||"https://az416426.vo.msecnd.net/scripts/b/ai.2.min.js",i.get
ElementsByTagName("script")[0].parentNode.appendChild(n)});try{t.cookie=i.cookie}catch(e){}t.queue=
[],t.version=2;for(var r=
["Event","PageView","Exception","Trace","DependencyData","Metric","PageViewPerformance"];r.length;)n(
"track"+r.pop());n("startTrackPage"),n("stopTrackPage");var
s="Track"+r[0];if(n("start"+s),n("stop"+s),n("addTelemetryInitializer"),n("setAuthenticatedUserContex
t"),n("clearAuthenticatedUserContext"),n("flush"),t.SeverityLevel=
{Verbose:0,Information:1,Warning:2,Error:3,Critical:4},!
(!0===e.disableExceptionTracking||e.extensionConfig&&e.extensionConfig.ApplicationInsightsAnalytics&&
!0===e.extensionConfig.ApplicationInsightsAnalytics.disableExceptionTracking)){n("_"+
(r="onerror"));var o=a[r];a[r]=function(e,n,i,a,s){var c=o&&o(e,n,i,a,s);return!0!==c&&t["_"+r]
({message:e,url:n,lineNumber:i,columnNumber:a,error:s}),c},e.autoExceptionInstrumented=!0}return t}(
{
instrumentationKey:"INSTRUMENTATION_KEY"
}
);window[aiName]=aisdk,aisdk.queue&&0===aisdk.queue.length&&aisdk.trackPageView({});
</script>
Configuration
Most configuration fields are named such that they can be defaulted to false. All fields are optional except
for instrumentationKey .
Currently, we offer a separate React plugin which you can initialize with this SDK. It will also accomplish
route change tracking for you, as well as collect other React specific telemetry.
React extensions
EX TENSIONS
React
React Native
You can also view your data from the JavaScript SDK via the Browser experience in the portal.
Select Browser and then choose Failures or Performance.
Performance
Dependencies
Analytics
To query your telemetry collected by the JavaScript SDK, select the View in Logs (Analytics) button. By
adding a where statement of client_Type == "Browser" , you will only see data from the JavaScript SDK
and any server-side telemetry collected by other SDKs will be excluded.
This version comes with the bare minimum number of features and functionalities and relies on you to
build it up as you see fit. For example, it performs no autocollection (uncaught exceptions, AJAX, etc.). The
APIs to send certain telemetry types, like trackTrace , trackException , etc., are not included in this
version, so you will need to provide your own wrapper. The only API that is available is track . A sample
is located here.
Examples
For runnable examples, see Application Insights JavaScript SDK Samples
If you're using the current application insights PRODUCTION SDK (1.0.20) and want to see if the new
SDK works in runtime, update the URL depending on your current SDK loading scenario.
Download via CDN scenario: Update the code snippet that you currently use to point to the
following URL:
"https://az416426.vo.msecnd.net/scripts/b/ai.2.min.js"
npm scenario: Call downloadAndSetup to download the full ApplicationInsights script from CDN
and initialize it with instrumentation key:
appInsights.downloadAndSetup({
instrumentationKey: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx",
url: "https://az416426.vo.msecnd.net/scripts/b/ai.2.min.js"
});
Test in internal environment to verify monitoring telemetry is working as expected. If all works, update
your API signatures appropriately to SDK V2 version and deploy in your production environments.
SDK performance/overhead
At just 25 KB gzipped, and taking only ~15 ms to initialize, Application Insights adds a negligible amount
of loadtime to your website. By using the snippet, minimal components of the library are quickly loaded.
In the meantime, the full script is downloaded in the background.
While the script is downloading from the CDN, all tracking of your page is queued. Once the downloaded
script finishes asynchronously initializing, all events that were queued are tracked. As a result, you will not
lose any telemetry during the entire life cycle of your page. This setup process provides your page with a
seamless analytics system, invisible to your users.
Summary:
25 KB gzipped
15 ms overall initialization time
Zero tracking missed during life cycle of page
Browser support
Chrome Latest ✔ Firefox Latest ✔ IE 9+ & Edge ✔ Opera Latest ✔ Safari Latest ✔
Open-source SDK
The Application Insights JavaScript SDK is open-source to view the source code or to contribute to the
project visit the official GitHub repository.
Next steps
Track usage
Custom events and metrics
Build-measure-learn
Application Insights API for custom events and
metrics
12/19/2019 • 26 minutes to read • Edit Online
Insert a few lines of code in your application to find out what users are doing with it, or to help diagnose
issues. You can send telemetry from device and desktop apps, web clients, and web servers. Use the
Azure Application Insights core telemetry API to send custom events and metrics, and your own
versions of standard telemetry. This API is the same API that the standard Application Insights data
collectors use.
API summary
The core API is uniform across all platforms, apart from a few variations like GetMetric (.NET only).
You can attach properties and metrics to most of these telemetry calls.
Visual Basic
Java
Node.js
TelemetryClient is thread-safe.
For ASP.NET and Java projects, incoming HTTP Requests are automatically captured. You might want to
create additional instances of TelemetryClient for other module of your app. For instance, you may have
one TelemetryClient instance in your middleware class to report business logic events. You can set
properties such as UserId and DeviceId to identify the machine. This information is attached to all events
that the instance sends.
C#
TelemetryClient.Context.User.Id = "...";
TelemetryClient.Context.Device.Id = "...";
Java
telemetry.getContext().getUser().setId("...");
telemetry.getContext().getDevice().setId("...");
In Node.js projects, you can use new applicationInsights.TelemetryClient(instrumentationKey?) to create
a new instance, but this is recommended only for scenarios that require isolated configuration from the
singleton defaultClient .
TrackEvent
In Application Insights, a custom event is a data point that you can display in Metrics Explorer as an
aggregated count, and in Diagnostic Search as individual occurrences. (It isn't related to MVC or other
framework "events.")
Insert TrackEvent calls in your code to count various events. How often users choose a particular
feature, how often they achieve particular goals, or maybe how often they make particular types of
mistakes.
For example, in a game app, send an event whenever a user wins the game:
JavaScript
appInsights.trackEvent({name:"WinGame"});
C#
telemetry.TrackEvent("WinGame");
Visual Basic
telemetry.TrackEvent("WinGame")
Java
telemetry.trackEvent("WinGame");
Node.js
telemetry.trackEvent({name: "WinGame"});
GetMetric
Examples
C#
namespace User.Namespace.Example01
{
{
using System;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
/// <summary>
/// Most simple cases are one-liners.
/// This is all possible without even importing an additional namespace.
/// </summary>
// Recall how you send custom telemetry with Application Insights in other cases, e.g.
Events.
// The following will result in an EventTelemetry object to be sent to the cloud right
away.
TelemetryClient client = new TelemetryClient();
client.TrackEvent("SomethingInterestingHappened");
// Metrics work very similar. However, the value is not sent right away.
// It is aggregated with other values for the same metric, and the resulting summary
(aka "aggregate" is sent automatically every minute.
// To mark this difference, we use a pattern that is similar, but different from the
established TrackXxx(..) pattern that sends telemetry right away:
client.GetMetric("CowsSold").TrackValue(42);
animalsSold.TrackValue(42, "Pigs");
animalsSold.TrackValue(24, "Horses");
// The values for Pigs and Horses will be aggregated separately from each other and will
result in two distinct aggregates.
// You can control the maximum number of number data series per metric (and thus your
resource usage and cost).
// The default limits are no more than 1000 total data series per metric, and no more
than 100 different values per dimension.
// We discuss elsewhere how to change them.
// We use a common .NET pattern: TryXxx(..) to make sure that the limits are observed.
// If the limits are already reached, Metric.TrackValue(..) will return False and the
value will not be tracked. Otherwise it will return True.
// This is particularly useful if the data for a metric originates from user input, e.g.
a file:
if (!animalsSold.TrackValue(count, species))
{
client.TrackTrace($"Data series or dimension cap was reached for metric
{animalsSold.Identifier.MetricId}.", SeverityLevel.Error);
}
// You can inspect a metric object to reason about its current state. For example:
// You can inspect a metric object to reason about its current state. For example:
int currentNumberOfSpecies = animalsSold.GetDimensionValues(1).Count;
}
TrackMetric
NOTE
Microsoft.ApplicationInsights.TelemetryClient.TrackMetric is not the preferred method for sending metrics.
Metrics should always be pre-aggregated across a time period before being sent. Use one of the GetMetric(..)
overloads to get a metric object for accessing SDK pre-aggregation capabilities. If you are implementing your
own pre-aggregation logic, you can use the TrackMetric() method to send the resulting aggregates. If your
application requires sending a separate telemetry item at every occasion without aggregation across time, you
likely have a use case for event telemetry; see TelemetryClient.TrackEvent
(Microsoft.ApplicationInsights.DataContracts.EventTelemetry).
Application Insights can chart metrics that are not attached to particular events. For example, you could
monitor a queue length at regular intervals. With metrics, the individual measurements are of less
interest than the variations and trends, and so statistical charts are useful.
In order to send metrics to Application Insights, you can use the TrackMetric(..) API. There are two
ways to send a metric:
Single value. Every time you perform a measurement in your application, you send the
corresponding value to Application Insights. For example, assume that you have a metric
describing the number of items in a container. During a particular time period, you first put three
items into the container and then you remove two items. Accordingly, you would call
TrackMetric twice: first passing the value 3 and then the value -2 . Application Insights stores
both values on your behalf.
Aggregation. When working with metrics, every single measurement is rarely of interest. Instead
a summary of what happened during a particular time period is important. Such a summary is
called aggregation. In the above example, the aggregate metric sum for that time period is 1
and the count of the metric values is 2 . When using the aggregation approach, you only invoke
TrackMetric once per time period and send the aggregate values. This is the recommended
approach since it can significantly reduce the cost and performance overhead by sending fewer
data points to Application Insights, while still collecting all relevant information.
Examples
Single values
To send a single metric value:
JavaScript
appInsights.trackMetric("queueLength", 42.0);
C#
Java
telemetry.trackMetric("queueLength", 42.0);
Node.js
Page views
In a device or webpage app, page view telemetry is sent by default when each screen or page is loaded.
But you can change that to track page views at additional or different times. For example, in an app that
displays tabs or blades, you might want to track a page whenever the user opens a new blade.
User and session data is sent as properties along with page views, so the user and session charts come
alive when there is page view telemetry.
Custom page views
JavaScript
appInsights.trackPageView("tab1");
C#
telemetry.TrackPageView("GameReviewPage");
Visual Basic
telemetry.TrackPageView("GameReviewPage")
Java
telemetry.trackPageView("GameReviewPage");
If you have several tabs within different HTML pages, you can specify the URL too:
appInsights.trackPageView("tab1", "http://fabrikam.com/page1.htm");
JavaScript
...
The name that you use as the first parameter associates the start and stop calls. It defaults to the current
page name.
The resulting page load durations displayed in Metrics Explorer are derived from the interval between
the start and stop calls. It's up to you what interval you actually time.
Page telemetry in Analytics
In Analytics two tables show data from browser operations:
The pageViews table contains data about the URL and page title
The browserTimings table contains data about client performance, such as the time taken to process
the incoming data
To find how long the browser takes to process different pages:
browserTimings
| summarize avg(networkDuration), avg(processingDuration), avg(totalDuration) by name
pageViews
| summarize count() by client_Browser
pageViews
| join (dependencies) on operation_Id
TrackRequest
The server SDK uses TrackRequest to log HTTP requests.
You can also call it yourself if you want to simulate requests in a context where you don't have the web
service module running.
However, the recommended way to send request telemetry is where the request acts as an operation
context.
Operation context
You can correlate telemetry items together by associating them with operation context. The standard
request-tracking module does this for exceptions and other events that are sent while an HTTP request
is being processed. In Search and Analytics, you can easily find any events associated with the request
using its operation ID.
See Telemetry correlation in Application Insights for more details on correlation.
When tracking telemetry manually, the easiest way to ensure telemetry correlation by using this pattern:
C#
Along with setting an operation context, StartOperation creates a telemetry item of the type that you
specify. It sends the telemetry item when you dispose the operation, or if you explicitly call
StopOperation . If you use RequestTelemetry as the telemetry type, its duration is set to the timed
interval between start and stop.
Telemetry items reported within a scope of operation become 'children' of such operation. Operation
contexts could be nested.
In Search, the operation context is used to create the Related Items list:
See Track custom operations with Application Insights .NET SDK for more information on custom
operations tracking.
Requests in Analytics
In Application Insights Analytics, requests show up in the requests table.
If sampling is in operation, the itemCount property will show a value greater than 1. For example
itemCount==10 means that of 10 calls to trackRequest(), the sampling process only transmitted one of
them. To get a correct count of requests and average duration segmented by request names, use code
such as:
requests
| summarize count = sum(itemCount), avgduration = avg(duration) by name
TrackException
Send exceptions to Application Insights:
To count them, as an indication of the frequency of a problem.
To examine individual occurrences.
The reports include the stack traces.
C#
try
{
...
}
catch (Exception ex)
{
telemetry.TrackException(ex);
}
Java
try {
...
} catch (Exception ex) {
telemetry.trackException(ex);
}
JavaScript
try
{
...
}
catch (ex)
{
appInsights.trackException(ex);
}
Node.js
try
{
...
}
catch (ex)
{
telemetry.trackException({exception: ex});
}
The SDKs catch many exceptions automatically, so you don't always have to call TrackException explicitly.
ASP.NET: Write code to catch exceptions.
Java EE: Exceptions are caught automatically.
JavaScript: Exceptions are caught automatically. If you want to disable automatic collection, add a line
to the code snippet that you insert in your webpages:
({
instrumentationKey: "your key",
disableExceptionTracking: true
})
Exceptions in Analytics
In Application Insights Analytics, exceptions show up in the exceptions table.
If sampling is in operation, the itemCount property shows a value greater than 1. For example
itemCount==10 means that of 10 calls to trackException(), the sampling process only transmitted one of
them. To get a correct count of exceptions segmented by type of exception, use code such as:
exceptions
| summarize sum(itemCount) by type
Most of the important stack information is already extracted into separate variables, but you can pull
apart the details structure to get more. Since this structure is dynamic, you should cast the result to
the type you expect. For example:
exceptions
| extend method2 = tostring(details[0].parsedStack[1].method)
exceptions
| join (requests) on operation_Id
TrackTrace
Use TrackTrace to help diagnose problems by sending a "breadcrumb trail" to Application Insights. You
can send chunks of diagnostic data and inspect them in Diagnostic Search.
In .NET Log adapters use this API to send third-party logs to the portal.
In Java for Standard loggers like Log4J, Logback use Application Insights Log4j or Logback Appenders
to send third-party logs to the portal.
C#
Java
Node.js
telemetry.trackTrace({
message: message,
severity: applicationInsights.Contracts.SeverityLevel.Warning,
properties: properties
});
Client/Browser-side JavaScript
PARAMETER DESCRIPTION
You can search on message content, but (unlike property values) you can't filter on it.
The size limit on message is much higher than the limit on properties. An advantage of TrackTrace is that
you can put relatively long data in the message. For example, you can encode POST data there.
In addition, you can add a severity level to your message. And, like other telemetry, you can add
property values to help you filter or search for different sets of traces. For example:
C#
Java
In Search, you can then easily filter out all the messages of a particular severity level that relate to a
particular database.
Traces in Analytics
In Application Insights Analytics, calls to TrackTrace show up in the traces table.
If sampling is in operation, the itemCount property shows a value greater than 1. For example
itemCount==10 means that of 10 calls to trackTrace() , the sampling process only transmitted one of
them. To get a correct count of trace calls, you should use therefore code such as
traces | summarize sum(itemCount) .
TrackDependency
Use the TrackDependency call to track the response times and success rates of calls to an external piece
of code. The results appear in the dependency charts in the portal. The code snippet below needs to be
added wherever a dependency call is made.
C#
Java
boolean success = false;
Instant startTime = Instant.now();
try {
success = dependency.call();
}
finally {
Instant endTime = Instant.now();
Duration delta = Duration.between(startTime, endTime);
RemoteDependencyTelemetry dependencyTelemetry = new RemoteDependencyTelemetry("My Dependency",
"myCall", delta, success);
RemoteDependencyTelemetry.setTimeStamp(startTime);
RemoteDependencyTelemetry.trackDependency(dependencyTelemetry);
}
Node.js
Remember that the server SDKs include a dependency module that discovers and tracks certain
dependency calls automatically--for example, to databases and REST APIs. You have to install an agent
on your server to make the module work.
In Java, certain dependency calls can be automatically tracked using Java Agent.
You use this call if you want to track calls that the automated tracking doesn't catch, or if you don't want
to install the agent.
To turn off the standard dependency-tracking module in C#, edit ApplicationInsights.config and delete
the reference to DependencyCollector.DependencyTrackingTelemetryModule . In Java, please do not install
java agent if you do not want to collect standard dependencies automatically.
Dependencies in Analytics
In Application Insights Analytics, trackDependency calls show up in the dependencies table.
If sampling is in operation, the itemCount property shows a value greater than 1. For example
itemCount==10 means that of 10 calls to trackDependency(), the sampling process only transmitted
one of them. To get a correct count of dependencies segmented by target component, use code such as:
dependencies
| summarize sum(itemCount) by target
Flushing data
Normally, the SDK sends data at fixed intervals (typically 30 secs) or whenever buffer is full (typically
500 items). However, in some cases, you might want to flush the buffer--for example, if you are using
the SDK in an application that shuts down.
C#
telemetry.Flush();
// Allow some time for flushing before shutdown.
System.Threading.Thread.Sleep(5000);
Java
telemetry.flush();
//Allow some time for flushing before shutting down
Thread.sleep(5000);
Node.js
telemetry.flush();
Authenticated users
In a web app, users are (by default) identified by cookies. A user might be counted more than once if
they access your app from a different machine or browser, or if they delete cookies.
If users sign in to your app, you can get a more accurate count by setting the authenticated user ID in
the browser code:
JavaScript
It isn't necessary to use the user's actual sign-in name. It only has to be an ID that is unique to that user.
It must not include spaces or any of the characters ,;=| .
The user ID is also set in a session cookie and sent to the server. If the server SDK is installed, the
authenticated user ID is sent as part of the context properties of both client and server telemetry. You
can then filter and search on it.
If your app groups users into accounts, you can also pass an identifier for the account (with the same
character restrictions).
appInsights.setAuthenticatedUserContext(validatedId, accountId);
In Metrics Explorer, you can create a chart that counts Users, Authenticated, and User accounts.
You can also search for client data points with specific user names and accounts.
appInsights.trackPageView
("page name", "http://fabrikam.com/pageurl.html",
// String properties:
{Game: currentGame.name, Difficulty: currentGame.difficulty},
// Numeric metrics:
{Score: currentGame.score, Opponents: currentGame.opponentCount}
);
C#
Node.js
Visual Basic
Java
Map<String, String> properties = new HashMap<String, String>();
properties.put("game", currentGame.getName());
properties.put("difficulty", currentGame.getDifficulty());
NOTE
Take care not to log personally identifiable information in properties.
event.Name = "WinGame";
event.Metrics["processingTime"] = stopwatch.Elapsed.TotalMilliseconds;
event.Properties["game"] = currentGame.Name;
event.Properties["difficulty"] = currentGame.Difficulty;
event.Metrics["Score"] = currentGame.Score;
event.Metrics["Opponents"] = currentGame.Opponents.Length;
telemetry.TrackEvent(event);
WARNING
Don't reuse the same telemetry item instance ( event in this example) to call Track*() multiple times. This may
cause telemetry to be sent with incorrect configuration.
requests
| summarize sum(itemCount), avg(todouble(customMeasurements.score)) by
tostring(customDimensions.game)
Notice that:
When you extract a value from the customDimensions or customMeasurements JSON, it has
dynamic type, and so you must cast it tostring or todouble .
To take account of the possibility of sampling, you should use sum(itemCount) , not count() .
Timing events
Sometimes you want to chart how long it takes to perform an action. For example, you might want to
know how long users take to consider choices in a game. You can use the measurement parameter for
this.
C#
stopwatch.Stop();
Java
using Microsoft.ApplicationInsights.DataContracts;
Visual Basic
Java
import com.microsoft.applicationinsights.TelemetryClient;
import com.microsoft.applicationinsights.TelemetryContext;
...
gameTelemetry.TrackEvent("WinGame");
Node.js
gameTelemetry.TrackEvent({name: "WinGame"});
Individual telemetry calls can override the default values in their property dictionaries.
For JavaScript web clients, use JavaScript telemetry initializers.
To add properties to all telemetry, including the data from standard collection modules, implement
ITelemetryInitializer .
Disabling telemetry
To dynamically stop and start the collection and transmission of telemetry:
C#
using Microsoft.ApplicationInsights.Extensibility;
TelemetryConfiguration.Active.DisableTelemetry = true;
Java
telemetry.getConfiguration().setTrackingDisabled(true);
telemetry.config.disableAppInsights = true;
applicationInsights.setup()
.setAutoCollectRequests(false)
.setAutoCollectPerformance(false)
.setAutoCollectExceptions(false)
.setAutoCollectDependencies(false)
.setAutoCollectConsole(false)
.start();
Developer mode
During debugging, it's useful to have your telemetry expedited through the pipeline so that you can see
results immediately. You also get additional messages that help you trace any problems with the
telemetry. Switch it off in production, because it may slow down your app.
C#
TelemetryConfiguration.Active.TelemetryChannel.DeveloperMode = true;
Visual Basic
TelemetryConfiguration.Active.TelemetryChannel.DeveloperMode = True
Node.js
For Node.js, you can enable developer mode by enabling internal logging via setInternalLogging and
setting maxBatchSize to 0, which causes your telemetry to be sent as soon as it is collected.
applicationInsights.setup("ikey")
.setInternalLogging(true, true)
.start()
applicationInsights.defaultClient.config.maxBatchSize = 0;
JavaScript
appInsights.config.instrumentationKey = myKey;
In webpages, you might want to set it from the web server's state, rather than coding it literally into the
script. For example, in a webpage generated in an ASP.NET app:
JavaScript in Razor
<script type="text/javascript">
// Standard Application Insights webpage script:
var appInsights = window.appInsights || function(config){ ...
// Modify this part:
}({instrumentationKey:
// Generate from server property:
@Microsoft.ApplicationInsights.Extensibility.
TelemetryConfiguration.Active.InstrumentationKey;
}) // ...
if (instrumentationKey != null)
{
TelemetryConfiguration.getActive().setInstrumentationKey(instrumentationKey);
}
TelemetryContext
TelemetryClient has a Context property, which contains values that are sent along with all telemetry
data. They are normally set by the standard telemetry modules, but you can also set them yourself. For
example:
telemetry.Context.Operation.Name = "MyOperationName";
If you set any of these values yourself, consider removing the relevant line from
ApplicationInsights.config, so that your values and the standard values don't get confused.
Component: The app and its version.
Device: Data about the device where the app is running. (In web apps, this is the server or client
device that the telemetry is sent from.)
InstrumentationKey: The Application Insights resource in Azure where the telemetry appears. It's
usually picked up from ApplicationInsights.config.
Location: The geographic location of the device.
Operation: In web apps, the current HTTP request. In other app types, you can set this to group
events together.
ID: A generated value that correlates different events, so that when you inspect any event in
Diagnostic Search, you can find related items.
Name: An identifier, usually the URL of the HTTP request.
SyntheticSource: If not null or empty, a string that indicates that the source of the request
has been identified as a robot or web test. By default, it is excluded from calculations in Metrics
Explorer.
Properties: Properties that are sent with all telemetry data. It can be overridden in individual Track*
calls.
Session: The user's session. The ID is set to a generated value, which is changed when the user has
not been active for a while.
User: User information.
Limits
There are some limits on the number of metrics and events per application, that is, per instrumentation
key. Limits depend on the pricing plan that you choose.
Total data per day 100 GB You can reduce data by setting a
cap. If you need more data, you can
increase the limit in the portal, up
to 1,000 GB. For capacities greater
than 1,000 GB, send email to
AIDataCap@microsoft.com.
For more information, see About pricing and quotas in Application Insights.
To avoid hitting the data rate limit, use sampling.
To determine how long data is kept, see Data retention and privacy.
Reference docs
ASP.NET reference
Java reference
JavaScript reference
SDK code
ASP.NET Core SDK
ASP.NET
Windows Server packages
Java SDK
Node.js SDK
JavaScript SDK
Questions
What exceptions might Track_ () calls throw?
None. You don't need to wrap them in try-catch clauses. If the SDK encounters problems, it will
log messages in the debug console output and--if the messages get through--in Diagnostic
Search.
Is there a REST API to get data from the portal?
Yes, the data access API. Other ways to extract data include export from Analytics to Power BI and
continuous export.
Next steps
Search events and logs
Troubleshooting
Troubleshooting no data - Application Insights for
.NET/.NET Core
12/5/2019 • 11 minutes to read • Edit Online
No performance data
Performance data (CPU, IO rate, and so on) is available for Java web services, Windows desktop apps, IIS web
apps and services if you install status monitor, and Azure Cloud Services. you'll find it under Settings, Servers.
NOTE
If you need the first 3 octets of the IP address, you can use a telemetry initializer to add a custom attribute. This does not
affect data collected prior to February 5, 2018.
Troubleshooting Logs
Follow these instructions to capture troubleshooting logs for your framework.
.NET Framework
1. Install the Microsoft.AspNet.ApplicationInsights.HostingStartup package from NuGet. The version you
install must match the current installed version of Microsoft.ApplicationInsighs
2. Modify your applicationinsights.config file to include the following:
<TelemetryModules>
<Add Type="Microsoft.ApplicationInsights.Extensibility.HostingStartup.FileDiagnosticsTelemetryModule,
Microsoft.AspNet.ApplicationInsights.HostingStartup">
<Severity>Verbose</Severity>
<LogFileName>mylog.txt</LogFileName>
<LogFilePath>C:\\SDKLOGS</LogFilePath>
</Add>
</TelemetryModules>
services.AddSingleton<ITelemetryModule, FileDiagnosticsTelemetryModule>();
services.ConfigureTelemetryModule<FileDiagnosticsTelemetryModule>( (module, options) => {
module.LogFilePath = "C:\\SDKLOGS";
module.LogFileName = "mylog.txt";
module.Severity = "Verbose";
} );
Enabling monitoring on your .NET based web applications running on Azure virtual machines and Azure virtual
machine scale sets is now easier than ever. Get all the benefits of using Application Insights without modifying
your code.
This article walks you through enabling Application Insights monitoring using the Application Insights Agent
and provides preliminary guidance for automating the process for large-scale deployments.
IMPORTANT
Azure Application Insights Agent for .NET is currently in public preview. This preview version is provided without a service-
level agreement, and we don't recommend it for production workloads. Some features might not be supported, and some
might have constrained capabilities. For more information, see Supplemental Terms of Use for Microsoft Azure Previews.
NOTE
Currently only .Net IIS-hosted applications are supported. Use an SDK to instrument ASP.NET Core, Java,
and Node.js applications hosted on an Azure virtual machines and virtual machine scale sets.
NOTE
New to powershell? Check out the Get Started Guide.
Install or update the Application Insights Agent as an extension for Azure virtual machines
$publicCfgJsonString = '
{
"redfieldConfiguration": {
"instrumentationKeyMap": {
"filters": [
{
"appFilter": ".*",
"machineFilter": ".*",
"virtualPathFilter": ".*",
"instrumentationSettings" : {
"connectionString": "InstrumentationKey=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
}
}
]
}
}
}
';
$privateCfgJsonString = '{}';
NOTE
You may install or update the Application Insights Agent as an extension across multiple Virtual Machines at-scale using a
Powershell loop.
Get-AzResource -ResourceId
"/subscriptions/<mySubscriptionId>/resourceGroups/<myVmResourceGroup>/providers/Microsoft.Compute/virtualMac
hines/<myVmName>/extensions"
# Name : ApplicationMonitoring
# ResourceGroupName : <myVmResourceGroup>
# ResourceType : Microsoft.Compute/virtualMachines/extensions
# Location : southcentralus
# ResourceId :
/subscriptions/<mySubscriptionId>/resourceGroups/<myVmResourceGroup>/providers/Microsoft.Compute/virtualMach
ines/<myVmName>/extensions/ApplicationMonitoring
You may also view installed extensions in the Azure virtual machine blade in the Portal.
NOTE
Verify installation by clicking on Live Metrics Stream within the Application Insights Resource associated with the
connection string you used to deploy the Application Insights Agent Extension. If you are sending data from multiple
Virtual Machines, select the target Azure virtual machines under Server Name. It may take up to a minute for data to
begin flowing.
# Note: depending on your update policy, you might need to run Update-AzVmssInstance for each instance
Uninstall application monitoring extension from Azure virtual machine scale sets
# Note: depending on your update policy, you might need to run Update-AzVmssInstance for each instance
Query application monitoring extension status for Azure virtual machine scale sets
Get list of installed extensions for Azure virtual machine scale sets
Get-AzResource -ResourceId
/subscriptions/<mySubscriptionId>/resourceGroups/<myResourceGroup>/providers/Microsoft.Compute/virtualMachin
eScaleSets/<myVmssName>/extensions
# Name : ApplicationMonitoringWindows
# ResourceGroupName : <myResourceGroup>
# ResourceType : Microsoft.Compute/virtualMachineScaleSets/extensions
# Location :
# ResourceId :
/subscriptions/<mySubscriptionId>/resourceGroups/<myResourceGroup>/providers/Microsoft.Compute/virtualMachin
eScaleSets/<myVmssName>/extensions/ApplicationMonitoringWindows
Troubleshooting
Find troubleshooting tips for Application Insights Monitoring Agent Extension for .NET applications running on
Azure virtual machines and virtual machine scale sets.
NOTE
.NET Core, Java, and Node.js applications are only supported on Azure virtual machines and Azure virtual machine scale
sets via manual SDK based instrumentation and therefore the steps below do not apply to these scenarios.
C:\WindowsAzure\Logs\Plugins\Microsoft.Azure.Diagnostics.ApplicationMonitoringWindows\<version>\
Next steps
Learn how to deploy an application to an Azure virtual machine scale set.
Set up Availability web tests to be alerted if your endpoint is down.
Monitor Azure App Service performance
12/12/2019 • 12 minutes to read • Edit Online
Enabling monitoring on your ASP.NET and ASP.NET Core based web applications running on Azure App
Services is now easier than ever. Whereas previously you needed to manually install a site extension, the latest
extension/agent is now built into the app service image by default. This article will walk you through enabling
Application Insights monitoring as well as provide preliminary guidance for automating the process for large-
scale deployments.
NOTE
Manually adding an Application Insights site extension via Development Tools > Extensions is deprecated. This method
of extension installation was dependent on manual updates for each new version. The latest stable release of the
extension is now preinstalled as part of the App Service image. The files are located in
d:\Program Files (x86)\SiteExtensions\ApplicationInsightsAgent and are automatically updated with each stable
release. If you follow the agent based instructions to enable monitoring below, it will automatically remove the deprecated
extension for you.
NOTE
If both agent-based monitoring and manual SDK-based instrumentation is detected, only the manual instrumentation
settings will be honored. This is to prevent duplicate data from being sent. To learn more about this, check out the
troubleshooting section below.
NOTE
The combination of APPINSIGHTS_JAVASCRIPT_ENABLED and urlCompression is not supported. For more info see the
explanation in the troubleshooting section.
1. Select Application Insights in the Azure control panel for your app service.
Choose to create a new resource, unless you already set up an Application Insights resource for
this application.
NOTE
When you click OK to create the new resource you will be prompted to Apply monitoring settings.
Selecting Continue will link your new Application Insights resource to your app service, doing so will also
trigger a restart of your app service.
2. After specifying which resource to use, you can choose how you want application insights to collect data
per platform for your application. ASP.NET app monitoring is on-by-default with two different levels of
collection.
For the list of supported adaptive sampling telemetry processor settings, you can consult the code
and associated documentation.
Value: true
To disable client-side monitoring either remove the associated key value pair from the Application settings, or
set the value to false.
Automate monitoring
In order to enable telemetry collection with Application Insights, only the Application settings need to be set:
For an example of an Azure Resource Manager template with Application settings configured for Application
Insights, this template can be helpful, specifically the section starting on line 238.
Automate the creation of an Application Insights resource and link to your newly created App Service.
To create an Azure Resource Manager template with all the default Application Insights settings configured,
begin the process as if you were going to create a new Web App with Application Insights enabled.
Select Automation options
This option generates the latest Azure Resource Manager template with all required settings configured.
Below is a sample, replace all instances of AppMonitoredSite with your site name:
{
"resources": [
{
"name": "[parameters('name')]",
"type": "Microsoft.Web/sites",
"properties": {
"siteConfig": {
"appSettings": [
{
"name": "APPINSIGHTS_INSTRUMENTATIONKEY",
"value": "[reference('microsoft.insights/components/AppMonitoredSite', '2015-
05-01').InstrumentationKey]"
},
{
"name": "ApplicationInsightsAgent_EXTENSION_VERSION",
"value": "~2"
}
]
},
"name": "[parameters('name')]",
"serverFarmId": "[concat('/subscriptions/',
parameters('subscriptionId'),'/resourcegroups/', parameters('serverFarmResourceGroup'),
'/providers/Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]",
"hostingEnvironment": "[parameters('hostingEnvironment')]"
},
"dependsOn": [
"[concat('Microsoft.Web/serverfarms/', parameters('hostingPlanName'))]",
"microsoft.insights/components/AppMonitoredSite"
],
"apiVersion": "2016-03-01",
"location": "[parameters('location')]"
},
{
"apiVersion": "2016-09-01",
"name": "[parameters('hostingPlanName')]",
"type": "Microsoft.Web/serverfarms",
"location": "[parameters('location')]",
"properties": {
"name": "[parameters('hostingPlanName')]",
"workerSizeId": "[parameters('workerSize')]",
"numberOfWorkers": "1",
"hostingEnvironment": "[parameters('hostingEnvironment')]"
},
"sku": {
"Tier": "[parameters('sku')]",
"Name": "[parameters('skuCode')]"
"Name": "[parameters('skuCode')]"
}
},
{
"apiVersion": "2015-05-01",
"name": "AppMonitoredSite",
"type": "microsoft.insights/components",
"location": "West US 2",
"properties": {
"ApplicationId": "[parameters('name')]",
"Request_Source": "IbizaWebAppExtensionCreate"
}
}
],
"parameters": {
"name": {
"type": "string"
},
"hostingPlanName": {
"type": "string"
},
"hostingEnvironment": {
"type": "string"
},
"location": {
"type": "string"
},
"sku": {
"type": "string"
},
"skuCode": {
"type": "string"
},
"workerSize": {
"type": "string"
},
"serverFarmResourceGroup": {
"type": "string"
},
"subscriptionId": {
"type": "string"
}
},
"$schema": "https://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
"contentVersion": "1.0.0.0"
}
NOTE
The template will generate application settings in “default” mode. This mode is performance optimized, though you can
modify the template to activate whichever features you prefer.
NOTE
Java applications are only supported on Azure App Services via manual SDK based instrumentation and therefore the
steps below do not apply to these scenarios.
If a similar value is not present, it means the application is not currently running or is not
supported. To ensure that the application is running, try manually visiting the application
url/application endpoints, which will allow the runtime information to become available.
Confirm that IKeyExists is true
If it is false, add `APPINSIGHTS_INSTRUMENTATIONKEY with your ikey guid to your
application settings.
Confirm that there are no entries for AppAlreadyInstrumented ,
AppContainsDiagnosticSourceAssembly , and AppContainsAspNetTelemetryCorrelationAssembly .
If any of these entries exist, remove the following packages from your application:
Microsoft.ApplicationInsights , System.Diagnostics.DiagnosticSource , and
Microsoft.AspNet.TelemetryCorrelation .
The table below provides a more detailed explanation of what these values mean, their underlying causes, and
recommended fixes:
AppAlreadyInstrumented:true This value indicates that the extension Remove the references. Some of these
detected that some aspect of the SDK references are added by default from
is already present in the Application, certain Visual Studio templates, and
and will back-off. It can be due to a older versions of Visual Studio may
reference to add references to
System.Diagnostics.DiagnosticSource Microsoft.ApplicationInsights .
,
Microsoft.AspNet.TelemetryCorrelation
, or
Microsoft.ApplicationInsights
AppAlreadyInstrumented:true If the application is targeting .NET Customers on .NET Core 2.1,2.2 are
Core 2.1 or 2.2, and refers to recommended to use
Microsoft.AspNetCore.All meta- Microsoft.AspNetCore.App meta-
package, then it brings in Application package instead.
Insights, and extension will back-off.
AppAlreadyInstrumented:true This value can also be caused by the Clean the app folder to ensure that
presence of the above dlls in the app these dlls are removed. Check both
folder from a previous deployment. your local app's bin directory, and the
wwwroot directory on the App Service.
(To check the wwwroot directory of
your App Service web app: Advanced
Tools (Kudu) > Debug console > CMD
> home\site\wwwroot).
IKeyExists:false This value indicates that the Make sure the setting is present in the
instrumentation key is not present in App Service application settings.
the AppSetting,
APPINSIGHTS_INSTRUMENTATIONKEY .
Possible causes: The values may have
been accidentally removed, forgot to
set the values in automation script,
etc.
Next steps
Run the profiler on your live app.
Azure Functions - monitor Azure Functions with Application Insights
Enable Azure diagnostics to be sent to Application Insights.
Monitor service health metrics to make sure your service is available and responsive.
Receive alert notifications whenever operational events happen or metrics cross a threshold.
Use Application Insights for JavaScript apps and web pages to get client telemetry from the browsers that
visit a web page.
Set up Availability web tests to be alerted if your site is down.
Application Insights for Azure cloud services
1/14/2020 • 10 minutes to read • Edit Online
Application Insights can monitor Azure cloud service apps for availability, performance, failures, and usage by
combining data from Application Insights SDKs with Azure Diagnostics data from your cloud services. With the
feedback you get about the performance and effectiveness of your app in the wild, you can make informed
choices about the direction of the design in each development lifecycle.
Prerequisites
Before you begin, you need:
An Azure subscription. Sign in with your Microsoft account for Windows, Xbox Live, or other Microsoft
cloud services.
Microsoft Azure tools 2.9 or later.
Developer Analytics Tools 7.10 or later.
This has the effect of inserting your Application Insights instrumentation keys into the files named
ServiceConfiguration.*.cscfg. Here is the Sample code.
If you want to vary the level of diagnostics information that's sent to Application Insights, you can do so by
editing the .cscfg files directly.
TelemetryConfiguration.Active.InstrumentationKey =
RoleEnvironment.GetConfigurationSettingValue("APPINSIGHTS_INSTRUMENTATIONKEY");
b. Repeat "step a" for each role in your app. See the examples:
Web role
Worker role
For webpages
4. Set the ApplicationInsights.config file to be copied always to the output directory.
A message in the .config file asks you to place the instrumentation key there. However, for cloud apps,
it's better to set it from the .cscfg file. This approach ensures that the role is correctly identified in the
portal.
<Startup>
<Task commandLine="AppInsightsAgent\InstallAgent.bat" executionContext="elevated"
taskType="simple">
<Environment>
<Variable name="ApplicationInsightsAgent.DownloadLink" value="http://go.microsoft.com/fwlink/?
LinkID=522371" />
<Variable name="RoleEnvironment.IsEmulated">
<RoleInstanceValue xpath="/RoleEnvironment/Deployment/@emulated" />
</Variable>
</Environment>
</Task>
</Startup>
2. Download InstallAgent.bat and InstallAgent.ps1, put them into the AppInsightsAgent folder on each role
project. Make sure to copy them to the output directory through Visual Studio file properties or build
scripts.
3. On all Worker Roles, add environment variables:
<Environment>
<Variable name="COR_ENABLE_PROFILING" value="1" />
<Variable name="COR_PROFILER" value="{324F817A-7420-4E6D-B3C1-143FBED6D855}" />
<Variable name="MicrosoftInstrumentationEngine_Host" value="{CA487940-57D2-10BF-11B2-
A3AD5A13CBC0}" />
</Environment>
More telemetry
The next sections discuss how to get additional telemetry from various aspects of your app.
Exceptions
For information about how to collect unhandled exceptions from various web app types, see Monitoring
exceptions in Application Insights.
The sample web role has MVC5 and Web API 2 controllers. The unhandled exceptions from the two are
captured with the following handlers:
AiHandleErrorAttribute set up for MVC5 controllers as shown in this example
AiWebApiExceptionLogger set up for Web API 2 controllers as shown in this example
For worker roles, you can track exceptions in two ways:
Use TrackException(ex).
If you have added the Application Insights trace listener NuGet package, you can use
System.Diagnostics.Trace to log exceptions as shown in this example.
Performance counters
The following counters are collected by default:
\Process(??APP_WIN32_PROC??)% Processor Time
\Memory\Available Bytes
.NET CLR Exceptions(??APP_CLR_PROC??)# of Exceps Thrown / sec
\Process(??APP_WIN32_PROC??)\Private Bytes
\Process(??APP_WIN32_PROC??)\IO Data Bytes/sec
\Processor(_Total)% Processor Time
For web roles, these counters are also collected:
\ASP.NET Applications(??APP_W3SVC_PROC??)\Requests/Sec
\ASP.NET Applications(??APP_W3SVC_PROC??)\Request Execution Time
\ASP.NET Applications(??APP_W3SVC_PROC??)\Requests In Application Queue
You can specify additional custom or other Windows performance counters by editing
ApplicationInsights.config as shown in this example.
Correlated telemetry for worker roles
For a rich diagnostics experience, you can view what led to a failed or high latency request. With web roles, the
SDK automatically sets up a correlation between related telemetry.
To achieve this view for worker roles, you can use a custom telemetry initializer to set a common Operation.Id
context attribute for all the telemetry. Doing so lets you view at a glance whether the latency or failure issue was
caused by a dependency or your code.
Here's how:
Set the correlationId into a CallContext as shown in this example. In this case, we are using the Request ID
as the correlationId.
Add a custom TelemetryInitializer implementation, to set the Operation.Id to the correlationId that was set
previously. For an example, see ItemCorrelationTelemetryInitializer.
Add the custom telemetry initializer. You could do so in the ApplicationInsights.config file or in code as
shown in this example.
Client telemetry
To get browser-based telemetry, such as page view counts, page load times, or script exceptions, and to write
custom telemetry in your page scripts, see Add the JavaScript SDK to your webpages.
Availability tests
To make sure your app stays live and responsive, Set up web tests.
Example
The example monitors a service that has a web role and two worker roles.
Video
Next steps
Configure sending Azure Diagnostics to Application Insights
Automatically create Application Insights resources
Automate Azure Diagnostics
Azure Functions
Monitor Azure Functions
1/20/2020 • 20 minutes to read • Edit Online
Azure Functions offers built-in integration with Azure Application Insights to monitor functions. This article shows
you how to configure Azure Functions to send system-generated log files to Application Insights.
We recommend using Application Insights because it collects log, performance, and error data. It automatically
detects performance anomalies and includes powerful analytics tools to help you diagnose issues and to
understand how your functions are used. It's designed to help you continuously improve performance and
usability. You can even use Application Insights during local function app project development. For more
information, see What is Application Insights?.
As the required Application Insights instrumentation is built into Azure Functions, all you need is a valid
instrumentation key to connect your function app to an Application Insights resource.
Name Unique app name It's easiest to use the same name as
your function app, which must be
unique in your subscription.
3. Select OK. The Application Insights resource is created in the same resource group and subscription as your
function app. After the resource is created, close the Application Insights window.
4. Back in your function app, select Application settings, and then scroll down to Application settings. If
you see a setting named APPINSIGHTS_INSTRUMENTATIONKEY , Application Insights integration is enabled for
your function app running in Azure.
Early versions of Functions used built-in monitoring, which is no longer recommended. When enabling
Application Insights integration for such a function app, you must also disable built-in logging.
3. To see the logs for a particular function invocation, select the Date column link for that invocation.
The logging output for that invocation appears in a new page.
You can see that both pages have a Run in Application Insights link to the Application Insights Analytics query
that retrieves the data.
The following query is displayed. You can see that the query results are limited to the last 30 days (
where timestamp > ago(30d) ). In addition, the results show no more than 20 rows ( take 20 ). In contrast, the
invocation details list for your function is for the last 30 days with no limit.
For more information, see Query telemetry data later in this article.
TAB DESCRIPTION
Failures Create charts and alerts based on function failures and server
exceptions. The Operation Name is the function name.
Failures in dependencies aren't shown unless you implement
custom telemetry for dependencies.
Servers View resource utilization and throughput per server. This data
can be useful for debugging scenarios where functions are
bogging down your underlying resources. Servers are referred
to as Cloud role instances.
Metrics Create charts and alerts that are based on metrics. Metrics
include the number of function invocations, execution time,
and success rates.
Live Metrics Stream View metrics data as it's created in near real-time.
requests
| where timestamp > ago(30m)
| summarize count() by cloud_RoleInstance, bin(timestamp, 1m)
| render timechart
The tables that are available are shown in the Schema tab on the left. You can find data generated by function
invocations in the following tables:
TABLE DESCRIPTION
The other tables are for availability tests, and client and browser telemetry. You can implement custom telemetry
to add data to them.
Within each table, some of the Functions-specific data is in a customDimensions field. For example, the following
query retrieves all traces that have log level Error .
traces
| where customDimensions.LogLevel == "Error"
The runtime provides the customDimensions.LogLevel and customDimensions.Category fields. You can provide
additional fields in logs that you write in your function code. See Structured logging later in this article.
LOGLEVEL CODE
Trace 0
LOGLEVEL CODE
Debug 1
Information 2
Warning 3
Error 4
Critical 5
None 6
{
"logging": {
"fileLoggingMode": "always",
"logLevel": {
"default": "Information",
"Host.Results": "Error",
"Function": "Error",
"Host.Aggregator": "Trace"
}
}
}
Version 1.x
{
"logger": {
"categoryFilter": {
"defaultLevel": "Information",
"categoryLevels": {
"Host.Results": "Error",
"Function": "Error",
"Host.Aggregator": "Trace"
}
}
}
}
{
"logging": {
"fileLoggingMode": "always",
"logLevel": {
"default": "Information",
"Host": "Error",
"Function": "Error",
"Host.Aggregator": "Information"
}
}
}
Version 1.x
{
"logger": {
"categoryFilter": {
"defaultLevel": "Information",
"categoryLevels": {
"Host": "Error",
"Function": "Error",
"Host.Aggregator": "Information"
}
}
}
}
To suppress all logs for a category, you can use log level None . No logs are written with that category and there's
no log level above it.
The following sections describe the main categories of logs that the runtime creates.
Category Host.Results
These logs show as "requests" in Application Insights. They indicate success or failure of a function.
All of these logs are written at Information level. If you filter at Warning or above, you won't see any of this data.
Category Host.Aggregator
These logs provide counts and averages of function invocations over a configurable period of time. The default
period is 30 seconds or 1,000 results, whichever comes first.
The logs are available in the customMetrics table in Application Insights. Examples are the number of runs,
success rate, and duration.
All of these logs are written at Information level. If you filter at Warning or above, you won't see any of this data.
Other categories
All logs for categories other than the ones already listed are available in the traces table in Application Insights.
All logs with categories that begin with Host are written by the Functions runtime. The "Function started" and
"Function completed" logs have category Host.Executor . For successful runs, these logs are Information level.
Exceptions are logged at Error level. The runtime also creates Warning level logs, for example: queue messages
sent to the poison queue.
Logs written by your function code have category Function and can be any log level.
{
"aggregator": {
"batchSize": 1000,
"flushTimeout": "00:00:30"
}
}
Configure sampling
Application Insights has a sampling feature that can protect you from producing too much telemetry data on
completed executions at times of peak load. When the rate of incoming executions exceeds a specified threshold,
Application Insights starts to randomly ignore some of the incoming executions. The default setting for maximum
number of executions per second is 20 (five in version 1.x). You can configure sampling in host.json. Here's an
example:
Version 2.x and later
{
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"maxTelemetryItemsPerSecond" : 20
}
}
}
}
Version 1.x
{
"applicationInsights": {
"sampling": {
"isEnabled": true,
"maxTelemetryItemsPerSecond" : 5
}
}
}
NOTE
Sampling is enabled by default. If you appear to be missing data, you might need to adjust the sampling settings to fit your
particular monitoring scenario.
With an ILogger object, you call Log<level> extension methods on ILogger to create logs. The following code
writes Information logs with category "Function.<YOUR_FUNCTION_NAME>.User."
Structured logging
The order of placeholders, not their names, determines which parameters are used in the log message. Suppose
you have the following code:
If you keep the same message string and reverse the order of the parameters, the resulting message text would
have the values in the wrong places.
Placeholders are handled this way so that you can do structured logging. Application Insights stores the parameter
name-value pairs and the message string. The result is that the message arguments become fields that you can
query on.
If your logger method call looks like the previous example, you can query the field customDimensions.prop__rowKey .
The prop__ prefix is added to ensure there are no collisions between fields the runtime adds and fields your
function code adds.
You can also query on the original message string by referencing the field
customDimensions.prop__{OriginalFormat} .
{
customDimensions: {
"prop__{OriginalFormat}":"C# Queue trigger function processed: {message}",
"Category":"Function",
"LogLevel":"Information",
"prop__message":"c9519cbf-b1e6-4b9b-bf24-cb7d10b1bb89"
}
}
logger.LogMetric("TestMetric", 1234);
This code is an alternative to calling TrackMetric by using the Application Insights API for .NET.
context.log.metric("TestMetric", 1234);
This code is an alternative to calling trackMetric by using the Node.js SDK for Application Insights.
namespace functionapp0915
{
public class HttpTrigger2
{
private readonly TelemetryClient telemetryClient;
/// Using dependency injection will guarantee that you use the same configuration for telemetry
collected automatically and manually.
public HttpTrigger2(TelemetryConfiguration telemetryConfiguration)
{
this.telemetryClient = new TelemetryClient(telemetryConfiguration);
}
[FunctionName("HttpTrigger2")]
public Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)]
HttpRequest req, ExecutionContext context, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
DateTime start = DateTime.UtcNow;
// Track an Event
var evt = new EventTelemetry("Function called");
evt.Context.User.Id = name;
this.telemetryClient.TrackEvent(evt);
// Track a Metric
var metric = new MetricTelemetry("Test Metric", DateTime.Now.Millisecond);
metric.Context.User.Id = name;
this.telemetryClient.TrackMetric(metric);
// Track a Dependency
var dependency = new DependencyTelemetry
{
Name = "GET api/planets/1/",
Target = "swapi.co",
Data = "https://swapi.co/api/planets/1/",
Timestamp = start,
Duration = DateTime.UtcNow - start,
Success = true
};
dependency.Context.User.Id = name;
this.telemetryClient.TrackDependency(dependency);
Version 1.x
using System;
using System.Net;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.Azure.WebJobs;
using System.Net.Http;
using System.Threading.Tasks;
using Microsoft.Azure.WebJobs.Extensions.Http;
using Microsoft.Extensions.Logging;
using System.Linq;
namespace functionapp0915
{
public static class HttpTrigger2
{
private static string key = TelemetryConfiguration.Active.InstrumentationKey =
System.Environment.GetEnvironmentVariable(
"APPINSIGHTS_INSTRUMENTATIONKEY", EnvironmentVariableTarget.Process);
[FunctionName("HttpTrigger2")]
public static async Task<HttpResponseMessage> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)]
HttpRequestMessage req, ExecutionContext context, ILogger log)
{
log.LogInformation("C# HTTP trigger function processed a request.");
DateTime start = DateTime.UtcNow;
// Track an Event
var evt = new EventTelemetry("Function called");
UpdateTelemetryContext(evt.Context, context, name);
telemetryClient.TrackEvent(evt);
// Track a Metric
var metric = new MetricTelemetry("Test Metric", DateTime.Now.Millisecond);
UpdateTelemetryContext(metric.Context, context, name);
telemetryClient.TrackMetric(metric);
// Track a Dependency
var dependency = new DependencyTelemetry
{
Name = "GET api/planets/1/",
Target = "swapi.co",
Data = "https://swapi.co/api/planets/1/",
Timestamp = start,
Duration = DateTime.UtcNow - start,
Success = true
};
UpdateTelemetryContext(dependency.Context, context, name);
telemetryClient.TrackDependency(dependency);
}
Don't call TrackRequest or StartOperation<RequestTelemetry> because you'll see duplicate requests for a function
invocation. The Functions runtime automatically tracks requests.
Don't set telemetryClient.Context.Operation.Id . This global setting causes incorrect correlation when many
functions are running simultaneously. Instead, create a new telemetry instance ( DependencyTelemetry ,
EventTelemetry ) and modify its Context property. Then pass in the telemetry instance to the corresponding
Track method on TelemetryClient ( TrackDependency() , TrackEvent() , TrackMetric() ). This method ensures that
the telemetry has the correct correlation details for the current function invocation.
context.done();
};
The tagOverrides parameter sets the operation_Id to the function's invocation ID. This setting enables you to
correlate all of the automatically generated and custom telemetry for a given function invocation.
Dependencies
Functions v2 automatically collects dependencies for HTTP requests, ServiceBus, EventHub, and SQL.
You can write custom code to show the dependencies. For examples, see the sample code in the C# custom
telemetry section. The sample code results in an application map in Application Insights that looks like the
following image:
Report issues
To report an issue with Application Insights integration in Functions, or to make a suggestion or request, create an
issue in GitHub.
Streaming Logs
While developing an application, you often want to see what's being written to the logs in near real-time when
running in Azure.
There are two ways to view a stream of log files being generated by your function executions.
Built-in log streaming: the App Service platform lets you view a stream of your application log files. This
is equivalent to the output seen when you debug your functions during local development and when you
use the Test tab in the portal. All log-based information is displayed. For more information, see Stream logs.
This streaming method supports only a single instance, and can't be used with an app running on Linux in a
Consumption plan.
Live Metrics Stream: when your function app is connected to Application Insights, you can view log data
and other metrics in near real-time in the Azure portal using Live Metrics Stream. Use this method when
monitoring functions running on multiple-instances or on Linux in a Consumption plan. This method uses
sampled data.
Log streams can be viewed both in the portal and in most local development environments.
Portal
You can view both types of log streams in the portal.
Built-in log streaming
To view streaming logs in the portal, select the Platform features tab in your function app. Then, under
Monitoring, choose Log streaming.
This connects your app to the log streaming service and application logs are displayed in the window. You can
toggle between Application logs and Web server logs.
Live Metrics Stream
To view the Live Metrics Stream for your app, select the Overview tab of your function app. When you have
Application Insights enables, you see an Application Insights link under Configured features. This link takes
you to the Application Insights page for your app.
In Application Insights, select Live Metrics Stream. Sampled log entries are displayed under Sample Telemetry.
Visual Studio Code
To turn on the streaming logs for your function app in Azure:
1. Select F1 to open the command palette, and then search for and run the command Azure Functions: Start
Streaming Logs.
2. Select your function app in Azure, and then select Yes to enable application logging for the function app.
3. Trigger your functions in Azure. Notice that log data is displayed in the Output window in Visual Studio
Code.
4. When you're done, remember to run the command Azure Functions: Stop Streaming Logs to disable
logging for the function app.
Core Tools
Built-in log streaming
Use the logstream option to start receiving streaming logs of a specific function app running in Azure, as in the
following example:
Azure CLI
You can enable streaming logs by using the Azure CLI. Use the following commands to sign in, choose your
subscription, and stream log files:
az login
az account list
az account set --subscription <subscriptionNameOrId>
az webapp log tail --resource-group <RESOURCE_GROUP_NAME> --name <FUNCTION_APP_NAME>
Azure PowerShell
You can enable streaming logs by using Azure PowerShell. For PowerShell, use the following commands to add
your Azure account, choose your subscription, and stream log files:
Add-AzAccount
Get-AzSubscription
Get-AzSubscription -SubscriptionName "<subscription name>" | Select-AzSubscription
Get-AzWebSiteLog -Name <FUNCTION_APP_NAME> -Tail
Next steps
For more information, see the following resources:
Application Insights
ASP.NET Core logging
Zero instrumentation application monitoring for
Kubernetes hosted applications
12/18/2019 • 4 minutes to read • Edit Online
IMPORTANT
This functionality is currently in public preview. This preview version is provided without a service level agreement, and it's not
recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For
more information, see Supplemental Terms of Use for Microsoft Azure Previews.
Azure Monitor now leverages service mesh tech on your Kubernetes cluster to provide out of the box application
monitoring for any Kubernetes hosted app. With default Application Insight features like Application Map to model
your dependencies, Live Metrics Stream for real-time monitoring, powerful visualizations with the default
dashboard, Metric Explorer, and Workbooks. This feature will help users spot performance bottlenecks and failure
hotspots across all of their Kubernetes workloads within a selected Kubernetes namespace. By capitalizing on your
existing service mesh investments with technologies like Istio, Azure Monitor enables auto-instrumented app
monitoring without any modification to your application's code.
NOTE
This is one of many ways to perform application monitoring on Kubernetes. You can also instrument any app hosted in
Kubernetes by using the Application Insights SDK without the need for a service mesh. To monitor Kubernetes without
instrumenting the application with an SDK you can use the below method.
Prerequisites
A Kubernetes cluster.
Console access to the cluster to run kubectl.
An Application Insight resource
Have a service mesh. If your cluster doesn't have Istio deployed, you can learn how to install and use Istio in
Azure Kubernetes Service.
Capabilities
By using zero instrumentation application monitoring for Kubernetes hosted apps, you will be able to use:
Application Map
Live Stream Metrics
Dashboards
Metrics Explorer
Distributed-tracing
End-to-end transaction monitoring
Installation steps
To enable the solution, we'll be performing the following steps:
Deploy the application (if not already deployed).
Ensure the application is part of the service mesh.
Observe collected telemetry.
Configure your app to work with a service mesh
Istio supports two ways of instrumenting a pod. In most cases, it's easiest to mark the Kubernetes namespace
containing your application with the istio -injection label:
NOTE
Since service mesh lifts data off the wire, we cannot intercept the encrypted traffic. For traffic that doesn't leave the cluster,
use an unencrypted protocol (for example, HTTP). For external traffic that must be encrypted, consider setting up SSL
termination at the ingress controller.
kubectl apply -f .
Verify deployment
Ensure Application Insights adapter has been deployed:
Troubleshooting
Below is the troubleshooting flow to use when telemetry doesn't appear in the Azure portal as expected.
1. Ensure the application is under load and is sending/receiving requests in plain HTTP. Since telemetry is
lifted off the wire, encrypted traffic is not supported. If there are no incoming or outgoing requests, there
will be no telemetry either.
2. Ensure the correct instrumentation key is provided in the
ISTIO_MIXER_PLUGIN_AI_INSTRUMENTATIONKEY environment variable in application-insights-istio -
mixer-adapter-deployment.yaml. The instrumentation key is found on the Overview tab of the Application
Insights resource in the Azure portal.
3. Ensure the correct Kubernetes namespace is provided in the
ISTIO_MIXER_PLUGIN_WATCHLIST_NAMESPACES environment variable in application-insights-istio -
mixer-adapter-deployment.yaml. Leave it blank to monitor all namespaces.
4. Ensure your application's pods have been sidecar-injected by Istio. Verify that Istio's sidecar exists on each
pod.
Verify that there is a container named istio -proxy running on the pod.
5. View the Application Insights adapter’s traces.
The count of received telemetry items is updated once a minute. If it doesn't grow minute over minute - no
telemetry is being sent to the adapter by Istio. Look for any errors in the log.
6. If it has been established that Application Insight for Kubernetes adapter is not being fed telemetry, check
Istio's Mixer logs to figure out why it's not sending data to the adapter:
Look for any errors, especially pertaining to communications with applicationinsightsadapter adapter.
FAQ
For the latest info for the progress on this project, visit the Application Insights adapter for Istio Mixer project's
GitHub.
Uninstall
To uninstall the product, for every YAML file found under src/kubernetes/ run:
Next steps
To learn more about how Azure Monitor and containers work together visit Azure Monitor for containers overview
Instrument web apps at runtime with Application
Insights Codeless Attach
10/23/2019 • 9 minutes to read • Edit Online
IMPORTANT
Status Monitor is no longer recommended for use. It has been replaced by the Azure Monitor Application Insights
Agent (formerly named Status Monitor v2). See our documentation for on-premises server deployments or Azure
virtual machine and virtual machine scale set deployments.
You can instrument a live web app with Azure Application Insights, without having to modify or redeploy
your code. You need a Microsoft Azure subscription.
Status Monitor is used to instrument a .NET application hosted in IIS either on-premises or in a VM.
If your app is deployed into Azure VM or Azure virtual machine scale set, follow these instructions.
If your app is deployed into Azure app services, follow these instructions.
If your app is deployed in an Azure VM, you can switch on Application Insights monitoring from the
Azure control panel.
(There are also separate articles about instrumenting Azure Cloud Services.)
You have a choice of two routes to apply Application Insights to your .NET web applications:
Build time: Add the Application Insights SDK to your web app code.
Run time: Instrument your web app on the server, as described below, without rebuilding and
redeploying the code.
NOTE
If you use build time instrumentation, run time instrumentation will not work even if it is turned on.
Dependency diagnostics On .NET 4.6+, but less detail Yes, full detail: result codes, SQL
command text, HTTP verb
4. Restart IIS.
Troubleshooting
Confirm a valid installation
These are some steps that you can perform to confirm that your installation was successful.
Confirm that the applicationInsights.config file is present in the target app directory and contains
your ikey.
If you suspect that data is missing you can run a simple query in Analytics to list all the cloud roles
currently sending telemetry.
If you need to confirm that Application Insights is successfully attached you can run Sysinternals
Handle in a command window to confirm that applicationinsights.dll has been loaded by IIS.
handle.exe /p w3wp.exe
<dependentAssembly>
<assemblyIdentity name="System.Diagnostics.DiagnosticSource" publicKeyToken="cc7b13ffcd2ddd51"/>
<bindingRedirect oldVersion="0.0.0.0-4.*.*.*" newVersion="4.0.2.1"/>
</dependentAssembly>
Detailed logs
By default Status Monitor will output diagnostic logs at:
C:\Program Files\Microsoft Application Insights\Status Monitor\diagnostics.log
System Requirements
OS support for Application Insights Status Monitor on Server:
Windows Server 2008
Windows Server 2008 R2
Windows Server 2012
Windows server 2012 R2
Windows Server 2016
with latest SP and .NET Framework 4.5 (Status Monitor is built on this version of the framework)
On the client side: Windows 7, 8, 8.1 and 10, again with .NET Framework 4.5
IIS support is: IIS 7, 7.5, 8, 8.5 (IIS is required)
Update-ApplicationInsightsVersion
Video
Next steps
View your telemetry:
Explore metrics to monitor performance and usage
Search events and logs to diagnose problems
Analytics for more advanced queries
Add more telemetry:
Create web tests to make sure your site stays live.
Add web client telemetry to see exceptions from web page code and to let you insert trace calls.
Add Application Insights SDK to your code so that you can insert trace and log calls
Deploy Azure Monitor Application Insights Agent for
on-premises servers
10/24/2019 • 2 minutes to read • Edit Online
IMPORTANT
This guidance is recommended for On-Premises and non-Azure cloud deployments of Application Insights Agent. Here's the
recommended approach for Azure virtual machine and virtual machine scale set deployments .
Application Insights Agent (formerly named Status Monitor V2) is a PowerShell module published to the
PowerShell Gallery. It replaces Status Monitor. Telemetry is sent to the Azure portal, where you can monitor your
app.
NOTE
The module only currently supports codeless instrumentation of .NET web apps hosted with IIS. Use an SDK to instrument
ASP.NET Core, Java, and Node.js applications.
PowerShell Gallery
Application Insights Agent is located here: https://www.powershellgallery.com/packages/Az.ApplicationMonitor.
C U R R E NT V E R S I O N V 1.0.1
Instructions
See the getting started instructions to get a start with concise code samples.
See the detailed instructions for a deep dive on how to get started.
Troubleshooting
Troubleshooting
Known issues
FAQ
Does Application Insights Agent support proxy installations?
Yes. There are multiple ways to download Application Insights Agent. If your computer has internet access,
you can onboard to the PowerShell Gallery by using -Proxy parameters. You can also manually download
the module and either install it on your computer or use it directly. Each of these options is described in the
detailed instructions.
Does Status Monitor v2 support ASP.NET Core applications?
No. For instructions to enable monitoring of ASP.NET Core applications, see Application Insights for
ASP.NET Core applications. There's no need to install StatusMonitor for an ASP.NET Core application. This
is true even if ASP.NET Core application is hosted in IIS.
How do I verify that the enablement succeeded?
The Get-ApplicationInsightsMonitoringStatus cmdlet can be used to verify that enablement
succeeded.
We recommend you use Live Metrics to quickly determine if your app is sending telemetry.
You can also use Log Analytics to list all the cloud roles currently sending telemetry:
Next steps
View your telemetry:
Explore metrics to monitor performance and usage.
Search events and logs to diagnose problems.
Use Analytics for more advanced queries.
Create dashboards.
Add more telemetry:
Create web tests to make sure your site stays live.
Add web client telemetry to see exceptions from web page code and to enable trace calls.
Add the Application Insights SDK to your code so you can insert trace and log calls.
Get started with Azure Monitor Application Insights
Agent for on-premises servers
10/24/2019 • 2 minutes to read • Edit Online
This article contains the quickstart commands expected to work for most environments. The instructions depend
on the PowerShell Gallery to distribute updates. These commands support the PowerShell -Proxy parameter.
For an explanation of these commands, customization instructions, and information about troubleshooting, see the
detailed instructions.
If you don't have an Azure subscription, create a free account before you begin.
Close PowerShell.
Install Application Insights Agent
Run PowerShell as Admin.
Enable monitoring
$pathToNupkg = "C:\Users\t\Desktop\Az.ApplicationMonitor.0.3.0-alpha.nupkg"
$pathToZip = ([io.path]::ChangeExtension($pathToNupkg, "zip"))
$pathToNupkg | rename-item -newname $pathToZip
$pathInstalledModule = "$Env:ProgramFiles\WindowsPowerShell\Modules\Az.ApplicationMonitor"
Expand-Archive -LiteralPath $pathToZip -DestinationPath $pathInstalledModule
Enable monitoring
Enable-ApplicationInsightsMonitoring -InstrumentationKey xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Next steps
View your telemetry:
Explore metrics to monitor performance and usage.
Search events and logs to diagnose problems.
Use Analytics for more advanced queries.
Create dashboards.
Add more telemetry:
Create web tests to make sure your site stays live.
Add web client telemetry to see exceptions from web page code and to enable trace calls.
Add the Application Insights SDK to your code so you can insert trace and log calls.
Do more with Application Insights Agent:
Review the detailed instructions for an explanation of the commands found here.
Use our guide to troubleshoot Application Insights Agent.
Application Insights Agent (formerly named Status
Monitor v2): Detailed instructions
11/19/2019 • 6 minutes to read • Edit Online
This article describes how to onboard to the PowerShell Gallery and download the ApplicationMonitor module.
Included are the most common parameters that you'll need to get started. We've also provided manual download
instructions in case you don't have internet access.
Example errors
Install-Module : The 'Install-Module' command was found in the module 'PowerShellGet', but the module could
not be
loaded. For more information, run 'Import-Module PowerShellGet'.
These instructions were written and tested on a computer running Windows 10 and the versions listed above.
NOTE
PowerShell Gallery is supported on Windows 10, Windows Server 2016, and PowerShell 6. For information about earlier
versions, see Installing PowerShellGet.
You can confirm this change and audit all PSRepositories by running the Get-PSRepository command.
4. Install the newest version of PowerShellGet.
Description: This module contains the tooling used to get other modules from PowerShell Gallery.
Version 1.0.0.1 ships with Windows 10 and Windows Server. Version 1.6.0 or higher is required. To
determine which version is installed, run the Get-Command -Module PowerShellGet command.
Reference: Installing PowerShellGet.
Command: Install-Module -Name PowerShellGet .
Optional parameters:
-Proxy . Specifies a proxy server for the request.
-Force . Bypasses the "already installed" warning and installs the latest version.
You'll receive this error if you're not using the newest version of PowerShellGet:
5. Restart PowerShell. You can't load the new version in the current session. New PowerShell sessions will
load the latest version of PowerShellGet.
$pathToNupkg = "C:\az.applicationmonitor.0.3.0-alpha.nupkg"
$pathToZip = ([io.path]::ChangeExtension($pathToNupkg, "zip"))
$pathToNupkg | rename-item -newname $pathToZip
$pathInstalledModule = "$Env:ProgramFiles\WindowsPowerShell\Modules\az.applicationmonitor"
Expand-Archive -LiteralPath $pathToZip -DestinationPath $pathInstalledModule
$pathToNupkg = "C:\az.applicationmonitor.0.2.1-alpha.nupkg"
$pathInstalledModule = "$Env:ProgramFiles\WindowsPowerShell\Modules\az.applicationmonitor"
Expand-Archive -LiteralPath $pathToNupkg -DestinationPath $pathInstalledModule
IMPORTANT
DLLs will install via relative paths. Store the contents of the package in your intended runtime directory and confirm that
access permissions allow read but not write.
1. Change the extension to ".zip" and extract the contents of the package into your intended installation directory.
2. Find the file path of Az.ApplicationMonitor.psd1.
3. Run PowerShell as Admin with an elevated execution policy.
4. Load the module by using the Import-Module Az.ApplicationMonitor.psd1 command.
The Application Insights SDK will need to send your app's telemetry to Microsoft. We recommend that you
configure proxy settings for your app in your web.config file. For more information, see Application Insights FAQ:
Proxy passthrough.
Enable monitoring
Use the Enable-ApplicationInsightsMonitoring command to enable monitoring.
See the API reference for a detailed description of how to use this cmdlet.
Next steps
View your telemetry:
Explore metrics to monitor performance and usage.
Search events and logs to diagnose problems.
Use Analytics for more advanced queries.
Create dashboards.
Add more telemetry:
Create web tests to make sure your site stays live.
Add web client telemetry to see exceptions from web page code and to enable trace calls.
Add the Application Insights SDK to your code so you can insert trace and log calls.
Do more with Application Insights Agent:
Use our guide to troubleshoot Application Insights Agent.
Troubleshooting Application Insights Agent (formerly
named Status Monitor v2)
10/24/2019 • 3 minutes to read • Edit Online
When you enable monitoring, you might experience issues that prevent data collection. This article lists all known
issues and provides troubleshooting examples. If you come across an issue that's not listed here, you can contact
us on GitHub.
Known issues
Conflicting DLLs in an app's bin directory
If any of these DLLs are present in the bin directory, monitoring might fail:
Microsoft.ApplicationInsights.dll
Microsoft.AspNet.TelemetryCorrelation.dll
System.Diagnostics.DiagnosticSource.dll
Some of these DLLs are included in the Visual Studio default app templates, even if your app doesn't use them.
You can use troubleshooting tools to see symptomatic behavior:
PerfView:
ThreadID="7,500"
ProcessorNumber="0"
msg="Found 'System.Diagnostics.DiagnosticSource, Version=4.0.2.1, Culture=neutral,
PublicKeyToken=cc7b13ffcd2ddd51' assembly, skipping attaching redfield binaries"
ExtVer="2.8.13.5972"
SubscriptionId=""
AppName=""
FormattedMessage="Found 'System.Diagnostics.DiagnosticSource, Version=4.0.2.1, Culture=neutral,
PublicKeyToken=cc7b13ffcd2ddd51' assembly, skipping attaching redfield binaries"
IISReset and app load (without telemetry). Investigate with Sysinternals (Handle.exe and ListDLLs.exe):
<modules>
<!-- Registered global managed http module handler. The
'Microsoft.AppInsights.IIS.ManagedHttpModuleHelper.dll' must be installed in the GAC before this
config is applied. -->
<add name="ManagedHttpModuleHelper"
type="Microsoft.AppInsights.IIS.ManagedHttpModuleHelper.ManagedHttpModuleHelper,
Microsoft.AppInsights.IIS.ManagedHttpModuleHelper, Version=1.0.0.0, Culture=neutral,
PublicKeyToken=31bf3856ad364e35" preCondition="managedHandler,runtimeVersionv4.0" />
</modules>
Troubleshooting
Troubleshooting PowerShell
Determine which modules are available
You can use the Get-Module -ListAvailable command to determine which modules are installed.
Import a module into the current session
If a module hasn't been loaded into a PowerShell session, you can manually load it by using the
Import-Module <path to psd1> command.
Collecting logs
1. In a command console with Admin privileges, run the iisreset /stop command to turn off IIS and all web
apps.
2. In PerfView, select Start Collection.
3. In a command console with Admin privileges, run the iisreset /start command to start IIS.
4. Try to browse to your app.
5. After your app is loaded, return to PerfView and select Stop Collection.
Next steps
Review the API reference to learn about parameters you might have missed.
If you come across an issue that's not listed here, you can contact us on GitHub.
Application Insights Agent API: Disable-
InstrumentationEngine
10/24/2019 • 2 minutes to read • Edit Online
This article describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
Disables the instrumentation engine by removing some registry keys. Restart IIS for the changes to take effect.
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions.
Examples
PS C:\> Disable-InstrumentationEngine
Parameters
-Verbose
Common parameter. Use this switch to output detailed logs.
Output
Example output from successfully disabling the instrumentation engine
Next steps
Do more with Application Insights Agent:
Use our guide to troubleshoot Application Insights Agent.
Application Insights Agent API: Disable-
ApplicationInsightsMonitoring
10/24/2019 • 2 minutes to read • Edit Online
This article describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
Disables monitoring on the target computer. This cmdlet will remove edits to the IIS applicationHost.config and
remove registry keys.
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions.
Examples
PS C:\> Disable-ApplicationInsightsMonitoring
Parameters
-Verbose
Common parameter. Use this switch to display detailed logs.
Output
Example output from successfully disabling monitoring
This article describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
Enables the instrumentation engine by setting some registry keys. Restart IIS for the changes to take effect.
The instrumentation engine can supplement data collected by the .NET SDKs. It collects events and messages that
describe the execution of a managed process. These events and messages include dependency result codes, HTTP
verbs, and SQL command text.
Enable the instrumentation engine if:
You've already enabled monitoring with the Enable cmdlet but didn't enable the instrumentation engine.
You've manually instrumented your app with the .NET SDKs and want to collect additional telemetry.
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions.
NOTE
This cmdlet requires that you review and accept our license and privacy statement.
The instrumentation engine adds additional overhead and is off by default.
Examples
PS C:\> Enable-InstrumentationEngine
Parameters
-AcceptLicense
Optional. Use this switch to accept the license and privacy statement in headless installations.
-Verbose
Common parameter. Use this switch to output detailed logs.
Output
Example output from successfully enabling the instrumentation engine
This article describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
Enables codeless attach monitoring of IIS apps on a target computer.
This cmdlet will modify the IIS applicationHost.config and set some registry keys. It will also create an
applicationinsights.ikey.config file, which defines the instrumentation key used by each app. IIS will load the
RedfieldModule on startup, which will inject the Application Insights SDK into applications as the applications
start. Restart IIS for your changes to take effect.
After you enable monitoring, we recommend that you use Live Metrics to quickly check if your app is sending us
telemetry.
NOTE
To get started, you need an instrumentation key. For more information, see Create a resource.
This cmdlet requires that you review and accept our license and privacy statement.
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions and an elevated execution policy. For more information,
see Run PowerShell as administrator with an elevated execution policy.
Examples
Example with a single instrumentation key
In this example, all apps on the current computer are assigned a single instrumentation key.
Parameters
-InstrumentationKey
Required. Use this parameter to supply a single instrumentation key for use by all apps on the target computer.
-InstrumentationKeyMap
Required. Use this parameter to supply multiple instrumentation keys and a mapping of the instrumentation keys
used by each app. You can create a single installation script for several computers by setting MachineFilter .
IMPORTANT
Apps will match against rules in the order that the rules are provided. So you should specify the most specific rules first and
the most generic rules last.
Schema
@(@{MachineFilter='.*';AppFilter='.*';InstrumentationSettings=@{InstrumentationKey='xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx'}})
Output
Example output from a successful enablement
Next steps
View your telemetry:
Explore metrics to monitor performance and usage.
Search events and logs to diagnose problems.
Use Analytics for more advanced queries.
Create dashboards.
Add more telemetry:
Create web tests to make sure your site stays live.
Add web client telemetry to see exceptions from web page code and to enable trace calls.
Add the Application Insights SDK to your code so you can insert trace and log calls.
Do more with Application Insights Agent:
Use our guide to troubleshoot Application Insights Agent.
Get the config to confirm that your settings were recorded correctly.
Get the status to inspect monitoring.
Application Insights Agent API: Get-
ApplicationInsightsMonitoringConfig
10/24/2019 • 2 minutes to read • Edit Online
This article describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
Gets the config file and prints the values to the console.
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions.
Examples
PS C:\> Get-ApplicationInsightsMonitoringConfig
Parameters
No parameters required.
Output
Example output from reading the config file
RedfieldConfiguration:
Filters:
0)InstrumentationKey: AppFilter: WebAppExclude MachineFilter: .*
1)InstrumentationKey: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx2 AppFilter: WebAppTwo MachineFilter: .*
2)InstrumentationKey: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxdefault AppFilter: .* MachineFilter: .*
Next steps
View your telemetry:
Explore metrics to monitor performance and usage.
Search events and logs to diagnose problems.
Use analytics for more advanced queries.
Create dashboards.
Add more telemetry:
Create web tests to make sure your site stays live.
Add web client telemetry to see exceptions from web page code and to enable trace calls.
Add the Application Insights SDK to your code so you can insert trace and log calls.
Do more with Application Insights Agent:
Use our guide to troubleshoot Application Insights Agent.
Make changes to the config by using the Set config cmdlet.
Application Insights Agent API: Get-
ApplicationInsightsMonitoringStatus
10/24/2019 • 3 minutes to read • Edit Online
This article describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
This cmdlet provides troubleshooting information about Status Monitor. Use this cmdlet to investigate the
monitoring status, version of the PowerShell Module, and to inspect the running process. This cmdlet will report
version information and information about key files required for monitoring.
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions.
Examples
Example: Application status
Run the command Get-ApplicationInsightsMonitoringStatus to display the monitoring status of web sites.
PS C:\Windows\system32> Get-ApplicationInsightsMonitoringStatus
Machine Identifier:
811D43F7EC807E389FEA2E732381288ACCD70AFFF9F569559AC3A75F023FA639
IIS Websites:
SiteName : DemoWebApp111
ApplicationPoolName : DemoWebApp111
SiteId : 2
SiteState : Started
ProcessId : not found
SiteName : DemoWebApp222
ApplicationPoolName : DemoWebApp222
SiteId : 3
SiteState : Started
ProcessId : 2024
Instrumented : true
InstrumentationKey : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx123
SiteName : DemoWebApp333
ApplicationPoolName : DemoWebApp333
SiteId : 4
SiteState : Started
ProcessId : 5184
AppAlreadyInstrumented : true
In this example;
Machine Identifier is an anonymous ID used to uniquely identify your server. If you create a support
request, we'll need this ID to find logs for your server.
Default Web Site is Stopped in IIS
DemoWebApp111 has been started in IIS, but hasn't received any requests. This report shows that there's
no running process (ProcessId: not found).
DemoWebApp222 is running and is being monitored (Instrumented: true). Based on the user configuration,
Instrumentation Key xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx123 was matched for this site.
DemoWebApp333 has been manually instrumented using the Application Insights SDK. Status Monitor
detected the SDK and won't monitor this site.
Example: PowerShell module information
Run the command Get-ApplicationInsightsMonitoringStatus -PowerShellModule to display information about the
current module:
PS C:\> Get-ApplicationInsightsMonitoringStatus -PowerShellModule
Runtime Paths:
ParentDirectory (Exists: True)
C:\Program Files\WindowsPowerShell\Modules\Az.ApplicationMonitor\content
iisreset.exe /status
Status for IIS Admin Service ( IISADMIN ) : Running
Status for Windows Process Activation Service ( WAS ) : Running
Status for Net.Msmq Listener Adapter ( NetMsmqActivator ) : Running
Status for Net.Pipe Listener Adapter ( NetPipeActivator ) : Running
Status for Net.Tcp Listener Adapter ( NetTcpActivator ) : Running
Status for World Wide Web Publishing Service ( W3SVC ) : Running
Parameters
(No parameters)
By default, this cmdlet will report the monitoring status of web applications. Use this option to review if your
application was successfully instrumented. You can also review which Instrumentation Key was matched to your
site.
-PowerShellModule
Optional. Use this switch to report the version numbers and paths of DLLs required for monitoring. Use this
option if you need to identify the version of any DLL, including the Application Insights SDK.
-InspectProcess
Optional. Use this switch to report whether IIS is running. It will also download external tools to determine if the
necessary DLLs are loaded into the IIS runtime.
If this process fails for any reason, you can run these commands manually:
iisreset.exe /status
handle64.exe -p w3wp | findstr /I "InstrumentationEngine AI. ApplicationInsights"
listdlls64.exe w3wp | findstr /I "InstrumentationEngine AI ApplicationInsights"
-Force
Optional. Used only with InspectProcess. Use this switch to skip the user prompt that appears before additional
tools are downloaded.
Next steps
Do more with Application Insights Agent:
Use our guide to troubleshoot Application Insights Agent.
Application Insights Agent API: Set-
ApplicationInsightsMonitoringConfig
10/24/2019 • 2 minutes to read • Edit Online
This document describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
Sets the config file without doing a full reinstallation. Restart IIS for your changes to take effect.
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions.
Examples
Example with a single instrumentation key
In this example, all apps on the current computer will be assigned a single instrumentation key.
Parameters
-InstrumentationKey
Required. Use this parameter to supply a single instrumentation key for use by all apps on the target computer.
-InstrumentationKeyMap
Required. Use this parameter to supply multiple instrumentation keys and a mapping of the instrumentation keys
used by each app. You can create a single installation script for several computers by setting MachineFilter .
IMPORTANT
Apps will match against rules in the order that the rules are provided. So you should specify the most specific rules first and
the most generic rules last.
Schema
@(@{MachineFilter='.*';AppFilter='.*';InstrumentationKey='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'})
Output
By default, no output.
Example verbose output from setting the config file via -InstrumentationKey
Example verbose output from setting the config file via -InstrumentationKeyMap
Next steps
View your telemetry:
Explore metrics to monitor performance and usage.
Search events and logs to diagnose problems.
Use Analytics for more advanced queries.
Create dashboards.
Add more telemetry:
Create web tests to make sure your site stays live.
Add web client telemetry to see exceptions from web page code and to enable trace calls.
Add the Application Insights SDK to your code so you can insert trace and log calls
Do more with Application Insights Agent:
Use our guide to troubleshoot Application Insights Agent.
Get the config to confirm that your settings were recorded correctly.
Get the status to inspect monitoring.
Application Insights Agent API: Start-
ApplicationInsightsMonitoringTrace
10/24/2019 • 3 minutes to read • Edit Online
This article describes a cmdlet that's a member of the Az.ApplicationMonitor PowerShell module.
Description
Collects ETW Events from the codeless attach runtime. This cmdlet is an alternative to running PerfView.
Collected events will be printed to the console in real-time and saved to an ETL file. The output ETL file can be
opened by PerfView for further investigation.
This cmdlet will run until it reaches the timeout duration (default 5 minutes) or is stopped manually ( Ctrl + C ).
IMPORTANT
This cmdlet requires a PowerShell session with Admin permissions.
Examples
How to collect events
Normally we would ask that you collect events to investigate why your application isn't being instrumented.
The codeless attach runtime will emit ETW events when IIS starts up and when your application starts up.
To collect these events:
1. In a cmd console with admin privileges, execute iisreset /stop To turn off IIS and all web apps.
2. Execute this cmdlet
3. In a cmd console with admin privileges, execute iisreset /start To start IIS.
4. Try to browse to your app.
5. After your app finishes loading, you can manually stop it ( Ctrl + C ) or wait for the timeout.
What events to collect
You have three options when collecting events:
1. Use the switch -CollectSdkEvents to collect events emitted from the Application Insights SDK.
2. Use the switch -CollectRedfieldEvents to collect events emitted by Status Monitor and the Redfield Runtime.
These logs are helpful when diagnosing IIS and application startup.
3. Use both switches to collect both event types.
4. By default, if no switch is specified both event types will be collected.
Parameters
-MaxDurationInMinutes
Optional. Use this parameter to set how long this script should collect events. Default is 5 minutes.
-LogDirectory
Optional. Use this switch to set the output directory of the ETL file. By default, this file will be created in the
PowerShell Modules directory. The full path will be displayed during script execution.
-CollectSdkEvents
Optional. Use this switch to collect Application Insights SDK events.
-CollectRedfieldEvents
Optional. Use this switch to collect events from Status Monitor and the Redfield runtime.
-Verbose
Common parameter. Use this switch to output detailed logs.
Output
Example of application startup logs
PS C:\Windows\system32> Start-ApplicationInsightsMonitoringTrace -ColectRedfieldEvents
Starting...
Log File: C:\Program
Files\WindowsPowerShell\Modules\Az.ApplicationMonitor\content\logs\20190627_144217_ApplicationInsights_ETW_Tra
ce.etl
Tracing enabled, waiting for events.
Tracing will timeout in 5 minutes. Press CTRL+C to cancel.
Next steps
Additional Troubleshooting:
Review additional troubleshooting steps here: https://docs.microsoft.com/azure/azure-monitor/app/status-
monitor-v2-troubleshoot
Review the API Reference to learn about parameters you might have missed.
If you need additional help, you can contact us on GitHub.
Do more with Application Insights Agent:
Use our guide to troubleshoot Application Insights Agent.
Get the config to confirm that your settings were recorded correctly.
Get the status to inspect monitoring.
Create an Application Insights resource
12/13/2019 • 3 minutes to read • Edit Online
Azure Application Insights displays data about your application in a Microsoft Azure resource. Creating a new
resource is therefore part of setting up Application Insights to monitor a new application. After you have
created your new resource, you can get its instrumentation key and use that to configure the Application
Insights SDK. The instrumentation key links your telemetry to the resource.
Name Unique value Name that identifies the app you are
monitoring.
SETTINGS VALUE DESCRIPTION
NOTE
While you can use the same resource name across different resource groups, it can be beneficial to use a globally unique
name. This can be useful if you plan to perform cross resource queries as it simplifies the required syntax.
Enter the appropriate values into the required fields, and then select Review + create.
When your app has been created, a new pane opens. This pane is where you see performance and usage data
about your monitored application.
Example
Results
Id :
/subscriptions/{subid}/resourceGroups/testgroup/providers/microsoft.insights/components/test1027
ResourceGroupName : testgroup
Name : test1027
Kind : web
Location : eastus
Type : microsoft.insights/components
AppId : 8323fb13-32aa-46af-b467-8355cf4f8f98
ApplicationType : web
Tags : {}
CreationDate : 10/27/2017 4:56:40 PM
FlowType :
HockeyAppId :
HockeyAppToken :
InstrumentationKey : 00000000-aaaa-bbbb-cccc-dddddddddddd
ProvisioningState : Succeeded
RequestSource : AzurePowerShell
SamplingPercentage :
TenantId : {subid}
For the full PowerShell documentation for this cmdlet, and to learn how to retrieve the instrumentation key
consult the Azure PowerShell documentation.
Azure CLI (preview)
To access the preview Application Insights Azure CLI commands you first need to run:
az extension add -n application-insights
If you don't run the az extension add command you will see an error message that states:
az : ERROR: az monitor: 'app-insights' is not in the 'az monitor' command group. See 'az monitor --help'.
Now you can run the following to create your Application Insights resource:
Example
az monitor app-insights component create --app demoApp --location westus2 --kind web -g demoRg --
application-type web
Results
az monitor app-insights component create --app demoApp --location eastus --kind web -g demoApp --
application-type web
{
"appId": "87ba512c-e8c9-48d7-b6eb-118d4aee2697",
"applicationId": "demoApp",
"applicationType": "web",
"creationDate": "2019-08-16T18:15:59.740014+00:00",
"etag": "\"0300edb9-0000-0100-0000-5d56f2e00000\"",
"flowType": "Bluefield",
"hockeyAppId": null,
"hockeyAppToken": null,
"id": "/subscriptions/{subid}/resourceGroups/demoApp/providers/microsoft.insights/components/demoApp",
"instrumentationKey": "00000000-aaaa-bbbb-cccc-dddddddddddd",
"kind": "web",
"location": "eastus",
"name": "demoApp",
"provisioningState": "Succeeded",
"requestSource": "rest",
"resourceGroup": "demoApp",
"samplingPercentage": null,
"tags": {},
"tenantId": {tenantID},
"type": "microsoft.insights/components"
}
For the full Azure CLI documentation for this command, and to learn how to retrieve the instrumentation key
consult the Azure CLI documentation.
Next steps
Diagnostic Search
Explore metrics
Write Analytics queries
Application Insights Overview dashboard
12/13/2019 • 2 minutes to read • Edit Online
Application Insights has always provided a summary overview pane to allow quick, at-a-glance assessment of
your application's health and performance. The new overview dashboard provides a faster more flexible
experience.
Better performance
Time range selection has been simplified to a simple one-click interface.
Overall performance has been greatly increased. You have one-click access to popular features like Search and
Analytics. Each default dynamically updating KPI tile provides insight into corresponding Application Insights
features. To learn more about failed requests select Failures under the Investigate header:
Application dashboard
Application dashboard leverages the existing dashboard technology within Azure to provide a fully
customizable single pane view of your application health and performance.
To access the default dashboard select Application Dashboard in the upper left corner.
If this is your first time accessing the dashboard, it will launch a default view:
You can keep the default view if you like it. Or you can also add, and delete from the dashboard to best fit the
needs of your team.
NOTE
All users with access to the Application Insights resource share the same Application dashboard experience. Changes
made by one user will modify the view for all users.
Next steps
Funnels
Retention
User Flows
Application Map: Triage Distributed Applications
12/16/2019 • 7 minutes to read • Edit Online
Application Map helps you spot performance bottlenecks or failure hotspots across all components of your
distributed application. Each node on the map represents an application component or its dependencies; and
has health KPI and alerts status. You can click through from any component to more detailed diagnostics, such
as Application Insights events. If your app uses Azure services, you can also click through to Azure diagnostics,
such as SQL Database Advisor recommendations.
What is a Component?
Components are independently deployable parts of your distributed/microservices application. Developers and
operations teams have code-level visibility or access to telemetry generated by these application components.
Components are different from "observed" external dependencies such as SQL, EventHub etc. which your
team/organization may not have access to (code or telemetry).
Components run on any number of server/role/container instances.
Components can be separate Application Insights instrumentation keys (even if subscriptions are different)
or different roles reporting to a single Application Insights instrumentation key. The preview map experience
shows the components regardless of how they are set up.
Investigate failures
Select investigate failures to launch the failures pane.
Investigate performance
To troubleshoot performance problems, select investigate performance.
Go to details
Select go to details to explore the end-to-end transaction experience, which can offer views down to the call
stack level.
View Logs (Analytics)
To query and investigate your applications data further, click view in Logs (Analytics).
Alerts
To view active alerts and the underlying rules that cause the alerts to be triggered, select alerts.
Set cloud role name
Application Map uses the cloud role name property to identify the components on the map. The Application
Insights SDK automatically adds the cloud role name property to the telemetry emitted by components. For
example, the SDK will add a web site name or service role name to the cloud role name property. However,
there are cases where you may want to override the default value. To override cloud role name and change
what gets displayed on the Application Map:
.NET/.NET Core
Write custom TelemetryInitializer as below.
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
namespace CustomInitializer.Telemetry
{
public class MyTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
if (string.IsNullOrEmpty(telemetry.Context.Cloud.RoleName))
{
//set custom role name here
telemetry.Context.Cloud.RoleName = "Custom RoleName";
telemetry.Context.Cloud.RoleInstance = "Custom RoleInstance";
}
}
}
}
<ApplicationInsights>
<TelemetryInitializers>
<!-- Fully qualified type name, assembly name: -->
<Add Type="CustomInitializer.Telemetry.MyTelemetryInitializer, CustomInitializer"/>
...
</TelemetryInitializers>
</ApplicationInsights>
An alternate method for ASP.NET Web apps is to instantiate the initializer in code, for example in
Global.aspx.cs:
using Microsoft.ApplicationInsights.Extensibility;
using CustomInitializer.Telemetry;
NOTE
Adding initializer using ApplicationInsights.config or using TelemetryConfiguration.Active is not valid for
ASP.NET Core applications.
using Microsoft.ApplicationInsights.Extensibility;
using CustomInitializer.Telemetry;
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}
Node.js
appInsights.defaultClient.addTelemetryProcessor(envelope => {
envelope.tags["ai.cloud.role"] = "your role name";
envelope.tags["ai.cloud.roleInstance"] = "your role instance"
});
Java
Starting with Application Insights Java SDK 2.5.0, you can specify the cloud role name by adding <RoleName> to
your ApplicationInsights.xml file, e.g.
If you use Spring Boot with the Application Insights Spring Boot starter, the only required change is to set your
custom name for the application in the application.properties file.
spring.application.name=<name-of-app>
The Spring Boot starter will automatically assign cloud role name to the value you enter for the
spring.application.name property.
Client/browser-side JavaScript
appInsights.queue.push(() => {
appInsights.addTelemetryInitializer((envelope) => {
envelope.tags["ai.cloud.role"] = "your role name";
envelope.tags["ai.cloud.roleInstance"] = "your role instance";
});
});
Understanding cloud role name within the context of the Application Map
As far as how to think about cloud role name, it can be helpful to look at an Application Map that has multiple
cloud role names present:
In the Application Map above each of the names in green boxes are cloud role name values for different aspects
of this particular distributed application. So for this app its roles consist of: Authentication , acmefrontend ,
Inventory Management , a Payment Processing Worker Role .
In the case of this app each of those cloud role names also represents a different unique Application Insights
resource with their own instrumentation keys. Since the owner of this application has access to each of those
four disparate Application Insights resources, Application Map is able to stitch together a map of the underlying
relationships.
For the official definitions:
[Description("Name of the role the application is a part of. Maps directly to the role name in azure.")]
[MaxStringLength("256")]
705: string CloudRole = "ai.cloud.role";
[Description("Name of the instance where the application is running. Computer name for on-premises,
instance name for Azure.")]
[MaxStringLength("256")]
715: string CloudRoleInstance = "ai.cloud.roleInstance";
Alternatively, cloud role instance can be helpful for scenarios where cloud role name tells you the problem
is somewhere in your web front-end, but you might be running your web front-end across multiple load-
balanced servers so being able to drill in a layer deeper via Kusto queries and knowing if the issue is impacting
all web front-end servers/instances or just one can be extremely important.
A scenario where you might want to override the value for cloud role instance could be if your app is running in
a containerized environment where just knowing the individual server might not be enough information to
locate a given issue.
For more information about how to override the cloud role name property with telemetry initializers, see Add
properties: ITelemetryInitializer.
Troubleshooting
If you're having trouble getting Application Map to work as expected, try these steps:
General
1. Make sure you're using an officially supported SDK. Unsupported/community SDKs might not support
correlation.
Refer to this article for a list of supported SDKs.
2. Upgrade all components to the latest SDK version.
3. If you're using Azure Functions with C#, upgrade to Functions V2.
4. Confirm cloud role name is correctly configured.
5. If you're missing a dependency, make sure it's in the list ofauto-collected dependencies. If not, you can
still track it manually with a track dependency call.
Too many nodes on the map
Application Map constructs an application node for each unique cloud role name present in your request
telemetry and a dependency node for each unique combination of type, target, and cloud role name in your
dependency telemetry. If there are more than 10,000 nodes in your telemetry, Application Map will not be able
to fetch all the nodes and links, so your map will be incomplete. If this happens, a warning message will appear
when viewing the map.
In addition, Application Map only supports up to 1000 separate ungrouped nodes rendered at once.
Application Map reduces visual complexity by grouping dependencies together that have the same type and
callers, but if your telemetry has too many unique cloud role names or too many dependency types, that
grouping will be insufficient, and the map will be unable to render.
To fix this, you'll need to change your instrumentation to properly set the cloud role name, dependency type,
and dependency target fields.
Dependency target should represent the logical name of a dependency. In many cases, it’s equivalent to
the server or resource name of the dependency. For example, in the case of HTTP dependencies it is set
to the hostname. It should not contain unique IDs or parameters that change from one request to
another.
Dependency type should represent the logical type of a dependency. For example, HTTP, SQL or Azure
Blob are typical dependency types. It should not contain unique IDs.
The purpose of cloud role name is described in the above section.
Portal feedback
To provide feedback, use the feedback option.
Next steps
To learn more about how correlation works in Application Insights consult the telemetry correlation article.
The end-to-end transaction diagnostic experience correlates server-side telemetry from across all your
Application Insights monitored components into a single view.
For advanced correlation scenarios in ASP.NET Core and ASP.NET consult the track custom operations
article.
Unified cross-component transaction diagnostics
10/24/2019 • 4 minutes to read • Edit Online
The unified diagnostics experience automatically correlates server-side telemetry from across all your
Application Insights monitored components into a single view. It doesn't matter if you have multiple resources
with separate instrumentation keys. Application Insights detects the underlying relationship and allows you to
easily diagnose the application component, dependency, or exception that caused a transaction slowdown or
failure.
What is a Component?
Components are independently deployable parts of your distributed/microservices application. Developers and
operations teams have code-level visibility or access to telemetry generated by these application components.
Components are different from "observed" external dependencies such as SQL, EventHub etc. which your
team/organization may not have access to (code or telemetry).
Components run on any number of server/role/container instances.
Components can be separate Application Insights instrumentation keys (even if subscriptions are different) or
different roles reporting to a single Application Insights instrumentation key. The new experience shows
details across all components, regardless of how they have been set up.
NOTE
Missing the related item links? All of the related telemetry are in the top and bottom sections of the left side.
NOTE
Calls to other components have two rows: one row represents the outbound call (dependency) from the caller component,
and the other row corresponds to the inbound request at the called component. The leading icon and distinct styling of
the duration bars help differentiate between them.
After you've deployed your web app/website, you can set up recurring tests to monitor availability and
responsiveness. Azure Application Insights sends web requests to your application at regular intervals from points
around the world. It can alert you if your application isn't responding, or if it responds too slowly.
You can set up availability tests for any HTTP or HTTPS endpoint that is accessible from the public internet. You
don't have to make any changes to the website you're testing. In fact, it doesn't even have to be a site you own. You
can test the availability of a REST API that your service depends on.
Types of availability tests:
There are three types of availability tests:
URL ping test: a simple test that you can create in the Azure portal.
Multi-step web test: A recording of a sequence of web requests, which can be played back to test more complex
scenarios. Multi-step web tests are created in Visual Studio Enterprise and uploaded to the portal for execution.
Custom Track Availability Tests: If you decide to create a custom application to run availability tests, the
TrackAvailability() method can be used to send the results to Application Insights.
You can create up to 100 availability tests per Application Insights resource.
URL The URL can be any web page you want to test, but it must be
visible from the public internet. The URL can include a query
string. So, for example, you can exercise your database a little.
If the URL resolves to a redirect, we follow it up to 10 redirects.
Parse dependent requests Test requests images, scripts, style files, and other files that are
part of the web page under test. The recorded response time
includes the time taken to get these files. The test fails if any of
these resources cannot be successfully downloaded within the
timeout for the whole test. If the option is not checked, the
test only requests the file at the URL you specified. Enabling
this option results in a stricter check. The test could fail for
cases, which may not be noticeable when manually browsing
the site.
Enable retries when the test fails, it is retried after a short interval. A failure is
reported only if three successive attempts fail. Subsequent
tests are then performed at the usual test frequency. Retry is
temporarily suspended until the next success. This rule is
applied independently at each test location. We recommend
this option. On average, about 80% of failures disappear on
retry.
Test frequency Sets how often the test is run from each test location. With a
default frequency of five minutes and five test locations, your
site is tested on average every minute.
Test locations Are the places from where our servers send web requests to
your URL. Our minimum number of recommended test
locations is five in order to insure that you can distinguish
problems in your website from network issues. You can select
up to 16 locations.
If your URL is not visible from the public internet, you can choose to selectively open up your firewall to
allow only the test transactions through. To learn more about the firewall exceptions for our availability test
agents, consult the IP address guide.
NOTE
We strongly recommend testing from multiple locations with a minimum of five locations. This is to prevent false alarms
that may result from transient issues with a specific location. In addition we have found that the optimal configuration is to
have the number of test locations be equal to the alert location threshold + 2.
Success criteria
SETTING EXPLANATION
Test timeout Decrease this value to be alerted about slow responses. The
test is counted as a failure if the responses from your site have
not been received within this period. If you selected Parse
dependent requests, then all the images, style files, scripts,
and other dependent resources must have been received
within this period.
HTTP response The returned status code that is counted as a success. 200 is
the code that indicates that a normal web page has been
returned.
Alerts
SETTING EXPLANATION
Select a particular test, location, or reduce the time period to see more results around the time period of interest.
Use Search Explorer to see results from all executions, or use Analytics queries to run custom reports on this data.
From an availability test result, you can see the transaction details across all components. Here you can:
Inspect the response received from your server.
Diagnose failure with correlated server-side telemetry collected while processing the failed availability test.
Log an issue or work item in Git or Azure Boards to track the problem. The bug will contain a link to this event.
Open the web test result in Visual Studio.
Learn more about the end to end transaction diagnostics experience here.
Click on the exception row to see the details of the server-side exception that caused the synthetic availability test to
fail. You can also get the debug snapshot for richer code level diagnostics.
In addition to the raw results, you can also view two key Availability metrics in Metrics Explorer:
1. Availability: Percentage of the tests that were successful, across all test executions.
2. Test Duration: Average test duration across all test executions.
Automation
Use PowerShell scripts to set up an availability test automatically.
Set up a webhook that is called when an alert is raised.
Troubleshooting
Dedicated troubleshooting article.
Next steps
Availability Alerts
Multi-step web tests
Multi-step web tests
12/8/2019 • 6 minutes to read • Edit Online
You can monitor a recorded sequence of URLs and interactions with a website via multi-step web tests. This
article will walk you through the process of creating a multi-step web test with Visual Studio Enterprise.
NOTE
Multi-step web tests depend on Visual Studio webtest files. It was announced that Visual Studio 2019 will be the last
version with webtest functionality. It is important to understand that while no new features will be added, webtest
functionality in Visual Studio 2019 is still currently supported and will continue to be supported during the support
lifecycle of the product. The Azure Monitor product team has addressed questions regarding the future of multi-step
availability tests here.
Pre-requisites
Visual Studio 2017 Enterprise or greater.
Visual Studio web performance and load testing tools.
To locate the testing tools pre-requisite. Launch the Visual Studio Installer > Individual components >
Debugging and testing > Web performance and load testing tools.
NOTE
Multi-step web tests have additional costs associated with them. To learn more consult the official pricing guide.
For guidance on creating Visual Studio web tests consult the official Visual Studio 2019 documentation.
Test frequency Sets how often the test is run from each test location. With a
default frequency of five minutes and five test locations, your
site is tested on average every minute.
Test locations Are the places from where our servers send web requests to
your URL. Our minimum number of recommended test
locations is five in order to insure that you can distinguish
problems in your website from network issues. You can select
up to 16 locations.
Success criteria
SETTING EXPLANATION
Test timeout Decrease this value to be alerted about slow responses. The
test is counted as a failure if the responses from your site
have not been received within this period. If you selected
Parse dependent requests, then all the images, style files,
scripts, and other dependent resources must have been
received within this period.
HTTP response The returned status code that is counted as a success. 200 is
the code that indicates that a normal web page has been
returned.
Alerts
SETTING EXPLANATION
Configuration
Plugging time and random numbers into your test
Suppose you're testing a tool that gets time-dependent data such as stocks from an external feed. When you
record your web test, you have to use specific times, but you set them as parameters of the test, StartTime and
EndTime.
When you run the test, you'd like EndTime always to be the present time, and StartTime should be 15 minutes
ago.
The Web Test Date Time Plugin provides the way to handle parameterize times.
1. Add a web test plug-in for each variable parameter value you want. In the web test toolbar, choose Add
Web Test Plugin.
In this example, we use two instances of the Date Time Plug-in. One instance is for "15 minutes ago" and
another for "now."
2. Open the properties of each plug-in. Give it a name and set it to use the current time. For one of them, set
Add Minutes = -15.
3. In the web test parameters, use {{plug-in name}} to reference a plug-in name.
Now, upload your test to the portal. It will use the dynamic values on every run of the test.
Dealing with sign-in
If your users sign in to your app, you have various options for simulating sign-in so that you can test pages
behind the sign-in. The approach you use depends on the type of security provided by the app.
In all cases, you should create an account in your application just for the purpose of testing. If possible, restrict
the permissions of this test account so that there's no possibility of the web tests affecting real users.
Simple username and password Record a web test in the usual way. Delete cookies first.
SAML authentication
Audience Uri The audience URI for the SAML token. This is the URI for the
Access Control Service (ACS) – including ACS namespace and
host name.
PROPERTY NAME DESCRIPTION
Certificate Password The password for the client certificate which will grant access
to the embedded private key.
Client Certificate The client certificate value with private key in Base64
encoded format.
Not After The timespan for which the token will be valid. The default is
5 minutes.
Not Before The timespan for which a token created in the past will be
valid (to address time skews). The default is (negative) 5
minutes.
Target Context Parameter Name The context parameter that will receive the generated
assertion.
Client secret If your app has a sign-in route that involves a client secret, use that route. Azure Active Directory
(AAD ) is an example of a service that provides a client secret sign-in. In AAD, the client secret is the App Key.
Here's a sample web test of an Azure web app using an app key:
Get token from AAD using client secret (AppKey). Extract bearer token from response. Call API using bearer
token in the authorization header. Make sure that the web test is an actual client - that is, it has its own app in
AAD - and use its clientId + app key. Your service under test also has its own app in AAD: the appID URI of this
app is reflected in the web test in the resource field.
Open Authentication
An example of open authentication is signing in with your Microsoft or Google account. Many apps that use
OAuth provide the client secret alternative, so your first tactic should be to investigate that possibility.
If your test must sign in using OAuth, the general approach is:
Use a tool such as Fiddler to examine the traffic between your web browser, the authentication site, and your
app. Perform two or more sign-ins using different machines or browsers, or at long intervals (to allow tokens to
expire). By comparing different sessions, identify the token passed back from the authenticating site, that is then
passed to your app server after sign-in. Record a web test using Visual Studio. Parameterize the tokens, setting
the parameter when the token is returned from the authenticator, and using it in the query to the site. (Visual
Studio attempts to parameterize the test, but does not correctly parameterize the tokens.)
Troubleshooting
Dedicated troubleshooting article.
Next steps
Availability Alerts
Url ping web tests
Create and run custom availability tests using Azure
Functions
12/4/2019 • 4 minutes to read • Edit Online
This article will cover how to create an Azure Function with TrackAvailability() that will run periodically according to
the configuration given in TimerTrigger function with your own business logic. The results of this test will be sent to
your Application Insights resource, where you will be able to query for and alert on the availability results data.
This allows you to create customized tests similar to what you can do via Availability Monitoring in the portal.
Customized tests will allow you to write more complex availability tests than is possible using the portal UI,
monitor an app inside of your Azure VNET, change the endpoint address, or create an availability test even if this
feature is not available in your region.
NOTE
This example is designed solely to show you the mechanics of how the TrackAvailability() API call works within an Azure
Function. Not how to write the underlying HTTP Test code/business logic that would be required to turn this into a fully
functional availability test. By default if you walk through this example you will be creating an availability test that will always
generate a failure.
Sample code
Copy the code below into the run.csx file (this will replace the pre-existing code). To do this, go into your Azure
Functions application and select your timer trigger function on the left.
NOTE
For the Endpoint Address you would use: EndpointAddress= https://dc.services.visualstudio.com/v2/track . Unless
your resource is located in a region like Azure Government or Azure China in which case consult this article on overriding the
default endpoints and select the appropriate Telemetry Channel endpoint for your region.
#load "runAvailabilityTest.csx"
using System;
using System.Diagnostics;
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
// The Application Insights Instrumentation Key can be changed by going to the overview page of your Function
App, selecting configuration, and changing the value of the APPINSIGHTS_INSTRUMENTATIONKEY Application setting.
// DO NOT replace the code below with your instrumentation key, the key's value is pulled from the environment
variable/application setting key/value pair.
private static readonly string instrumentationKey =
Environment.GetEnvironmentVariable("APPINSIGHTS_INSTRUMENTATIONKEY");
//[CONFIGURATION_REQUIRED]
// If your resource is in a region like Azure Government or Azure China, change the endpoint address
accordingly.
// Visit https://docs.microsoft.com/azure/azure-monitor/app/custom-endpoints#regions-that-require-endpoint-
modification for more details.
private const string EndpointAddress = "https://dc.services.visualstudio.com/v2/track";
if (myTimer.IsPastDue)
{
log.LogWarning($"[Warning]: Timer is running late! Last ran at: {myTimer.ScheduleStatus.Last}");
}
try
{
await RunAvailbiltyTestAsync(log);
availability.Success = true;
availability.Success = true;
}
catch (Exception ex)
{
availability.Message = ex.Message;
telemetryClient.TrackAvailability(availability);
// call flush to ensure telemetry is sent
telemetryClient.Flush();
}
}
On the right under view files, select Add. Call the new file function.proj with the following configuration.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.ApplicationInsights.AspNetCore" Version="2.8.2" /> <!-- Ensure
you’re using the latest version -->
</ItemGroup>
</Project>
On the right under view files, select Add. Call the new file runAvailabilityTest.csx with the following
configuration.
Check availability
To make sure everything is working, you can look at the graph in the Availability tab of your Application Insights
resource.
NOTE
If you implemented your own business logic in runAvailabilityTest.csx then you will see successful results like in the
screenshots below, if you did not then you will see failed results.
When you set up your test using Azure Functions you will notice, that unlike using Add test in the Availability tab,
the name of your test will not appear and you will not be able to interact with it. The results are visualized but you
get a summary view instead of the same detailed view you get when you create an availability test via the portal.
To see the end-to-end transaction details, select Successful or Failed under drill into, then select a sample. You can
also get to the end-to-end transaction details by selecting a data point on the graph.
If you ran everything as is (without adding business logic), then you will see that the test failed.
Query in Logs (Analytics)
You can use Logs(analytics) to view you availability results, dependencies, and more. To learn more about Logs,
visit Log query overview .
Next steps
Application Map
Transaction diagnostics
Availability alerts
12/5/2019 • 2 minutes to read • Edit Online
Azure Application Insights sends web requests to your application at regular intervals from points around the
world. It can alert you if your application isn't responding, or if it responds too slowly.
Enable alerts
Alerts are now automatically enabled by default, but in order to fully configure the alert you first have to initially
create your availability test.
NOTE
With the new unified alerts, the alert rule severity and notification preferences with action groups must be configured in the
alerts experience. Without the following steps, you will only receive in-portal notifications.
1. After saving the availability test, on the details tab click on the ellipsis by the test you just made. Click on
"edit alert".
2. Set the desired severity level, rule description and most importantly - the action group that has the
notification preferences you would like to use for this alert rule.
Alert on X out of Y locations reporting failures
The X out of Y locations alert rule is enabled by default in the new unified alerts experience, when you create a
new availability test. You can opt out by selecting the "classic" option or choosing to disable the alert rule.
NOTE
Configure the action groups to receive notifications when the alert triggers by following the steps above. Without this step,
you will only receive in-portal notifications when the rule triggers.
2. Configure alerts option from the menu will take you to the new experience where you can select specific
tests or locations to set up alert rule on. You can also configure the action groups for this alert rule here.
Alert on custom analytics queries
Using the new unified alerts, you can alert on custom log queries. With custom queries, you can alert on any
arbitrary condition that helps you get the most reliable signal of availability issues. This is also applicable, if you
are sending custom availability results using the TrackAvailability SDK.
TIP
The metrics on availability data include any custom availability results you may be submitting by calling our TrackAvailability
SDK. You can use the alerting on metrics support to alert on custom availability results.
Automate alerts
To automate this process with Azure Resource Manager templates, refer to the Create a metric alert with Resource
Manager template documentation.
Troubleshooting
Dedicated troubleshooting article.
Next steps
Multi-step web tests
Url ping web tests
Performance testing
12/13/2019 • 2 minutes to read • Edit Online
NOTE
The cloud-based load testing service has been deprecated. More information about the deprecation, the service availability,
and alternative services can be found here.
Application Insights allows you to generate load tests for your websites. Like availability tests, you can send either
basic requests or multi-step requests from Azure test agents around the world. Performance tests allow you to
simulate up to 20,000 simultaneous users for up to 60 minutes.
Duration (Minutes) 60
This article will help you to troubleshoot common issues that may occur when using availability monitoring.
SSL/TLS errors
SYMPTOM/ERROR MESSAGE POSSIBLE CAUSES
Could not create SSL/TLS Secure Channel SSL version. Only TLS 1.0, 1.1, and 1.2 are supported. SSLv3
is not supported.
TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Bad See StackExchange thread for more information.
Record MAC)
URL that is failing is to a CDN (Content Delivery Network) This may be caused by a misconfiguration on your CDN
Possible workaround
If the URLs that are experiencing the issue are always to dependent resources, it is recommended to disable
parse dependent requests for the web test.
A connection attempt failed because the connected party did Test agents in certain locations are being blocked by a firewall.
not properly respond after a period of time
The server committed a protocol This occurs when malformed headers a. Contact web site host provider / CDN
violation. Section=ResponseHeader are detected. Specifically, some headers provider to fix the faulty servers.
Detail=CR must be followed by LF might not be using CRLF to indicate the b. In case the failed requests are
end of line, which violates the HTTP resources (e.g. style files, images,
specification. Application Insights scripts), you may consider disabling the
enforces this HTTP specification and parsing of dependent requests. Keep in
fails responses with malformed headers. mind, if you do this you will lose the
ability to monitor the availability of
those files).
NOTE
The URL may not fail on browsers that have a relaxed validation of HTTP headers. See this blog post for a detailed
explanation of this issue: http://mehdi.me/a-tale-of-debugging-the-linkedin-api-net-and-http-protocol-violations/
NOTE
The URL may not fail on browsers that have a relaxed validation of HTTP headers. See this blog post for a detailed
explanation of this issue: http://mehdi.me/a-tale-of-debugging-the-linkedin-api-net-and-http-protocol-violations/
Use the new alert experience/near-realtime alerts if you need to notify users based on their roles. With action
groups, you can configure email notifications to users with any of the contributor/owner/reader roles (not
combined together as a single option).
Next steps
Multi-step web testing
URL ping tests
What is Distributed Tracing?
10/31/2019 • 2 minutes to read • Edit Online
The advent of modern cloud and microservices architectures has given rise to simple, independently deployable
services that can help reduce costs while increasing availability and throughput. But while these movements have
made individual services easier to understand as a whole, they’ve made overall systems more difficult to reason
about and debug.
In monolithic architectures, we’ve gotten used to debugging with call stacks. Call stacks are brilliant tools for
showing the flow of execution (Method A called Method B, which called Method C ), along with details and
parameters about each of those calls. This is great for monoliths or services running on a single process, but how
do we debug when the call is across a process boundary, not simply a reference on the local stack?
That’s where distributed tracing comes in.
Distributed tracing is the equivalent of call stacks for modern cloud and microservices architectures, with the
addition of a simplistic performance profiler thrown in. In Azure Monitor, we provide two experiences for
consuming distributed trace data. The first is our transaction diagnostics view, which is like a call stack with a time
dimension added in. The transaction diagnostics view provides visibility into one single transaction/request, and is
helpful for finding the root cause of reliability issues and performance bottlenecks on a per request basis.
Azure Monitor also offers an application map view which aggregates many transactions to show a topological
view of how the systems interact, and what the average performance and error rates are.
Next steps
OpenCensus Python usage guide
Application map
End-to-end performance monitoring
Telemetry correlation in Application Insights
12/9/2019 • 12 minutes to read • Edit Online
In the world of microservices, every logical operation requires work to be done in various components of the
service. You can monitor each of these components separately by using Application Insights. Application Insights
supports distributed telemetry correlation, which you use to detect which component is responsible for failures or
performance degradation.
This article explains the data model used by Application Insights to correlate telemetry sent by multiple
components. It covers context-propagation techniques and protocols. It also covers the implementation of
correlation tactics on different languages and platforms.
In a microservices environment, traces from components can go to different storage items. Every component can
have its own instrumentation key in Application Insights. To get telemetry for the logical operation, Application
Insights queries data from every storage item. When the number of storage items is large, you'll need a hint about
where to look next. The Application Insights data model defines two fields to solve this problem: request.source
and dependency.target . The first field identifies the component that initiated the dependency request. The second
field identifies which component returned the response of the dependency call.
Example
Let's look at an example. An application called Stock Prices shows the current market price of a stock by using an
external API called Stock. The Stock Prices application has a page called Stock page that the client web browser
opens by using GET /Home/Stock . The application queries the Stock API by using the HTTP call
GET /api/stock/value .
When the call GET /api/stock/value is made to an external service, you need to know the identity of that server so
you can set the dependency.target field appropriately. When the external service doesn't support monitoring,
target is set to the host name of the service (for example, stock-prices-api.com ). But if the service identifies itself
by returning a predefined HTTP header, target contains the service identity that allows Application Insights to
build a distributed trace by querying telemetry from that service.
Correlation headers
Application Insights is transitioning to W3C Trace-Context, which defines:
traceparent : Carries the globally unique operation ID and unique identifier of the call.
tracestate : Carries system -specific tracing context.
The latest version of the Application Insights SDK supports the Trace-Context protocol, but you might need to opt
in to it. (Backward compatibility with the previous correlation protocol supported by the Application Insights SDK
will be maintained.)
The correlation HTTP protocol, also called Request-Id, is being deprecated. This protocol defines two headers:
Request-Id : Carries the globally unique ID of the call.
Correlation-Context : Carries the name-value pairs collection of the distributed trace properties.
Application Insights also defines the extension for the correlation HTTP protocol. It uses Request-Context name-
value pairs to propagate the collection of properties used by the immediate caller or callee. The Application Insights
SDK uses this header to set the dependency.target and request.source fields.
Enable W3C distributed tracing support for classic ASP.NET apps
NOTE
Starting with Microsoft.ApplicationInsights.Web and Microsoft.ApplicationInsights.DependencyCollector , no
configuration is needed.
W3C Trace-Context support is implemented in a backward-compatible way. Correlation is expected to work with
applications that are instrumented with previous versions of the SDK (without W3C support).
If you want to keep using the legacy Request-Id protocol, you can disable Trace-Context by using this
configuration:
Activity.DefaultIdFormat = ActivityIdFormat.Hierarchical;
Activity.ForceDefaultIdFormat = true;
If you run an older version of the SDK, we recommend that you update it or apply the following configuration to
enable Trace-Context. This feature is available in the Microsoft.ApplicationInsights.Web and
Microsoft.ApplicationInsights.DependencyCollector packages, starting with version 2.8.0 -beta1. It's disabled by
default. To enable it, make these changes to ApplicationInsights.config :
Under RequestTrackingTelemetryModule , add the EnableW3CHeadersExtraction element and set its value to true .
Under DependencyTrackingTelemetryModule , add the EnableW3CHeadersInjection element and set its value to
true .
Add W3COperationCorrelationTelemetryInitializer under TelemetryInitializers . It will look similar to this
example:
<TelemetryInitializers>
<Add Type="Microsoft.ApplicationInsights.Extensibility.W3C.W3COperationCorrelationTelemetryInitializer,
Microsoft.ApplicationInsights"/>
...
</TelemetryInitializers>
NOTE
Starting with Microsoft.ApplicationInsights.AspNetCore version 2.8.0, no configuration is needed.
W3C Trace-Context support is implemented in a backward-compatible way. Correlation is expected to work with
applications that are instrumented with previous versions of the SDK (without W3C support).
If you want to keep using the legacy Request-Id protocol, you can disable Trace-Context by using this
configuration:
Activity.DefaultIdFormat = ActivityIdFormat.Hierarchical;
Activity.ForceDefaultIdFormat = true;
If you run an older version of the SDK, we recommend that you update it or apply the following configuration to
enable Trace-Context.
This feature is in version 2.5.0-beta1 and in
Microsoft.ApplicationInsights.AspNetCore
Microsoft.ApplicationInsights.DependencyCollector version 2.8.0 -beta1. It's disabled by default. To enable it, set
ApplicationInsightsServiceOptions.RequestCollectionOptions.EnableW3CDistributedTracing to true :
<Instrumentation>
<BuiltIn enabled="true">
<HTTP enabled="true" W3C="true" enableW3CBackCompat="true"/>
</BuiltIn>
</Instrumentation>
NOTE
Backward compatibility mode is enabled by default, and the enableW3CBackCompat parameter is optional. Use it only
when you want to turn backward compatibility off.
Ideally, you would turn this off when all your services have been updated to newer versions of SDKs that support the
W3C protocol. We highly recommend that you move to these newer SDKs as soon as possible.
IMPORTANT
Make sure the incoming and outgoing configurations are exactly the same.
Operation_Id TraceId
app = Flask(__name__)
middleware = FlaskMiddleware(
app,
exporter=AzureExporter(),
sampler=ProbabilitySampler(rate=1.0),
)
@app.route('/')
def hello():
return 'Hello World!'
if __name__ == '__main__':
app.run(host='localhost', port=8080, threaded=True)
This code runs a sample Flask application on your local machine, listening to port 8080 . To correlate trace context,
you send a request to the endpoint. In this example, you can use a curl command:
By looking at the Trace-Context header format, you can derive the following information:
version : 00
trace-id : 4bf92f3577b34da6a3ce929d0e0e4736
parent-id/span-id : 00f067aa0ba902b7
trace-flags : 01
If you look at the request entry that was sent to Azure Monitor, you can see fields populated with the trace header
information. You can find this data under Logs (Analytics) in the Azure Monitor Application Insights resource.
The id field is in the format <trace-id>.<span-id> , where the trace-id is taken from the trace header that was
passed in the request and the span-id is a generated 8-byte array for this span.
The operation_ParentId field is in the format <trace-id>.<parent-id> , where both the trace-id and the
parent-id are taken from the trace header that was passed in the request.
Log correlation
OpenCensus Python enables you to correlate logs by adding a trace ID, a span ID, and a sampling flag to log
records. You add these attributes by installing OpenCensus logging integration. The following attributes will be
added to Python LogRecord objects: traceId , spanId , and traceSampled . Note that this takes effect only for
loggers that are created after the integration.
Here's a sample application that demonstrates this:
import logging
config_integration.trace_integrations(['logging'])
logging.basicConfig(format='%(asctime)s traceId=%(traceId)s spanId=%(spanId)s %(message)s')
tracer = Tracer(sampler=AlwaysOnSampler())
logger = logging.getLogger(__name__)
logger.warning('Before the span')
with tracer.span(name='hello'):
logger.warning('In the span')
logger.warning('After the span')
Notice that there's a spanId present for the log message that's within the span. This is the same spanId that
belongs to the span named hello .
You can export the log data by using AzureLogHandler . For more information, see this article.
NOTE
Only calls made via Apache HttpClient are supported for the correlation feature. Both Spring RestTemplate and Feign can be
used with Apache HttpClient under the hood.
Currently, automatic context propagation across messaging technologies (like Kafka, RabbitMQ, and Azure Service
Bus) isn't supported. It is possible to code such scenarios manually by using the trackDependency and
trackRequest methods. In these methods, a dependency telemetry represents a message being enqueued by a
producer. The request represents a message being processed by a consumer. In this case, both operation_id and
operation_parentId should be propagated in the message's properties.
Role name
You might want to customize the way component names are displayed in the Application Map. To do so, you can
manually set the cloud_RoleName by taking one of the following actions:
With Application Insights Java SDK 2.5.0 and later, you can specify the cloud_RoleName by adding
<RoleName> to your ApplicationInsights.xml file:
If you use Spring Boot with the Application Insights Spring Boot Starter, you just need to set your custom
name for the application in the application.properties file:
spring.application.name=<name-of-app>
The Spring Boot Starter automatically assigns cloudRoleName to the value you enter for the
spring.application.name property.
Next steps
Write custom telemetry.
For advanced correlation scenarios in ASP.NET Core and ASP.NET, see Track custom operations.
Learn more about setting cloud_RoleName for other SDKs.
Onboard all components of your microservice on Application Insights. Check out the supported platforms.
See the data model for Application Insights types.
Learn how to extend and filter telemetry.
Review the Application Insights config reference.
Local forwarder (Preview)
12/16/2019 • 4 minutes to read • Edit Online
Local forwarder is an agent that collects Application Insights or OpenCensus telemetry from a variety of SDKs and
routes it to Application Insights. It's capable of running under Windows and Linux. You may also be able to run it
under macOS, but that is not officially supported at this time.
NOTE
The local forwarder service requires a minimum of .NET Framework 4.7. If you do not have .NET Framework 4.7 the service
will install, but it won't start. To access the lastest version of the .NET Framework visit the .NET Framework download
page.
1. Download the LF.WindowsServiceHost.zip file from the local forwarder release page on GitHub.
2. In this example for ease of demonstration, we will just extract the .zip file to the path
C:\LF-WindowsServiceHost .
To register the service and configure it to start at system boot run the following from the command line as
Administrator:
To examine your new service via the Services GUI type services.msc
3. Right-click the new local forwarder and select Start. Your service will now enter a running state.
4. By default the service is created without any recovery actions. You can right-click and select Properties >
Recovery to configure automatic responses to a service failure.
Or if you prefer to set automatic recovery options programmatically for when failures occur, you can use:
E:\uncdrop\ConsoleHost\publish>dotnet Microsoft.LocalForwarder.ConsoleHost.dll
a self-contained .NET Core set of binaries for x86 and x64 platforms. These don't require .NET Core runtime to
run. /ConsoleHost/win-x86/publish/Microsoft.LocalForwarder.ConsoleHost.exe, /ConsoleHost/win-
x64/publish/Microsoft.LocalForwarder.ConsoleHost.exe.
E:\uncdrop\ConsoleHost\win-x86\publish>Microsoft.LocalForwarder.ConsoleHost.exe
E:\uncdrop\ConsoleHost\win-x64\publish>Microsoft.LocalForwarder.ConsoleHost.exe
Linux
As with Windows, the release comes with the following executable versions of the console host:
a framework-dependent .NET Core binary /ConsoleHost/publish/Microsoft.LocalForwarder.ConsoleHost.dll.
Running this binary requires a .NET Core runtime to be installed; refer to this download page for details.
dotnet Microsoft.LocalForwarder.ConsoleHost.dll
a self-contained .NET Core set of binaries for linux-64. This one doesn't require .NET Core runtime to run.
/ConsoleHost/linux-x64/publish/Microsoft.LocalForwarder.ConsoleHost.
Many Linux users will want to run local forwarder as a daemon. Linux systems come with a variety of solutions for
service management, like Upstart, sysv, or systemd. Whatever your particular version is, you can use it to run local
forwarder in a way that is most appropriate for your scenario.
As an example, let's create a daemon service using systemd. We'll use the framework-dependent version, but the
same can be done for a self-contained one as well.
create the following service file named localforwarder.service and place it into /lib/systemd/system. This sample
assumes your user name is SAMPLE_USER and you've copied local forwarder framework-dependent binaries
(from /ConsoleHost/publish) to /home/SAMPLE_USER/LOCALFORWARDER_DIR.
# localforwarder.service
# Place this file into /lib/systemd/system/
# Use 'systemctl enable localforwarder' to start the service automatically on each boot
# Use 'systemctl start localforwarder' to start the service immediately
[Unit]
Description=Local Forwarder service
After=network.target
StartLimitIntervalSec=0
[Service]
Type=simple
Restart=always
RestartSec=1
User=SAMPLE_USER
WorkingDirectory=/home/SAMPLE_USER/LOCALFORWARDER_DIR
ExecStart=/usr/bin/env dotnet /home/SAMPLE_USER/LOCALFORWARDER_DIR/Microsoft.LocalForwarder.ConsoleHost.dll
noninteractive
[Install]
WantedBy=multi-user.target
Run the following command to instruct systemd to start local forwarder on every boot
Run the following command to instruct systemd to start local forwarder immediately
using Library;
...
Host host = new Host();
host.Run(config, TimeSpan.FromSeconds(5));
...
host.Stop();
NOTE
Configuration may change from release to release, so pay attention to which version you're using.
Next steps
Open Census
Collect distributed traces from Go (Preview)
12/16/2019 • 4 minutes to read • Edit Online
Application Insights now supports distributed tracing of Go applications through integration with OpenCensus and
our new local forwarder. This article will walk you step-by-step through the process of setting up OpenCensus for
Go and getting your trace data to Application Insights.
Prerequisites
You need an Azure Subscription.
Go should be installed, this article uses the version 1.11 Go Download.
Follow the instructions to install the local forwarder as a Windows service.
If you don't have an Azure subscription, create a free account before you begin.
A configuration box appears; use the following table to fill out the input fields.
Name Globally Unique Value Name that identifies the app you are
monitoring
2. Click Create.
2. Edit your LocalForwarder.config file and add your instrumentation key. If you followed the instructions in
the pre-requisite the file is located at C:\LF-WindowsServiceHost
<OpenCensusToApplicationInsights>
<!--
Instrumentation key to track telemetry to.
-->
<InstrumentationKey>{enter-instrumentation-key}</InstrumentationKey>
</OpenCensusToApplicationInsights>
OpenCensus Go packages
1. Install the Open Census packages for Go from the command line:
go get -u go.opencensus.io
go get -u contrib.go.opencensus.io/exporter/ocagent
2. Add the following code to a .go file and then build and run. (This example is derived from the official
OpenCensus guidance with added code that facilitates the integration with the local forwarder)
import (
"bytes"
"fmt"
"log"
"net/http"
"net/http"
os "os"
ocagent "contrib.go.opencensus.io/exporter/ocagent"
"go.opencensus.io/plugin/ochttp"
"go.opencensus.io/plugin/ochttp/propagation/tracecontext"
"go.opencensus.io/trace"
func main() {
// Register stats and trace exporters to export the collected data.
serviceName := os.Getenv("SERVICE_NAME")
if len(serviceName) == 0 {
serviceName = "go-app"
}
fmt.Printf(serviceName)
agentEndpoint := os.Getenv("OCAGENT_TRACE_EXPORTER_ENDPOINT")
if len(agentEndpoint) == 0 {
agentEndpoint = fmt.Sprintf("%s:%d", ocagent.DefaultAgentHost, ocagent.DefaultAgentPort)
}
fmt.Printf(agentEndpoint)
exporter, err := ocagent.NewExporter(ocagent.WithInsecure(), ocagent.WithServiceName(serviceName),
ocagent.WithAddress(agentEndpoint))
if err != nil {
log.Printf("Failed to create the agent exporter: %v", err)
}
trace.RegisterExporter(exporter)
trace.ApplyConfig(trace.Config{DefaultSampler: trace.AlwaysSample()})
if err != nil {
log.Println(err)
} else {
// TODO: handle response
resp.Body.Close()
}
}
})
3. Once the simple go app is running navigate to http://localhost:50030 . Each refresh of the browser will
generate the text "hello world" accompanied by corresponding span data that is picked up by the local
forwarder.
4. To confirm that the local forwarder is picking up the traces check the LocalForwarder.config file. If you
followed the steps in the prerequisite, it will be located in C:\LF-WindowsServiceHost .
In the image below of the log file, you can see that prior to running the second script where we added an
exporter OpenCensus input BatchesReceived was 0. Once we started running the updated script
BatchesReceived incremented equal to the number of values we entered:
2. If you run the second Go app again and start refreshing the browser for http://localhost:50030 , you will
see live trace data as it arrives in Application Insights from the local forwarder service.
3. Navigate back to the Overview page and select Application Map for a visual layout of the dependency
relationships and call timing between your application components.
Since we were only tracing one method call, our application map isn't as interesting. But application map can
scale to visualize far more distributed applications:
4. Select Investigate Performance to perform detailed performance analysis and determine the root cause of
slow performance.
5. Selecting Samples and then clicking on any of the samples that appear in the right-hand pane will launch
the end-to-end transaction details experience. While our sample app will just show us a single event, a more
complex application would allow you to explore the end-to-end transaction down to level of an individual
event's call stack.
Next steps
Application map
End-to-end performance monitoring
Live Metrics Stream: Monitor & Diagnose with 1-
second latency
12/18/2019 • 6 minutes to read • Edit Online
Probe the beating heart of your live, in-production web application by using Live Metrics Stream from
Application Insights. Select and filter metrics and performance counters to watch in real time, without any
disturbance to your service. Inspect stack traces from sample failed requests and exceptions. Together with
Profiler, Snapshot debugger. Live Metrics Stream provides a powerful and non-invasive diagnostic tool for your
live web site.
With Live Metrics Stream, you can:
Validate a fix while it is released, by watching performance and failure counts.
Watch the effect of test loads, and diagnose issues live.
Focus on particular test sessions or filter out known issues, by selecting and filtering the metrics you want to
watch.
Get exception traces as they happen.
Experiment with filters to find the most relevant KPIs.
Monitor any Windows performance counter live.
Easily identify a server that is having issues, and filter all the KPI/live feed to just that server.
Live Metrics are currently supported for ASP.NET, ASP.NET Core, Azure Functions, Java, and Node.js apps.
Get started
1. If you haven't yet install Application Insights in your web app, do that now.
2. In addition to the standard Application Insights packages
Microsoft.ApplicationInsights.PerfCounterCollector is required to enable Live Metrics stream.
3. Update to the latest version of the Application Insights package. In Visual Studio, right-click your
project and choose Manage Nuget packages. Open the Updates tab, and select all the
Microsoft.ApplicationInsights.* packages.
Redeploy your app.
4. In the Azure portal, open the Application Insights resource for your app, and then open Live Stream.
5. Secure the control channel if you might use sensitive data such as customer names in your filters.
No data? Check your server firewall
Check the outgoing ports for Live Metrics Stream are open in the firewall of your servers.
How does Live Metrics Stream differ from Metrics Explorer and
Analytics?
LIVE STREAM METRICS EXPLORER AND ANALYTICS
No retention Data persists while it's on the chart, Data retained for 90 days
and is then discarded
On demand Data is streamed while you open Live Data is sent whenever the SDK is
Metrics installed and enabled
Sampling All selected metrics and counters are Events may be sampled
transmitted. Failures and stack traces
are sampled. TelemetryProcessors are
not applied.
Control channel Filter control signals are sent to the Communication is one way, to the
SDK. We recommend you secure this portal
channel.
In addition to Application Insights telemetry, you can also monitor any Windows performance counter by
selecting that from the stream options, and providing the name of the performance counter.
Live metrics are aggregated at two points: locally on each server, and then across all servers. You can change the
default at either by selecting other options in the respective drop-downs.
Note: Currently, for Exception message-based criteria, use the outermost exception message. In the preceding
example, to filter out the benign exception with inner exception message (follows the "<--" delimiter) "The client
disconnected." use a message not-contains "Error reading request content" criteria.
See the details of an item in the live feed by clicking it. You can pause the feed either by clicking Pause or simply
scrolling down, or clicking an item. Live feed will resume after you scroll back to the top, or by clicking the
counter of items collected while it was paused.
Filter by server instance
If you want to monitor a particular server role instance, you can filter by server.
Secure the control channel
The custom filters criteria you specify are sent back to the Live Metrics component in the Application Insights
SDK. The filters could potentially contain sensitive information such as customerIDs. You can make the channel
secure with a secret API key in addition to the instrumentation key.
Create an API Key
<Add
Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule,
Microsoft.AI.PerfCounterCollector">
<AuthenticationApiKey>YOUR-API-KEY-HERE</AuthenticationApiKey>
</Add>
using Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse;
using Microsoft.ApplicationInsights.Extensibility;
configuration.TelemetryProcessorChainBuilder
.Use((next) =>
{
processor = new QuickPulseTelemetryProcessor(next);
return processor;
})
.Build();
AuthenticationApiKey = "YOUR-API-KEY"
};
QuickPulse.Initialize(configuration);
QuickPulse.RegisterTelemetryProcessor(processor);
foreach (var telemetryProcessor in configuration.TelemetryProcessors)
{
if (telemetryProcessor is ITelemetryModule telemetryModule)
{
telemetryModule.Initialize(configuration);
}
}
using Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse;
However, if you recognize and trust all the connected servers, you can try the custom filters without the
authenticated channel. This option is available for six months. This override is required once every new session,
or when a new server comes online.
NOTE
We strongly recommend that you set up the authenticated channel before entering potentially sensitive information like
CustomerID in the filter criteria.
Basic metrics include request, dependency, and exception rate. Performance metrics (performance counters)
include memory and CPU. Sample telemetry shows a stream of detailed information for failed requests and
dependencies, exceptions, events, and traces.
* PerfCounters support varies slightly across versions of .NET Core that do not target the .NET Framework:
PerfCounters metrics are supported when running in Azure App Service for Windows. (AspNetCore SDK
Version 2.4.1 or higher)
PerfCounters are supported when app is running in ANY Windows machines (VM or Cloud Service or On-
prem etc.) (AspNetCore SDK Version 2.7.1 or higher), but for apps targeting .NET Core 2.0 or higher.
PerfCounters are supported when app is running ANYWHERE (Linux, Windows, app service for Linux,
containers, etc.) in the latest beta (i.e AspNetCore SDK Version 2.8.0-beta1 or higher), but for apps targeting
.NET Core 2.0 or higher.
By default Live Metrics is disabled in the Node.js SDK. To enable Live Metrics, add setSendLiveMetrics(true) to
your configuration methods as you initialize the SDK.
Troubleshooting
No data? If your application is in a protected network: Live Metrics Stream uses different IP addresses than other
Application Insights telemetry. Make sure those IP addresses are open in your firewall.
Next steps
Monitoring usage with Application Insights
Using Diagnostic Search
Profiler
Snapshot debugger
Log-based and pre-aggregated metrics in
Application Insights
12/13/2019 • 5 minutes to read • Edit Online
This article explains the difference between “traditional” Application Insights metrics that are based on logs, and
pre-aggregated metrics that are currently in public preview. Both types of metrics are available to the users of
Application Insights, and each brings a unique value in monitoring application health, diagnostics and analytics.
The developers who are instrumenting applications can decide which type of metric is best suited to a particular
scenario, depending on the size of the application, expected volume of telemetry, and business requirements for
metrics precision and alerting.
Log-based Metrics
Until recently, the application monitoring telemetry data model in Application Insights was solely based on a small
number of predefined types of events, such as requests, exceptions, dependency calls, page views, etc. Developers
can use the SDK to either emit these events manually (by writing code that explicitly invokes the SDK) or they can
rely on the automatic collection of events from auto-instrumentation. In either case, the Application Insights
backend stores all collected events as logs, and the Application Insights blades in the Azure portal act as an
analytical and diagnostic tool for visualizing event-based data from logs.
Using logs to retain a complete set of events can bring great analytical and diagnostic value. For example, you can
get an exact count of requests to a particular URL with the number of distinct users who made these calls. Or you
can get detailed diagnostic traces, including exceptions and dependency calls for any user session. Having this type
of information can significantly improve visibility into the application health and usage, allowing to cut down the
time necessary to diagnose issues with an app.
At the same time, collecting a complete set of events may be impractical (or even impossible) for applications that
generate a lot of telemetry. For situations when the volume of events is too high, Application Insights implements
several telemetry volume reduction techniques, such as sampling and filtering that reduce the number of collected
and stored events. Unfortunately, lowering the number of stored events also lowers the accuracy of the metrics
that, behind the scenes, must perform query-time aggregations of the events stored in logs.
NOTE
In Application Insights, the metrics that are based on the query-time aggregation of events and measurements stored in
logs are called log-based metrics. These metrics typically have many dimensions from the event properties, which makes
them superior for analytics, but the accuracy of these metrics is negatively affected by sampling and filtering.
Pre-aggregated metrics
In addition to log-based metrics, in Fall of 2018, the Application Insights team shipped a public preview of metrics
that are stored in a specialized repository that is optimized for time series. The new metrics are no longer kept as
individual events with lots of properties. Instead, they are stored as pre-aggregated time series, and only with key
dimensions. This makes the new metrics superior at query time: retrieving data happens much faster and requires
less compute power. This consequently enables new scenarios such as near real-time alerting on dimensions of
metrics, more responsive dashboards, and more.
IMPORTANT
Both, log-based and pre-aggregated metrics coexist in Application Insights. To differentiate the two, in the Application
Insights UX the pre-aggregated metrics are now called “Standard metrics (preview)”, while the traditional metrics from the
events were renamed to “Log-based metrics”.
The newer SDKs (Application Insights 2.7 SDK or later for .NET) pre-aggregate metrics during collection before
telemetry volume reduction techniques kick in. This means that the accuracy of the new metrics isn’t affected by
sampling and filtering when using the latest Application Insights SDKs.
For the SDKs that don’t implement pre-aggregation (that is, older versions of Application Insights SDKs or for
browser instrumentation) the Application Insights backend still populates the new metrics by aggregating the
events received by the Application Insights event collection endpoint. This means that while you don’t benefit from
the reduced volume of data transmitted over the wire, you can still use the pre-aggregated metrics and experience
better performance and support of the near real-time dimensional alerting with SDKs that don’t pre-aggregate
metrics during collection.
It is worth mentioning that the collection endpoint pre-aggregates events before ingestion sampling, which means
that ingestion sampling will never impact the accuracy of pre-aggregated metrics, regardless of the SDK version
you use with your application.
Next steps
Near real-time alerting
GetMetric and TrackValue
Application Insights log-based metrics
11/8/2019 • 10 minutes to read • Edit Online
Application Insights log-based metrics let you analyze the health of your monitored apps, create powerful
dashboards, and configure alerts. There are two kinds of metrics:
Log-based metrics behind the scene are translated into Kusto queries from stored events.
Standard metrics are stored as pre-aggregated time series.
Since standard metrics are pre-aggregated during collection, they have better performance at query time. This
makes them a better choice for dashboarding and in real-time alerting. The log -based metrics have more
dimensions, which makes them the superior option for data analysis and ad-hoc diagnostics. Use the namespace
selector to switch between log-based and standard metrics in metrics explorer.
NOTE
If you're new to the Kusto query language, you start by copying and pasting Kusto statements into the Log Analytics query
pane without making any modifications. Click Run to see basic chart. As you begin to understand the syntax of query
language, you can start making small modifications and see the impact of your change. Exploring your own data is a great
way to start realizing the full power of Log Analytics and Azure Monitor.
Availability metrics
Metrics in the Availability category enable you to see the health of your web application as observed from points
around the world. Configure the availability tests to start using any metrics from this category.
Availability (availabilityResults/availabilityPercentage )
The Availability metric shows the percentage of the web test runs that didn't detect any issues. The lowest possible
value is 0, which indicates that all of the web test runs have failed. The value of 100 means that all of the web test
runs passed the validation criteria.
UNIT OF MEASURE SUPPORTED AGGREGATIONS SUPPORTED DIMENSIONS
availabilityResults
| summarize sum(todouble(success == 1) * 100) / count() by bin(timestamp, 5m), location
| render timechart
Milliseconds Average, Min, Max Run location, Test name, Test result
availabilityResults
| where notempty(duration)
| extend availabilityResult_duration = iif(itemType == 'availabilityResult', duration, todouble(''))
| summarize sum(availabilityResult_duration)/sum(itemCount) by bin(timestamp, 5m), location
| render timechart
availabilityResults
| summarize sum(itemCount) by bin(timestamp, 5m)
| render timechart
Browser metrics
Browser metrics are collected by the Application Insights JavaScript SDK from real end-user browsers. They
provide great insights into your users' experience with your web app. Browser metrics are typically not sampled,
which means that they provide higher precision of the usage numbers compared to server-side metrics which
might be skewed by sampling.
NOTE
To collect browser metrics, your application must be instrumented with the Application Insights JavaScript SDK.
browserTimings
| where notempty(processingDuration)
| extend _sum = processingDuration
| extend _count = itemCount
| extend _sum = _sum * _count
| summarize sum(_sum)/sum(_count) by bin(timestamp, 5m)
| render timechart
browserTimings
| where notempty(networkDuration)
| extend _sum = networkDuration
| extend _count = itemCount
| extend _sum = _sum * _count
| summarize sum(_sum) / sum(_count) by bin(timestamp, 5m)
| render timechart
browserTimings
| where notempty(receiveDuration)
| extend _sum = receiveDuration
| extend _count = itemCount
| extend _sum = _sum * _count
| summarize sum(_sum) / sum(_count) by bin(timestamp, 5m)
| render timechart
Failure metrics
The metrics in Failures show problems with processing requests, dependency calls, and thrown exceptions.
Browser exceptions (exceptions/browser)
This metric reflects the number of thrown exceptions from your application code running in browser. Only
exceptions that are tracked with a trackException() Application Insights API call are included in the metric.
PRE-AGGREGATED
UNIT OF MEASURE SUPPORTED AGGREGATIONS DIMENSIONS NOTES
exceptions
| where notempty(client_Browser)
| summarize sum(itemCount) by bin(timestamp, 5m)
| render barchart
PRE-AGGREGATED
UNIT OF MEASURE SUPPORTED AGGREGATIONS DIMENSIONS NOTES
dependencies
| where success == 'False'
| summarize sum(itemCount) by bin(timestamp, 5m)
| render barchart
Exceptions (exceptions/count)
Each time when you log an exception to Application Insights, there is a call to the trackException() method of the
SDK. The Exceptions metric shows the number of logged exceptions.
PRE-AGGREGATED
UNIT OF MEASURE SUPPORTED AGGREGATIONS DIMENSIONS NOTES
Count Count Cloud role name, Cloud role Log-based version uses Sum
instance, Device type aggregation
exceptions
| summarize sum(itemCount) by bin(timestamp, 5m)
| render barchart
PRE-AGGREGATED
UNIT OF MEASURE SUPPORTED AGGREGATIONS DIMENSIONS NOTES
Count Count Cloud role instance, Cloud Log-based version uses Sum
role name, Real or synthetic aggregation
traffic, Request performance,
Response code
requests
| where success == 'False'
| summarize sum(itemCount) by bin(timestamp, 5m)
| render barchart
PRE-AGGREGATED
UNIT OF MEASURE SUPPORTED AGGREGATIONS DIMENSIONS NOTES
Count Count Cloud role name, Cloud role Log-based version uses Sum
instance aggregation
exceptions
| where isempty(client_Browser)
| summarize sum(itemCount) by bin(timestamp, 5m)
| render barchart
Performance counters
Use metrics in the Performance counters category to access system performance counters collected by
Application Insights.
Available memory (performanceCounters/availableMemory)
performanceCounters
| where ((category == "Memory" and counter == "Available Bytes") or name == "availableMemory")
| extend performanceCounter_value = iif(itemType == "performanceCounter", value, todouble(''))
| summarize sum(performanceCounter_value) / count() by bin(timestamp, 1m)
| render timechart
performanceCounters
| where ((category == "ASP.NET Applications" and counter == "Request Execution Time") or name ==
"requestExecutionTime")
| extend performanceCounter_value = iif(itemType == "performanceCounter", value, todouble(''))
| summarize sum(performanceCounter_value) / count() by bin(timestamp, 1m)
| render timechart
performanceCounters
| where ((category == "ASP.NET Applications" and counter == "Requests/Sec") or name == "requestsPerSecond")
| extend performanceCounter_value = iif(itemType == "performanceCounter", value, todouble(''))
| summarize sum(performanceCounter_value) / count() by bin(timestamp, 1m)
| render timechart
performanceCounters
| where ((category == "ASP.NET Applications" and counter == "Requests In Application Queue") or name ==
"requestsInQueue")
| extend performanceCounter_value = iif(itemType == "performanceCounter", value, todouble(''))
| summarize sum(performanceCounter_value) / count() by bin(timestamp, 1m)
| render timechart
performanceCounters
| where ((category == "Process" and counter == "% Processor Time Normalized") or name ==
"processCpuPercentage")
| extend performanceCounter_value = iif(itemType == "performanceCounter", value, todouble(''))
| summarize sum(performanceCounter_value) / count() by bin(timestamp, 1m)
| render timechart
performanceCounters
| where ((category == "Process" and counter == "Private Bytes") or name == "processPrivateBytes")
| extend performanceCounter_value = iif(itemType == "performanceCounter", value, todouble(''))
| summarize sum(performanceCounter_value) / count() by bin(timestamp, 1m)
| render timechart
NOTE
The processor time metric is not available for the applications hosted in Azure App Services. Use the Process CPU metric to
track CPU utilization of the web applications hosted in App Services.
performanceCounters
| where ((category == "Processor" and counter == "% Processor Time") or name == "processorCpuPercentage")
| extend performanceCounter_value = iif(itemType == "performanceCounter", value, todouble(''))
| summarize sum(performanceCounter_value) / count() by bin(timestamp, 1m)
| render timechart
Server metrics
Dependency calls (dependencies/count)
This metric is in relation to the number of dependency calls.
dependencies
| summarize sum(itemCount) by bin(timestamp, 5m)
| render barchart
requests
| summarize sum(itemCount) by bin(timestamp, 5m)
| render barchart
requests
| where notempty(duration)
| extend request_duration = iif(itemType == 'request', duration, todouble(''))
| extend _sum = request_duration
| extend _count = itemCount
| extend _sum = _sum*_count
| summarize sum(_sum) / sum(_count) by bin(timestamp, 1m)
| render timechart
Usage metrics
Page view load time (pageViews/duration)
This metric refers to the amount of time it took for PageView events to load.
pageViews
| where notempty(duration)
| extend pageView_duration = iif(itemType == 'pageView', duration, todouble(''))
| extend _sum = pageView_duration
| extend _count = itemCount
| extend _sum = _sum * _count
| summarize sum(_sum) / sum(_count) by bin(timestamp, 5m)
| render barchart
pageViews
| summarize sum(itemCount) by bin(timestamp, 1h)
| render barchart
Sessions (sessions/count)
This metric refers to the count of distinct session IDs.
union traces, requests, pageViews, dependencies, customEvents, availabilityResults, exceptions, customMetrics,
browserTimings
| where notempty(session_Id)
| summarize dcount(session_Id) by bin(timestamp, 1h)
| render barchart
Traces (traces/count)
The count of trace statements logged with the TrackTrace() Application Insights API call.
traces
| summarize sum(itemCount) by bin(timestamp, 1h)
| render barchart
Users (users/count)
The number of distinct users who accessed your application. The accuracy of this metric may be significantly
impacted by using telemetry sampling and filtering.
Metrics in Application Insights are measured values and counts of events that are sent in telemetry from
your application. They help you detect performance issues and watch trends in how your application is
being used. There's a wide range of standard metrics, and you can also create your own custom metrics
and events.
NOTE
This article describes the classic metrics explorer experience which is currently deprecated and will eventually be
retired. We recommend checking out the new experience which is described in this article.
Metrics and event counts are displayed in charts of aggregated values such as sums, averages, or counts.
Here's a sample set of charts:
You find metrics charts everywhere in the Application Insights portal. In most cases, they can be
customized, and you can add more charts to the blade. From the Overview blade, click through to more
detailed charts (which have titles such as "Servers"), or click Metrics Explorer to open a new blade where
you can create custom charts.
Time range
You can change the Time range covered by the charts or grids on any blade.
If you're expecting some data that hasn't appeared yet, click Refresh. Charts refresh themselves at intervals,
but the intervals are longer for larger time ranges. It can take a while for data to come through the analysis
pipeline onto a chart.
To zoom into part of a chart, drag over it:
The value of the metric at a particular point is aggregated over the preceding sampling interval.
The sampling interval or "granularity" is shown at the top of the blade.
You can display more than one metric on a chart, though there are restrictions about the combinations that
can be displayed together. As soon as you choose one metric, some of the others are disabled.
If you coded custom metrics into your app (calls to TrackMetric and TrackEvent) they will be listed here.
If you coded custom metrics into your app and they include property values, you'll be able to select the
property in the list.
Is the chart too small for segmented data? Adjust its height:
Aggregation types
The legend at the side by default usually shows the aggregated value over the period of the chart. If you
hover over the chart, it shows the value at that point.
Each data point on the chart is an aggregate of the data values received in the preceding sampling interval
or "granularity". The granularity is shown at the top of the blade, and varies with the overall timescale of the
chart.
Metrics can be aggregated in different ways:
Count is a count of the events received in the sampling interval. It is used for events such as
requests. Variations in the height of the chart indicates variations in the rate at which the events
occur. But note that the numeric value changes when you change the sampling interval.
Sum adds up the values of all the data points received over the sampling interval, or the period of
the chart.
Average divides the Sum by the number of data points received over the interval.
Unique counts are used for counts of users and accounts. Over the sampling interval, or over the
period of the chart, the figure shows the count of different users seen in that time.
% - percentage versions of each aggregation are used only with segmented charts. The total always
adds up to 100%, and the chart shows the relative contribution of different components of a total.
Pin Y-axis
By default a chart shows Y axis values starting from zero till maximum values in the data range, to give a
visual representation of quantum of the values. But in some cases more than the quantum it might be
interesting to visually inspect minor changes in values. For customizations like this use the Y -axis range
editing feature to pin the Y -axis minimum or maximum value at desired place. Click on "Advanced
Settings" check box to bring up the Y -axis range Settings
To see the blade again, go to the overview blade and open Favorites:
If you chose Relative time range when you saved, the blade will be updated with the latest metrics. If you
chose Absolute time range, it will show the same data every time.
Set alerts
To be notified by email of unusual values of any metric, add an alert. You can choose either to send the
email to the account administrators, or to specific email addresses.
Continuous Export
If you want data continuously exported so that you can process it externally, consider using Continuous
export.
Power BI
If you want even richer views of your data, you can export to Power BI.
Analytics
Analytics is a more versatile way to analyze your telemetry using a powerful query language. Use it if you
want to combine or compute results from metrics, or perform an in-depth exploration of your app's recent
performance.
From a metric chart, you can click the Analytics icon to get directly to the equivalent Analytics query.
Troubleshooting
I don't see any data on my chart.
Filters apply to all the charts on the blade. Make sure that, while you're focusing on one chart, you
didn't set a filter that excludes all the data on another.
If you want to set different filters on different charts, create them in different blades, save them as
separate favorites. If you want, you can pin them to the dashboard so that you can see them
alongside each other.
If you group a chart by a property that is not defined on the metric, then there will be nothing on the
chart. Try clearing 'group by', or choose a different grouping property.
Performance data (CPU, IO rate, and so on) is available for Java web services, Windows desktop
apps, IIS web apps and services if you install status monitor, and Azure Cloud Services. It isn't
available for Azure websites.
Video
Next steps
Monitoring usage with Application Insights
Using Diagnostic Search
Using Search in Application Insights
10/20/2019 • 5 minutes to read • Edit Online
Search is a feature of Application Insights that you use to find and explore individual telemetry items,
such as page views, exceptions, or web requests. And you can view log traces and events that you have
coded.
(For more complex queries over your data, use Analytics.)
Go to the Event types' drop-down menu to see a list of telemetry items- server requests, page views,
custom events that you have coded, and so on. At the top of the results' list, is a summary chart
showing counts of events over time.
Click out of the drop-down menu or Refresh to get new events.
In Visual Studio
In Visual Studio, there's also an Application Insights Search window. It's most useful for displaying
telemetry events generated by the application that you're debugging. But it can also show the events
collected from your published app at the Azure portal.
Open the Search window in Visual Studio:
The Search window has features similar to the web portal:
The Track Operation tab is available when you open a request or a page view. An 'operation' is a
sequence of events that is associated with to a single request or page view. For example, dependency
calls, exceptions, trace logs, and custom events might be part of a single operation. The Track
Operation tab shows graphically the timing and duration of these events in relation to the request or
page view.
You can search for terms in any of the property values. This is useful if you have written custom events
with property values.
You might want to set a time range, as searches over a shorter range are faster.
Search for complete words, not substrings. Use quotation marks to enclose special characters.
STRING NOT FOUND FOUND
apple Find all events in the time range whose fields include
the word "apple"
apple AND banana Find events that contain both words. Use capital "AND",
apple banana not "and".
Short form.
apple OR banana Find events that contain either word. Use "OR", not
"or".
apple NOT banana Find events that contain one word but not the other.
Sampling
If your app generates a large amount of telemetry (and you are using the ASP.NET SDK version 2.0.0-
beta3 or later), the adaptive sampling module automatically reduces the volume that is sent to the
portal by sending only a representative fraction of events. However, events that are related to the same
request are selected or deselected as a group, so that you can navigate between related events.
Learn about sampling.
Q&A
How much data is retained?
See the Limits summary.
How can I see POST data in my server requests?
We don't log the POST data automatically, but you can use TrackTrace or log calls. Put the POST data
in the message parameter. You can't filter on the message in the same way you can filter on properties,
but the size limit is longer.
Next steps
Write complex queries in Analytics
Send logs and custom telemetry to Application Insights
Set up availability and responsiveness tests
Troubleshooting
Profile production applications in Azure with
Application Insights
12/8/2019 • 6 minutes to read • Edit Online
Limitations
The default data retention period is five days. The maximum data that's ingested per day is 10 GB.
There are no charges for using the Profiler service. For you to use it, your web app must be hosted in at least the
basic tier of the Web Apps feature of Azure App Service.
Next steps
Enable Application Insights Profiler for your Azure application. Also see:
App Services
Azure Cloud Services
Azure Service Fabric
Azure Virtual Machines and virtual machine scale sets
Profile live Azure App Service apps with
Application Insights
10/24/2019 • 2 minutes to read • Edit Online
You can run Profiler on ASP.NET and ASP.NET Core apps that are running on Azure App Service using Basic
service tier or higher. Enabling Profiler on Linux is currently only possible via this method.
4. Either follow the instructions on the pane to create a new resource or select an existing App Insights
resource to monitor your app. Also make sure the Profiler is On. If your Application Insights resource is
in a different subscription from your App Service, you can't use this page to configure Application
Insights. You can still do it manually though by creating the necessary app settings manually. The next
section contains instructions for manually enabling Profiler.
5. Profiler is now enabled using an App Services App Setting.
Enable Profiler manually or with Azure Resource Manager
Application Insights Profiler can be enabled by creating app settings for your Azure App Service. The page
with the options shown above creates these app settings for you. But you can automate the creation of these
settings using a template or other means. These settings will also work if your Application Insights resource is
in a different subscription from your Azure App Service. Here are the settings needed to enable the profiler:
APPINSIGHTS_PROFILERFEATURE_VERSION 1.0.0
DiagnosticServices_EXTENSION_VERSION ~3
You can set these values using Azure Resource Manager Templates, Azure Powershell, Azure CLI.
Enabling Profiler for other clouds manually
If you want to enable the profiler for other clouds, you can use the below app settings.
Disable Profiler
To stop or restart Profiler for an individual app's instance, under Web Jobs, go to the app resource. To delete
Profiler, go to Extensions.
We recommend that you have Profiler enabled on all your apps to discover any performance issues as early as
possible.
Profiler's files can be deleted when using WebDeploy to deploy changes to your web application. You can
prevent the deletion by excluding the App_Data folder from being deleted during deployment.
Next steps
Working with Application Insights in Visual Studio
Profile live Azure Cloud Services with Application
Insights
10/23/2019 • 2 minutes to read • Edit Online
If you can't find the file, see Set up diagnostics for Azure Cloud Services and Virtual Machines.
b. Add the following SinksConfig section as a child element of WadCfg :
<WadCfg>
<DiagnosticMonitorConfiguration>...</DiagnosticMonitorConfiguration>
<SinksConfig>
<Sink name="MyApplicationInsightsProfiler">
<!-- Replace with your own Application Insights instrumentation key. -->
<ApplicationInsightsProfiler>00000000-0000-0000-0000-000000000000</ApplicationInsightsProfiler>
</Sink>
</SinksConfig>
</WadCfg>
NOTE
If the diagnostics.wadcfgx file also contains another sink of type ApplicationInsights, all three of the following
instrumentation keys must match:
The key that's used by your application.
The key that's used by the ApplicationInsights sink.
The key that's used by the ApplicationInsightsProfiler sink.
You can find the actual instrumentation key value that's used by the ApplicationInsights sink in the
ServiceConfiguration.*.cscfg files. After the Visual Studio 15.5 Azure SDK release, only the instrumentation keys that
are used by the application and the ApplicationInsightsProfiler sink need to match each other.
5. Deploy your service with the new Diagnostics configuration, and Application Insights Profiler is configured
to run on your service.
Next steps
Generate traffic to your application (for example, launch an availability test). Then, wait 10 to 15 minutes for
traces to start to be sent to the Application Insights instance.
See Profiler traces in the Azure portal.
To troubleshoot Profiler issues, see Profiler troubleshooting.
Profile live Azure Service Fabric applications with
Application Insights
12/16/2019 • 2 minutes to read • Edit Online
"SinksConfig": {
"Sink": [
{
"name": "MyApplicationInsightsProfilerSink",
"ApplicationInsightsProfiler": "00000000-0000-0000-0000-000000000000"
}
]
}
For information about adding the Diagnostics extension to your deployment template, see Use monitoring
and diagnostics with a Windows VM and Azure Resource Manager templates.
4. Deploy your Service Fabric cluster by using your Azure Resource Manager template.
If your settings are correct, Application Insights Profiler will be installed and enabled when the Azure
Diagnostics extension is installed.
5. Add Application Insights to your Service Fabric application.
For Profiler to collect profiles for your requests, your application must be tracking operations with
Application Insights. For stateless APIs, you can refer to instructions for tracking Requests for profiling. For
more information about tracking custom operations in other kinds of apps, see track custom operations
with Application Insights .NET SDK.
6. Redeploy your application.
Next steps
Generate traffic to your application (for example, launch an availability test). Then, wait 10 to 15 minutes for
traces to start to be sent to the Application Insights instance.
See Profiler traces in the Azure portal.
For help with troubleshooting Profiler issues, see Profiler troubleshooting.
Profile web apps running on an Azure virtual
machine or a virtual machine scale set by using
Application Insights Profiler
12/12/2019 • 3 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
You can also deploy Azure Application Insights Profiler on these services:
Azure App Service
Azure Cloud Services
Azure Service Fabric
"SinksConfig": {
"Sink": [
{
"name": "ApplicationInsightsSink",
"ApplicationInsights": "85f73556-b1ba-46de-9534-606e08c6120f"
},
{
"name": "MyApplicationInsightsProfilerSink",
"ApplicationInsightsProfiler": "85f73556-b1ba-46de-9534-606e08c6120f"
}
]
},
$ConfigFilePath = [IO.Path]::GetTempFileName()
# After you export the currently deployed Diagnostics config to a file, edit it to include the
ApplicationInsightsProfiler sink.
(Get-AzVMDiagnosticsExtension -ResourceGroupName "MyRG" -VMName "MyVM").PublicSettings | Out-File -
Verbose $ConfigFilePath
# Set-AzVMDiagnosticsExtension might require the -StorageAccountName argument
# If your original diagnostics configuration had the storageAccountName property in the
protectedSettings section (which is not downloadable), be sure to pass the same original value you had
in this cmdlet call.
Set-AzVMDiagnosticsExtension -ResourceGroupName "MyRG" -VMName "MyVM" -DiagnosticsConfigurationPath
$ConfigFilePath
4. If the intended application is running through IIS, enable the IIS Http Tracing Windows feature.
a. Establish remote access to the environment, and then use the Add Windows features window. Or run the
following command in PowerShell (as administrator):
b. If establishing remote access is a problem, you can use the Azure CLI to run the following command:
3. Add the Application Insights Profiler sink to the SinksConfig node under WadCfg. If you don't already
have a SinksConfig section, you may need to add one. Be sure to specify the proper Application Insights
iKey in your settings. You'll need to switch the explorers mode to Read/Write in the upper right corner and
Press the blue 'Edit' button.
4. When you're done editing the config, press 'Put'. If the put is successful, a green check will appear in the
middle of the screen.
Can Profiler run on on-premises servers?
We have no plan to support Application Insights Profiler for on-premises servers.
Next steps
Generate traffic to your application (for example, launch an availability test). Then, wait 10 to 15 minutes for
traces to start to be sent to the Application Insights instance.
See Profiler traces in the Azure portal.
For help with troubleshooting Profiler issues, see Profiler troubleshooting.
Profile ASP.NET Core Azure Linux web apps with
Application Insights Profiler
10/23/2019 • 3 minutes to read • Edit Online
Prerequisites
The following instructions apply to all Windows, Linux, and Mac development environments:
Install the .NET Core SDK 2.1.2 or later.
Install Git by following the instructions at Getting Started - Installing Git.
3. Change the working directory to the root folder for the project.
4. Add the NuGet package to collect the Profiler traces:
7. Add a line of code in the HomeController.cs section to randomly delay a few seconds:
using System.Threading;
...
git init
git add .
git commit -m "first commit"
NOTE
Record your password to use later when deploying your web app.
3. Choose the deployment options. Set up a local Git repository in the web app by following the instructions
on the Azure portal. A Git repository is automatically created.
Use the username that you used to create the deployment credentials.
Use the app name that you used to create the web app by using App Service on Linux.
2. Deploy the project by pushing the changes to Azure:
```
Counting objects: 9, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (9/9), 1.78 KiB | 911.00 KiB/s, done.
Total 9 (delta 3), reused 0 (delta 0)
remote: Updating branch 'master'.
remote: Updating submodules.
remote: Preparing deployment for commit id 'd7369a99d7'.
remote: Generating deployment script.
remote: Running deployment command...
remote: Handling ASP.NET Core Web Application deployment.
remote: ......
remote: Restoring packages for /home/site/repository/EventPipeExampleLinux.csproj...
remote: .
remote: Installing Newtonsoft.Json 10.0.3.
remote: Installing Microsoft.ApplicationInsights.Profiler.Core 1.1.0-LKG
…
```
When the app settings are changed, the site automatically restarts. After the new settings are applied, the
Profiler immediately runs for two minutes. The Profiler then runs for two minutes every hour.
3. Generate some traffic to your website. You can generate traffic by refreshing the site About page a few
times.
4. Wait two to five minutes for the events to aggregate to Application Insights.
5. Browse to the Application Insights Performance pane in the Azure portal. You can view the Profiler traces
at the bottom right of the pane.
Known issues
Profile Now button doesn't work for Linux Profiler
The Linux version of the App Insights profiler does not yet support on demand profiling using the profile now
button.
Next steps
If you use custom containers that are hosted by Azure App Service, follow the instructions in Enable Service
Profiler for a containerized ASP.NET Core application to enable Application Insights Profiler.
Report any issues or suggestions to the Application Insights GitHub repository: ApplicationInsights-Profiler-
AspNetCore: Issues.
Configure Application Insights Profiler
10/23/2019 • 4 minutes to read • Edit Online
Profile Now Starts profiling sessions for all apps that are linked to this
instance of Application Insights.
Triggers Allows you to configure triggers that cause the profiler to run.
Profile Now
This option allows you to start a profiling session on demand. When you click this link, all profiler agents that are
sending data to this Application Insights instance will start to capture a profile. After 5 to 10 minutes, the profile
session will show in the list below.
For a user to manually trigger a profiler session, they require at minimum "write" access on their role for the
Application Insights component. In most cases, you get this access automatically and no additional work is needed.
If you're having issues, the subscription scope role to add would be the "Application Insights Component
Contributor" role. See more about role access control with Azure Monitoring.
Trigger Settings
Clicking the Triggers button on the menu bar opens the trigger settings box. You can set up trigger to start
profiling when the percentage of CPU or Memory use hits the level you set.
On / Off Button On: profiler can be started by this trigger; Off: profiler won't
be started by this trigger.
Memory threshold When this percentage of memory is in use, the profiler will be
started.
Duration Sets the length of time the profiler will run when triggered.
Cooldown Sets the length of time the profiler will wait before checking
for the memory or CPU usage again after it's triggered.
Triggered by How the session was started, either by a trigger, Profile Now,
or default sampling.
Machine Instance Name of the machine the profiler agent ran on.
CPU % Percentage of CPU that was being used while the profiler was
running.
Memory % Percentage of memory that was being used while the profiler
was running.
3. In the New performance test pane, configure the test target URL. Accept all default settings, and then
select Run test to start running the load test.
Next steps
Enable Profiler and view traces
Write code to track requests with Application Insights
10/24/2019 • 2 minutes to read • Edit Online
To view profiles for your application on the Performance page, Azure Application Insights needs to track requests
for your application. Application Insights can automatically track requests for applications that are built on already-
instrumented frameworks. Two examples are ASP.NET and ASP.NET Core.
For other applications, such as Azure Cloud Services worker roles and Service Fabric stateless APIs, you need to
write code to tell Application Insights where your requests begin and end. After you've written this code, requests
telemetry is sent to Application Insights. You can view the telemetry on the Performance page, and profiles are
collected for those requests.
To manually track requests, do the following:
1. Early in the application lifetime, add the following code:
using Microsoft.ApplicationInsights.Extensibility;
...
// Replace with your own Application Insights instrumentation key.
TelemetryConfiguration.Active.InstrumentationKey = "00000000-0000-0000-0000-000000000000";
For more information about this global instrumentation key configuration, see Use Service Fabric with
Application Insights.
2. For any piece of code that you want to instrument, add a StartOperation<RequestTelemetry> using
statement around it, as shown in the following example:
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.DataContracts;
...
var client = new TelemetryClient();
...
using (var operation = client.StartOperation<RequestTelemetry>("Insert_Your_Custom_Event_Unique_Name"))
{
// ... Code I want to profile.
}
getDetailsOperation.Telemetry.Success = true;
return details;
}
catch(Exception ex)
{
getDetailsOperation.Telemetry.Success = false;
General troubleshooting
Profiles are uploaded only if there are requests to your application while Profiler is running
Azure Application Insights Profiler collects profiling data for two minutes each hour. It also collects data when you
select the Profile Now button in the Configure Application Insights Profiler pane. But the profiling data is
uploaded only when it can be attached to a request that happened while Profiler was running.
Profiler writes trace messages and custom events to your Application Insights resource. You can use these events
to see how Profiler is running. If you think Profiler should be running and capturing traces, but they're not
displayed in the Performance pane, you can check to see how Profiler is running:
1. Search for trace messages and custom events sent by Profiler to your Application Insights resource. You
can use this search string to find the relevant data:
The following image displays two examples of searches from two AI resources:
At the left, the application isn't receiving requests while Profiler is running. The message explains
that the upload was canceled because of no activity.
At the right, Profiler started and sent custom events when it detected requests that happened while
Profiler was running. If the ServiceProfilerSample custom event is displayed, it means that Profiler
attached a trace to a request and you can view the trace in the Application Insights Performance
pane.
If no telemetry is displayed, Profiler is not running. To troubleshoot, see the troubleshooting sections
for your specific app type later in this article.
2. If there were requests while Profiler ran, make sure that the requests are handled by the part of your
application that has Profiler enabled. Although applications sometimes consist of multiple components,
Profiler is enabled for only some of the components. The Configure Application Insights Profiler pane
displays the components that have uploaded traces.
Other things to check
Make sure that your app is running on .NET Framework 4.6.
If your web app is an ASP.NET Core application, it must be running at least ASP.NET Core 2.0.
If the data you're trying to view is older than a couple of weeks, try limiting your time filter and try again.
Traces are deleted after seven days.
Make sure that proxies or a firewall have not blocked access to https://gateway.azureserviceprofiler.net.
Profiler isn't supported on free or shared app service plans. If you're using one of those plans, try scaling up to
one of the basic plans and Profiler should start working.
Double counting in parallel threads
In some cases, the total time metric in the stack viewer is more than the duration of the request.
This situation might occur when two or more threads are associated with a request and they are operating in
parallel. In that case, the total thread time is more than the elapsed time. One thread might be waiting on the other
to be completed. The viewer tries to detect this situation and omits the uninteresting wait. In doing so, it errs on
the side of displaying too much information rather than omit what might be critical information.
When you see parallel threads in your traces, determine which threads are waiting so that you can ascertain the
critical path for the request. Usually, the thread that quickly goes into a wait state is simply waiting on the other
threads. Concentrate on the other threads, and ignore the time in the waiting threads.
Error report in the profile viewer
Submit a support ticket in the portal. Be sure to include the correlation ID from the error message.
APPINSIGHTS_PROFILERFEATURE_VERSION 1.0.0
DiagnosticServices_EXTENSION_VERSION ~3
If you can't figure out why Profiler isn't working for you, you can download the log and send it to our team for
assistance, serviceprofilerhelp@microsoft.com.
Manual installation
When you configure Profiler, updates are made to the web app's settings. If your environment requires it, you can
apply the updates manually. An example might be that your application is running in a Web Apps environment for
PowerApps. To apply updates manually:
1. In the Web App Control pane, open Settings.
2. Set .NET Framework version to v4.6.
3. Set Always On to On.
4. Create these app settings:
APPINSIGHTS_PROFILERFEATURE_VERSION 1.0.0
DiagnosticServices_EXTENSION_VERSION ~3
-skip:Directory='.*\\App_Data\\jobs\\continuous\\ApplicationInsightsProfiler.*' -
skip:skipAction=Delete,objectname='dirPath',absolutepath='.*\\App_Data\\jobs\\continuous$' -
skip:skipAction=Delete,objectname='dirPath',absolutepath='.*\\App_Data\\jobs$' -
skip:skipAction=Delete,objectname='dirPath',absolutepath='.*\\App_Data$'
These parameters delete the folder that's used by Application Insights Profiler and unblock the redeploy process.
They don't affect the Profiler instance that's currently running.
How do I determine whether Application Insights Profiler is running?
Profiler runs as a continuous webjob in the web app. You can open the web app resource in the Azure portal. In
the WebJobs pane, check the status of ApplicationInsightsProfiler. If it isn't running, open Logs to get more
information.
To see whether Profiler is configured correctly by Azure Diagnostics, do the following three things:
1. First, check to see whether the contents of the Azure Diagnostics configuration that are deployed are what
you expect.
2. Second, make sure that Azure Diagnostics passes the proper iKey on the Profiler command line.
3. Third, check the Profiler log file to see whether Profiler ran but returned an error.
To check the settings that were used to configure Azure Diagnostics:
1. Sign in to the virtual machine (VM ), and then open the log file at this location. (The drive could be c: or d:
and the plugin version could be different.)
c:\logs\Plugins\Microsoft.Azure.Diagnostics.PaaSDiagnostics\1.11.3.12\DiagnosticsPlugin.log
or
c:\WindowsAzure\logs\Plugins\Microsoft.Azure.Diagnostics.PaaSDiagnostics\1.11.3.12\DiagnosticsPlugin.lo
g
2. In the file, you can search for the string WadCfg to find the settings that were passed to the VM to
configure Azure Diagnostics. You can check to see whether the iKey used by the Profiler sink is correct.
3. Check the command line that's used to start Profiler. The arguments that are used to launch Profiler are in
the following file. (The drive could be c: or d:)
D:\ProgramData\ApplicationInsightsProfiler\config.json
4. Make sure that the iKey on the Profiler command line is correct.
5. Using the path found in the preceding config.json file, check the Profiler log file. It displays the debug
information that indicates the settings that Profiler is using. It also displays status and error messages from
Profiler.
If Profiler is running while your application is receiving requests, the following message is displayed:
Activity detected from iKey.
When the trace is being uploaded, the following message is displayed: Start to upload trace.
When an exception occurs, you can automatically collect a debug snapshot from your live web application. The
snapshot shows the state of source code and variables at the moment the exception was thrown. The Snapshot
Debugger (preview ) in Azure Application Insights monitors exception telemetry from your web app. It collects
snapshots on your top-throwing exceptions so that you have the information you need to diagnose issues in
production. Include the Snapshot collector NuGet package in your application, and optionally configure
collection parameters in ApplicationInsights.config. Snapshots appear on exceptions in the Application Insights
portal.
You can view debug snapshots in the portal to see the call stack and inspect variables at each call stack frame. To
get a more powerful debugging experience with source code, open snapshots with Visual Studio 2019
Enterprise. In Visual Studio, you can also set Snappoints to interactively take snapshots without waiting for an
exception.
Debug snapshots are stored for 15 days. This retention policy is set on a per-application basis. If you need to
increase this value, you can request an increase by opening a support case in the Azure portal.
NOTE
Client applications (for example, WPF, Windows Forms or UWP) are not supported.
If you've enabled Snapshot Debugger but aren't seeing snapshots, check our Troubleshooting guide.
Grant permissions
Access to snapshots is protected by role-based access control (RBAC ). To inspect a snapshot, you must first be
added to the necessary role by a subscription owner.
NOTE
Owners and contributors do not automatically have this role. If they want to view snapshots, they must add themselves to
the role.
Subscription owners should assign the Application Insights Snapshot Debugger role to users who will inspect
snapshots. This role can be assigned to individual users or groups by subscription owners for the target
Application Insights resource or its resource group or subscription.
1. Navigate to the Application Insights resource in the Azure portal.
2. Click Access control (IAM ).
3. Click the +Add role assignment button.
4. Select Application Insights Snapshot Debugger from the Roles drop-down list.
5. Search for and enter a name for the user to add.
6. Click the Save button to add the user to the role.
IMPORTANT
Snapshots can potentially contain personal and other sensitive information in variable and parameter values.
Select an operation or exception in the right pane to open the End-to-End Transaction Details pane, then
select the exception event. If a snapshot is available for the given exception, an Open Debug Snapshot button
appears on the right pane with details for the exception.
In the Debug Snapshot view, you see a call stack and a variables pane. When you select frames of the call stack
in the call stack pane, you can view local variables and parameters for that function call in the variables pane.
Snapshots might include sensitive information, and by default they aren't viewable. To view snapshots, you must
have the Application Insights Snapshot Debugger role assigned to you.
The downloaded snapshot includes any symbol files that were found on your web application server. These
symbol files are required to associate snapshot data with source code. For App Service apps, make sure to
enable symbol deployment when you publish your web apps.
Limitations
The default data retention period is 15 days. For each Application Insights instance, a maximum number of 50
snapshots is allowed per day.
Publish symbols
The Snapshot Debugger requires symbol files on the production server to decode variables and to provide a
debugging experience in Visual Studio. Version 15.2 (or above) of Visual Studio 2017 publishes symbols for
release builds by default when it publishes to App Service. In prior versions, you need to add the following line
to your publish profile .pubxml file so that symbols are published in release mode:
<ExcludeGeneratedDebugSymbol>False</ExcludeGeneratedDebugSymbol>
For Azure Compute and other types, make sure that the symbol files are in the same folder of the main
application .dll (typically, wwwroot/bin ) or are available on the current path.
NOTE
For more information on the different symbol options that are available consult the Visual Studio documentation. For best
results, we recommend using “Full”, “Portable” or “Embedded”.
Optimized builds
In some cases, local variables can't be viewed in release builds because of optimizations that are applied by the
JIT compiler. However, in Azure App Services, the Snapshot Collector can deoptimize throwing methods that are
part of its Collection Plan.
TIP
Install the Application Insights Site Extension in your App Service to get deoptimization support.
Next steps
Enable Application Insights Snapshot Debugger for your application:
Azure App Service
Azure Cloud Services
Azure Service Fabric services
Azure Virtual Machines and virtual machine scale sets
On-premises virtual or physical machines
Beyond Application Insights Snapshot Debugger:
Set snappoints in your code to get snapshots without waiting for an exception.
Diagnose exceptions in your web apps explains how to make more exceptions visible to Application Insights.
Smart Detection automatically discovers performance anomalies.
Enable Snapshot Debugger for .NET apps in Azure
App Service
10/24/2019 • 2 minutes to read • Edit Online
Snapshot Debugger currently works for ASP.NET and ASP.NET Core apps that are running on Azure App
Service on Windows service plans.
Next steps
Generate traffic to your application that can trigger an exception. Then, wait 10 to 15 minutes for snapshots to
be sent to the Application Insights instance.
See snapshots in the Azure portal.
For help with troubleshooting Snapshot Debugger issues, see Snapshot Debugger troubleshooting.
Enable Snapshot Debugger for .NET apps in Azure
Service Fabric, Cloud Service, and Virtual Machines
10/24/2019 • 4 minutes to read • Edit Online
If your ASP.NET or ASP.NET core application runs in Azure App Service, it's highly recommended to enable
Snapshot Debugger through the Application Insights portal page. However, if your application requires a
customized Snapshot Debugger configuration, or a preview version of .NET core, then this instruction should
be followed in addition to the instructions for enabling through the Application Insights portal page.
If your application runs in Azure Service Fabric, Cloud Service, Virtual Machines, or on-premises machines, the
following instructions should be used.
<TelemetryProcessors>
<Add Type="Microsoft.ApplicationInsights.SnapshotCollector.SnapshotCollectorTelemetryProcessor,
Microsoft.ApplicationInsights.SnapshotCollector">
<!-- The default is true, but you can disable Snapshot Debugging by setting it to false -->
<IsEnabled>true</IsEnabled>
<!-- Snapshot Debugging is usually disabled in developer mode, but you can enable it by setting
this to true. -->
<!-- DeveloperMode is a property on the active TelemetryChannel. -->
<IsEnabledInDeveloperMode>false</IsEnabledInDeveloperMode>
<!-- How many times we need to see an exception before we ask for snapshots. -->
<ThresholdForSnapshotting>1</ThresholdForSnapshotting>
<!-- The maximum number of examples we create for a single problem. -->
<MaximumSnapshotsRequired>3</MaximumSnapshotsRequired>
<!-- The maximum number of problems that we can be tracking at any time. -->
<MaximumCollectionPlanSize>50</MaximumCollectionPlanSize>
<!-- How often we reconnect to the stamp. The default value is 15 minutes.-->
<ReconnectInterval>00:15:00</ReconnectInterval>
<!-- How often to reset problem counters. -->
<ProblemCounterResetInterval>1.00:00:00</ProblemCounterResetInterval>
<!-- The maximum number of snapshots allowed in ten minutes.The default value is 1. -->
<SnapshotsPerTenMinutesLimit>3</SnapshotsPerTenMinutesLimit>
<!-- The maximum number of snapshots allowed per day. -->
<SnapshotsPerDayLimit>30</SnapshotsPerDayLimit>
<!-- Whether or not to collect snapshot in low IO priority thread. The default value is true. --
>
<SnapshotInLowPriorityThread>true</SnapshotInLowPriorityThread>
<!-- Agree to send anonymous data to Microsoft to make this product better. -->
<ProvideAnonymousTelemetry>true</ProvideAnonymousTelemetry>
<!-- The limit on the number of failed requests to request snapshots before the telemetry
processor is disabled. -->
<FailedRequestLimit>3</FailedRequestLimit>
</Add>
</TelemetryProcessors>
4. Snapshots are collected only on exceptions that are reported to Application Insights. In some cases (for
example, older versions of the .NET platform), you might need to configure exception collection to see
exceptions with snapshots in the portal.
NOTE
Be sure that your application references version 2.1.1, or newer, of the Microsoft.ApplicationInsights.AspNetCore
package.
using Microsoft.ApplicationInsights.SnapshotCollector;
Add the following at the end of the ConfigureServices method in the Startup class in
Startup.cs .
services.AddSnapshotCollector((configuration) =>
Configuration.Bind(nameof(SnapshotCollectorConfiguration), configuration));
using Microsoft.ApplicationInsights.SnapshotCollector;
using Microsoft.Extensions.Options;
using Microsoft.ApplicationInsights.AspNetCore;
using Microsoft.ApplicationInsights.Extensibility;
// This method gets called by the runtime. Use this method to add services to the
container.
public void ConfigureServices(IServiceCollection services)
{
// Configure SnapshotCollector from application settings
services.Configure<SnapshotCollectorConfiguration>
(Configuration.GetSection(nameof(SnapshotCollectorConfiguration)));
{
"SnapshotCollectorConfiguration": {
"IsEnabledInDeveloperMode": false,
"ThresholdForSnapshotting": 1,
"MaximumSnapshotsRequired": 3,
"MaximumCollectionPlanSize": 50,
"ReconnectInterval": "00:15:00",
"ProblemCounterResetInterval":"1.00:00:00",
"SnapshotsPerTenMinutesLimit": 1,
"SnapshotsPerDayLimit": 30,
"SnapshotInLowPriorityThread": true,
"ProvideAnonymousTelemetry": true,
"FailedRequestLimit": 3
}
}
void ExampleRequest()
{
try
{
// TODO: Handle the request.
}
catch (Exception ex)
{
// Report the exception to Application Insights.
_telemetryClient.TrackException(ex);
Next steps
Generate traffic to your application that can trigger an exception. Then, wait 10 to 15 minutes for snapshots
to be sent to the Application Insights instance.
See snapshots in the Azure portal.
For help with troubleshooting Snapshot Debugger issues, see Snapshot Debugger troubleshooting.
Upgrading the Snapshot Debugger
12/8/2019 • 2 minutes to read • Edit Online
To provide the best possible security for your data, Microsoft is moving away from TLS 1.0 and TLS 1.1, which
have been shown to be vulnerable to determined attackers. If you're using an older version of the site extension, it
will require an upgrade to continue working. This document outlines the steps needed to upgrade your Snapshot
debugger to the latest version. There are two primary upgrade paths depending on if you enabled the Snapshot
Debugger using a site extension or if you used an SDK/Nuget added to your application. Both upgrade paths are
discussed below.
If you enabled the Snapshot debugger using the site extension, you can upgrade using the following procedure:
1. Sign in to the Azure portal.
2. Navigate to your resource that has Application Insights and Snapshot debugger enabled. For example, for a
Web App, navigate to the App Service resource:
3. Once you've navigated to your resource, click on the Extensions blade and wait for the list of extensions to
populate:
4. If any version of Application Insights extension for Azure App Service is installed, then select it and click
Delete. Confirm Yes to delete the extension and wait for the delete to complete before moving to the next
step.
5. Go to the Overview blade of your resource and click on Application Insights:
6. If this is the first time you've viewed the Application Insights blade for this App Service, you'll be prompted
to turn on Application Insights. Select Turn on Application Insights.
7. The current Application Insights settings are displayed. Unless you want to take the opportunity to change
your settings, you can leave them as is. The Apply button on the bottom of the blade isn't enabled by
default and you'll have to toggle one of the settings to activate the button. You don’t have to change any
actual settings, rather you can change the setting and then immediately change it back. We recommend
toggling the Profiler setting and then selecting Apply.
8. Once you click Apply, you'll be asked to confirm the changes.
NOTE
The site will be restarted as part of the upgrade process.
9. Click Yes to apply the changes and wait for the process to complete.
The site has now been upgraded and is ready to use.
If you enabled Application Insights Snapshot Debugger for your application, but are not seeing snapshots for
exceptions, you can use these instructions to troubleshoot. There can be many different reasons why snapshots
are not generated. You can run the snapshot health check to identify some of the possible common causes.
NOTE
The example above is from version 1.2.0 of the Microsoft.ApplicationInsights.SnapshotCollector NuGet package. In earlier
versions, the uploader process is called MinidumpUploader.exe and the log is less detailed.
In the previous example, the instrumentation key is c12a605e73c44346a984e00000000000 . This value should match
the instrumentation key for your application. The minidump is associated with a snapshot with the ID
139e411a23934dc0b9ea08a626db16c5 . You can use this ID later to locate the associated exception telemetry in
Application Insights Analytics.
The uploader scans for new PDBs about once every 15 minutes. Here's an example:
For applications that aren't hosted in App Service, the uploader logs are in the same folder as the minidumps:
%TEMP%\Dumps\<ikey> (where <ikey> is your instrumentation key).
<LocalResources>
<LocalStorage name="SnapshotStore" cleanOnRoleRecycle="false" sizeInMB="5120" />
</LocalResources>
2. Modify your role's startup code to add an environment variable that points to the SnapshotStore local
resource. For Worker Roles, the code should be added to your role's OnStart method:
For Web Roles (ASP.NET), the code should be added to your web application's Application_Start method:
using Microsoft.WindowsAzure.ServiceRuntime;
using System;
namespace MyWebRoleApp
{
public class MyMvcApplication : System.Web.HttpApplication
{
protected void Application_Start()
{
Environment.SetEnvironmentVariable("SNAPSHOTSTORE",
RoleEnvironment.GetLocalResource("SnapshotStore").RootPath);
// TODO: The rest of your application startup code
}
}
}
3. Update your role's ApplicationInsights.config file to override the temporary folder location used by
SnapshotCollector
<TelemetryProcessors>
<Add Type="Microsoft.ApplicationInsights.SnapshotCollector.SnapshotCollectorTelemetryProcessor,
Microsoft.ApplicationInsights.SnapshotCollector">
<!-- Use the SnapshotStore local resource for snapshots -->
<TempFolder>%SNAPSHOTSTORE%</TempFolder>
<!-- Other SnapshotCollector configuration options -->
</Add>
</TelemetryProcessors>
<TelemetryProcessors>
<Add Type="Microsoft.ApplicationInsights.SnapshotCollector.SnapshotCollectorTelemetryProcessor,
Microsoft.ApplicationInsights.SnapshotCollector">
<!-- Override the default shadow copy folder. -->
<ShadowCopyFolder>D:\SnapshotUploader</ShadowCopyFolder>
<!-- Other SnapshotCollector configuration options -->
</Add>
</TelemetryProcessors>
{
"ApplicationInsights": {
"InstrumentationKey": "<your instrumentation key>"
},
"SnapshotCollectorConfiguration": {
"ShadowCopyFolder": "D:\\SnapshotUploader"
}
}
Smart Detection automatically warns you of potential performance problems and failure anomalies in your web
application. It performs proactive analysis of the telemetry that your app sends to Application Insights. If there is
a sudden rise in failure rates, or abnormal patterns in client or server performance, you get an alert. This feature
needs no configuration. It operates if your application sends enough telemetry.
You can access the detections issued by Smart Detection both from the emails you receive, and from the Smart
Detection blade.
Video
Next steps
These diagnostic tools help you inspect the telemetry from your app:
Metric explorer
Search explorer
Analytics - powerful query language
Smart Detection is completely automatic. But maybe you'd like to set up some more alerts?
Manually configured metric alerts
Availability web tests
Smart Detection - Failure Anomalies
1/13/2020 • 10 minutes to read • Edit Online
Application Insights automatically alerts you in near real time if your web app experiences an abnormal rise in the
rate of failed requests. It detects an unusual rise in the rate of HTTP requests or dependency calls that are reported
as failed. For requests, failed requests usually have response codes of 400 or higher. To help you triage and
diagnose the problem, an analysis of the characteristics of the failures and related application data is provided in
the alert details. There are also links to the Application Insights portal for further diagnosis. The feature needs no
set-up nor configuration, as it uses machine learning algorithms to predict the normal failure rate.
This feature works for any web app, hosted in the cloud or on your own servers, that generate application request
or dependency data. For example, if you have a worker role that calls TrackRequest() or TrackDependency().
After setting up Application Insights for your project, and if your app generates a certain minimum amount of
data, Smart Detection of failure anomalies takes 24 hours to learn the normal behavior of your app, before it is
switched on and can send alerts.
Here's a sample alert:
The alert details will tell you:
The failure rate compared to normal app behavior.
How many users are affected - so you know how much to worry.
A characteristic pattern associated with the failures. In this example, there's a particular response code, request
name (operation), and application version. That immediately tells you where to start looking in your code.
Other possibilities could be a specific browser or client operating system.
The exception, log traces, and dependency failure (databases or other external components) that appear to be
associated with the characterized failures.
Links directly to relevant searches on the data in Application Insights.
How it works
Smart Detection monitors the data received from your app, and in particular the failure rates. This rule counts the
number of requests for which the Successful request property is false, and the number of dependency calls for
which the Successful call property is false. For requests, by default, Successful request == (resultCode < 400)
(unless you have written custom code to filter or generate your own TrackRequest calls).
Your app's performance has a typical pattern of behavior. Some requests or dependency calls will be more prone
to failure than others; and the overall failure rate may go up as load increases. Smart Detection uses machine
learning to find these anomalies.
As data comes into Application Insights from your web app, Smart Detection compares the current behavior with
the patterns seen over the past few days. If an abnormal rise in failure rate is observed by comparison with
previous performance, an analysis is triggered.
When an analysis is triggered, the service performs a cluster analysis on the failed request, to try to identify a
pattern of values that characterize the failures.
In the example above, the analysis has discovered that most failures are about a specific result code, request name,
Server URL host, and role instance.
When your service is instrumented with these calls, the analyzer looks for an exception and a dependency failure
that are associated with requests in the cluster it has identified, together with an example of any trace logs
associated with those requests.
The resulting analysis is sent to you as alert, unless you have configured it not to.
Like the alerts you set manually, you can inspect the state of the fired alert, which can be resolved if the issue is
fixed. Configure the alert rules in the Alerts page of your Application Insights resource. But unlike other alerts, you
don't need to set up or configure Smart Detection. If you want, you can disable it or change its target email
addresses.
Alert logic details
The alerts are triggered by our proprietary machine learning algorithm so we can't share the exact
implementation details. With that said, we understand that you sometimes need to know more about how the
underlying logic works. The primary factors that are evaluated to determine if an alert should be triggered are:
Analysis of the failure percentage of requests/dependencies in a rolling time window of 20 minutes.
A comparison of the failure percentage of the last 20 minutes to the rate in the last 40 minutes and the past
seven days, and looking for significant deviations that exceed X-times that standard deviation.
Using an adaptive limit for the minimum failure percentage, which varies based on the app’s volume of
requests/dependencies.
There is logic that can automatically resolve the fired alert monitor condition, if the issue is no longer detected
for 8-24 hours.
Configure alerts
You can disable Smart Detection alert rule from the portal or using Azure Resource Manager ( see template
example).
This alert rule is created with an associated Action Group named "Application Insights Smart Detection" that
contains email and webhook actions, and can be extended to trigger additional actions when the alert fires.
NOTE
Email notifications sent from this alert rule are now sent by default to users associated with the subscription's Monitoring
Reader and Monitoring Contributor roles. More information on this is available here. Notifications sent from this alert rule
follow the common alert schema.
Open the Alerts page. Failure Anomalies alert rules are included along with any alerts that you have set manually,
and you can see whether it is currently in the alert state.
From the percentage of requests and number of users affected, you can decide how urgent the issue is. In the
example above, the failure rate of 78.5% compares with a normal rate of 2.2%, indicates that something bad is
going on. On the other hand, only 46 users were affected. If it was your app, you'd be able to assess how serious
that is.
In many cases, you will be able to diagnose the problem quickly from the request name, exception, dependency
failure, and trace data provided.
In this example, there was an exception from SQL database due to request limit being reached.
Review recent alerts
Click Alerts in the Application Insights resource page to get to the most recent fired alerts:
Next steps
These diagnostic tools help you inspect the data from your app:
Metric explorer
Search explorer
Analytics - powerful query language
Smart detections are automatic. But maybe you'd like to set up some more alerts?
Manually configured metric alerts
Availability web tests
Smart Detection - Performance Anomalies
10/23/2019 • 9 minutes to read • Edit Online
Application Insights automatically analyzes the performance of your web application, and can warn you about
potential problems. You might be reading this because you received one of our smart detection notifications.
This feature requires no special setup, other than configuring your app for Application Insights (on ASP.NET, Java,
or Node.js, and in web page code). It is active when your app generates enough telemetry.
FAQ
So, Microsoft staff look at my data?
No. The service is entirely automatic. Only you get the notifications. Your data is private.
Do you analyze all the data collected by Application Insights?
Not at present. Currently, we analyze request response time, dependency response time and page load
time. Analysis of additional metrics is on our backlog looking forward.
What types of application does this work for?
These degradations are detected in any application that generates the appropriate telemetry. If you
installed Application Insights in your web app, then requests and dependencies are automatically
tracked. But in backend services or other apps, if you inserted calls to TrackRequest() or
TrackDependency, then Smart Detection will work in the same way.
Can I create my own anomaly detection rules or customize existing rules?
Not yet, but you can:
Set up alerts that tell you when a metric crosses a threshold.
Export telemetry to a database or to PowerBI, where you can analyze it yourself.
How often is the analysis performed?
We run the analysis daily on the telemetry from the previous day (full day in UTC timezone).
So does this replace metric alerts?
No. We don't commit to detecting every behavior that you might consider abnormal.
If I don't do anything in response to a notification, will I get a reminder?
No, you get a message about each issue only once. If the issue persists it will be updated in the Smart
Detection feed blade.
I lost the email. Where can I find the notifications in the portal?
In the Application Insights overview of your app, click the Smart Detection tile. There you'll be able to
find all notifications up to 90 days back.
How can I improve performance?
Slow and failed responses are one of the biggest frustrations for web site users, as you know from your own
experience. So, it's important to address the issues.
Triage
First, does it matter? If a page is always slow to load, but only 1% of your site's users ever have to look at it, maybe
you have more important things to think about. On the other hand, if only 1% of users open it, but it throws
exceptions every time, that might be worth investigating.
Use the impact statement (affected users or % of traffic) as a general guide, but be aware that it isn't the whole
story. Gather other evidence to confirm.
Consider the parameters of the issue. If it's geography-dependent, set up availability tests including that region:
there might simply be network issues in that area.
Diagnose slow page loads
Where is the problem? Is the server slow to respond, is the page very long, or does the browser have to do a lot of
work to display it?
Open the Browsers metric blade. The segmented display of browser page load time shows where the time is
going.
If Send Request Time is high, either the server is responding slowly, or the request is a post with a lot of data.
Look at the performance metrics to investigate response times.
Set up dependency tracking to see whether the slowness is due to external services or your database.
If Receiving Response is predominant, your page and its dependent parts - JavaScript, CSS, images and so
on (but not asynchronously loaded data) are long. Set up an availability test, and be sure to set the option to
load dependent parts. When you get some results, open the detail of a result and expand it to see the load
times of different files.
High Client Processing time suggests scripts are running slowly. If the reason isn't obvious, consider adding
some timing code and send the times in trackMetric calls.
Improve slow pages
There's a web full of advice on improving your server responses and page load times, so we won't try to repeat it
all here. Here are a few tips that you probably already know about, just to get you thinking:
Slow loading because of big files: Load the scripts and other parts asynchronously. Use script bundling. Break
the main page into widgets that load their data separately. Don't send plain old HTML for long tables: use a
script to request the data as JSON or other compact format, then fill the table in place. There are great
frameworks to help with all this. (They also entail big scripts, of course.)
Slow server dependencies: Consider the geographical locations of your components. For example, if you're
using Azure, make sure the web server and the database are in the same region. Do queries retrieve more
information than they need? Would caching or batching help?
Capacity issues: Look at the server metrics of response times and request counts. If response times peak
disproportionately with peaks in request counts, it's likely that your servers are stretched.
Next steps
These diagnostic tools help you inspect the telemetry from your app:
Profiler
Snapshot debugger
Analytics
Analytics smart diagnostics
Smart detections are completely automatic. But maybe you'd like to set up some more alerts?
Manually configured metric alerts
Availability web tests
Degradation in trace severity ratio (preview)
12/12/2019 • 2 minutes to read • Edit Online
Traces are widely used in applications, as they help tell the story of what happens behind the scenes. When things
go wrong, traces provide crucial visibility into the sequence of events leading to the undesired state. While traces
are generally unstructured, there is one thing that can concretely be learned from them – their severity level. In an
application’s steady state, we would expect the ratio between “good” traces (Info and Verbose) and “bad” traces
(Warning, Error, and Critical) to remain stable. The assumption is that “bad” traces may happen on a regular basis
to a certain extent due to any number of reasons (transient network issues for instance). But when a real problem
begins growing, it usually manifests as an increase in the relative proportion of “bad” traces vs “good” traces.
Application Insights Smart Detection automatically analyzes the traces logged by your application, and can warn
you about unusual patterns in the severity of your trace telemetry.
This feature requires no special setup, other than configuring trace logging for your app (see how to configure a
trace log listener for .NET or Java). It is active when your app generates enough exception telemetry.
Application Insights automatically analyzes the exceptions thrown in your application, and can warn you about
unusual patterns in your exception telemetry.
This feature requires no special setup, other than configuring exception reporting for your app. It is active when
your app generates enough exception telemetry.
Application Insights automatically analyzes the memory consumption of each process in your application, and can
warn you about potential memory leaks or increased memory consumption.
This feature requires no special setup, other than configuring performance counters for your app. It's active when
your app generates enough memory performance counters telemetry (for example, Private Bytes).
Application Insights automatically analyzes the telemetry generated by your application and detects potential
security issues. This capability enables you to identify potential security problems, and handle them by fixing the
application or by taking the necessary security measures.
This feature requires no special setup, other than configuring your app to send telemetry.
In this article, we will describe how to set up alert rules that monitor for issues like startup failures, crashes, and
role recycle loops in Azure Cloud Services (web and worker roles).
The method described in this article is based on the Azure Diagnostics integration with Application Insights, and
the recently released Log Alerts for Application Insights capability.
NOTE
The base query below checks for issues in a time window of 30 minutes, and assumes a 10 minutes latency in ingesting the
telemetry records. These defaults can be configured as you see fit.
NOTE
In the examples below, an issue will be detected if more than three events are found during the analyzed time window. This
default can be configured to change the sensitivity of the alert rule.
// Detect failures in the OnStart method
EventLogs
| where eventId == '2001'
| where message contains '.OnStart()'
| summarize Failures = sum(itemCount) by cloud_RoleInstance, cloud_RoleName
| where Failures > 3
Create an alert
In the navigation menu within your Application Insights resource, go to Alerts, and then select New Alert Rule.
In the Create rule window, under the Define alert condition section, click on Add criteria, and then select
Custom log search.
In the Search query box, paste the combined query you prepared in the previous step.
Then, continue to the Threshold box, and set its value to 0. You may optionally tweak the Period and Frequency
fields. Click Done.
Under the Define alert details section, provide a Name and Description to the alert rule, and set its Severity.
Also, make sure that the Enable rule upon creation button is set to Yes.
Under the Define action group section, you can select an existing Action group or create a new one. You may
choose to have the action group contain multiple actions of various types.
Once you've defined the Action group, confirm your changes and click Create alert rule.
Next Steps
Learn more about automatically detecting:
Failure anomalies Memory Leaks Performance anomalies
Manage Application Insights smart detection rules
using Azure Resource Manager templates
1/8/2020 • 3 minutes to read • Edit Online
Smart detection rules in Application Insights can be managed and configured using Azure Resource Manager
templates. This method can be used when deploying new Application Insights resources with Azure Resource
Manager automation, or for modifying the settings of existing resources.
Examples
Below are a few examples showing how to configure the settings of smart detection rules using Azure Resource
Manager templates. All samples refer to an Application Insights resource named “myApplication”, and to the "long
dependency duration smart detection rule", which is internally named “longdependencyduration”. Make sure to
replace the Application Insights resource name, and to specify the relevant smart detection rule internal name.
Check the table below for a list of the corresponding internal Azure Resource Manager names for each smart
detection rule.
Disable a smart detection rule
{
"apiVersion": "2018-05-01-preview",
"name": "myApplication",
"type": "Microsoft.Insights/components",
"location": "[resourceGroup().location]",
"properties": {
"ApplicationId": "myApplication"
},
"resources": [
{
"apiVersion": "2018-05-01-preview",
"name": "longdependencyduration",
"type": "ProactiveDetectionConfigs",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Insights/components', 'myApplication')]"
],
"properties": {
"name": "longdependencyduration",
"sendEmailsToSubscriptionOwners": true,
"customEmails": [],
"enabled": false
}
}
]
}
{
"apiVersion": "2018-05-01-preview",
"name": "myApplication",
"type": "Microsoft.Insights/components",
"location": "[resourceGroup().location]",
"properties": {
"ApplicationId": "myApplication"
},
"resources": [
{
"apiVersion": "2018-05-01-preview",
"name": "longdependencyduration",
"type": "ProactiveDetectionConfigs",
"location": "[resourceGroup().location]",
"dependsOn": [
"[resourceId('Microsoft.Insights/components', 'myApplication')]"
],
"properties": {
"name": "longdependencyduration",
"sendEmailsToSubscriptionOwners": false,
"customEmails": [],
"enabled": true
}
}
]
}
NOTE
Failure Anomalies is a global service therefore rule location is create on the global location.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"type": "microsoft.alertsmanagement/smartdetectoralertrules",
"apiVersion": "2019-03-01",
"name": "Failure Anomalies - my-app",
"location": "global",
"properties": {
"description": "Failure Anomalies notifies you of an unusual rise in the rate of failed HTTP
requests or dependency calls.",
"state": "Enabled",
"severity": "2",
"frequency": "PT1M",
"detector": {
"id": "FailureAnomaliesDetector"
},
"scope": ["/subscriptions/00000000-1111-2222-3333-
444444444444/resourceGroups/MyResourceGroup/providers/microsoft.insights/components/my-app"],
"actionGroups": {
"groupIds": ["/subscriptions/00000000-1111-2222-3333-
444444444444/resourcegroups/MyResourceGroup/providers/microsoft.insights/actiongroups/MyActionGroup"]
}
}
}
]
}
NOTE
This Azure Resource Manager template is unique to the Failure Anomalies alert rule and is different from the other classic
Smart Detection rules described in this article.
NOTE
Smart detection rules marked as preview don’t support email notifications. Therefore, you can only set the enabled property
for these rules.
Next Steps
Learn more about automatically detecting:
Failure anomalies
Memory Leaks
Performance anomalies
Smart Detection e-mail notification change
12/16/2019 • 2 minutes to read • Edit Online
Based on customer feedback, on April 1, 2019, we’re changing the default roles who receive email notifications
from Smart Detection.
What is changing?
Currently, Smart Detection email notifications are sent by default to the Subscription Owner, Subscription
Contributor, and Subscription Reader roles. These roles often include users who are not actively involved in
monitoring, which causes many of these users to receive notifications unnecessarily. To improve this experience, we
are making a change so that email notifications only go to the Monitoring Reader and Monitoring Contributor
roles by default.
NOTE
Specific recipients of Smart Detection notifications, configured using the Additional email recipients option in the rule
settings, will not be affected by this change. These recipients will continue receiving the email notifications.
If you have any questions or concerns about this change, don’t hesitate to contact us.
Next steps
Learn more about Smart Detection:
Failure anomalies
Memory Leaks
Performance anomalies
Usage analysis with Application Insights
11/8/2019 • 5 minutes to read • Edit Online
Which features of your web or mobile app are most popular? Do your users achieve their goals with your app?
Do they drop out at particular points, and do they return later? Azure Application Insights helps you gain
powerful insights into how people use your app. Every time you update your app, you can assess how well it
works for users. With this knowledge, you can make data driven decisions about your next development cycles.
<script type="text/javascript">
var sdkInstance="appInsightsSDK";window[sdkInstance]="appInsights";var
aiName=window[sdkInstance],aisdk=window[aiName]||function(e){function n(e){t[e]=function(){var
n=arguments;t.queue.push(function(){t[e].apply(t,n)})}}var t={config:e};t.initialize=!0;var
i=document,a=window;setTimeout(function(){var
n=i.createElement("script");n.src=e.url||"https://az416426.vo.msecnd.net/scripts/b/ai.2.min.js",i.ge
tElementsByTagName("script")[0].parentNode.appendChild(n)});try{t.cookie=i.cookie}catch(e){}t.queue=
[],t.version=2;for(var r=
["Event","PageView","Exception","Trace","DependencyData","Metric","PageViewPerformance"];r.length;)n
("track"+r.pop());n("startTrackPage"),n("stopTrackPage");var
s="Track"+r[0];if(n("start"+s),n("stop"+s),n("setAuthenticatedUserContext"),n("clearAuthenticatedUse
rContext"),n("flush"),!
(!0===e.disableExceptionTracking||e.extensionConfig&&e.extensionConfig.ApplicationInsightsAnalytics&
&!0===e.extensionConfig.ApplicationInsightsAnalytics.disableExceptionTracking)){n("_"+
(r="onerror"));var o=a[r];a[r]=function(e,n,i,a,s){var c=o&&o(e,n,i,a,s);return!0!==c&&t["_"+r]
({message:e,url:n,lineNumber:i,columnNumber:a,error:s}),c},e.autoExceptionInstrumented=!0}return t}(
{
instrumentationKey:"INSTRUMENTATION_KEY"
}
);window[aiName]=aisdk,aisdk.queue&&0===aisdk.queue.length&&aisdk.trackPageView({});
</script>
To learn more advanced configurations for monitoring websites, check out the JavaScript SDK reference
article.
3. Mobile app code: Use the App Center SDK to collect events from your app, then send copies of these
events to Application Insights for analysis by following this guide.
4. Get telemetry: Run your project in debug mode for a few minutes, and then look for results in the
Overview blade in Application Insights.
Publish your app to monitor your app's performance and find out what your users are doing with your
app.
Insights on the right point out interesting patterns in the set of data.
The Users report counts the numbers of unique users that access your pages within your chosen time
periods. For web apps, users are counted by using cookies. If someone accesses your site with different
browsers or client machines, or clears their cookies, then they will be counted more than once.
The Sessions report counts the number of user sessions that access your site. A session is a period of
activity by a user, terminated by a period of inactivity of more than half an hour.
More about the Users, Sessions, and Events tools
A | B Testing
If you don't know which variant of a feature will be more successful, release both of them, making each
accessible to different users. Measure the success of each, and then move to a unified version.
For this technique, you attach distinct property values to all the telemetry that is sent by each version of your
app. You can do that by defining properties in the active TelemetryContext. These default properties are added
to every telemetry message that the application sends - not just your custom messages, but the standard
telemetry as well.
In the Application Insights portal, filter and split your data on the property values, so as to compare the
different versions.
To do this, set up a telemetry initializer:
ASP.NET apps
NOTE
Adding initializer using ApplicationInsights.config or using TelemetryConfiguration.Active is not valid for
ASP.NET Core applications.
For ASP.NET Core applications, adding a new TelemetryInitializer is done by adding it to the Dependency
Injection container, as shown below. This is done in ConfigureServices method of your Startup.cs class.
using Microsoft.ApplicationInsights.Extensibility;
using CustomInitializer.Telemetry;
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}
All new TelemetryClients automatically add the property value you specify. Individual telemetry events can
override the default values.
Next steps
Users, Sessions, Events
Funnels
Retention
User Flows
Workbooks
Add user context
Send user context IDs to enable usage experiences in
Azure Application Insights
12/16/2019 • 3 minutes to read • Edit Online
Tracking users
Application Insights enables you to monitor and track your users through a set of product usage tools:
Users, Sessions, Events
Funnels
Retention Cohorts
Workbooks
In order to track what a user does over time, Application Insights needs an ID for each user or session. Include the
following IDs in every custom event or page view.
Users, Funnels, Retention, and Cohorts: Include user ID.
Sessions: Include session ID.
NOTE
This is an advanced article outlining the manual steps for tracking user activity with Application Insights. With many web
applications these steps may not be required, as the default server-side SDKs in conjunction with the Client/Browser-side
JavaScript SDK, are often sufficient to automatically track user activity. If you haven't configured client-side monitoring in
addition to the server-side SDK, do that first and test to see if the user behavior analytics tools are performing as expected.
using System;
using System.Web;
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
namespace MvcWebRole.Telemetry
{
/*
* Custom TelemetryInitializer that sets the user ID.
*
*/
public class MyTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
var ctx = HttpContext.Current;
// If telemetry initializer is called as part of request execution and not from some async thread
if (ctx != null)
{
var requestTelemetry = ctx.GetRequestTelemetry();
// Set the user and session ids from requestTelemetry.Context.User.Id, which is populated in
Application_PostAcquireRequestState in Global.asax.cs.
if (requestTelemetry != null && !string.IsNullOrEmpty(requestTelemetry.Context.User.Id) &&
(string.IsNullOrEmpty(telemetry.Context.User.Id) ||
string.IsNullOrEmpty(telemetry.Context.Session.Id)))
{
// Set the user id on the Application Insights telemetry item.
telemetry.Context.User.Id = requestTelemetry.Context.User.Id;
Global.asax.cs
using System.Web;
using System.Web.Http;
using System.Web.Mvc;
using System.Web.Optimization;
using System.Web.Routing;
namespace MvcWebRole.Telemetry
{
public class MvcApplication : HttpApplication
{
protected void Application_Start()
{
AreaRegistration.RegisterAllAreas();
GlobalConfiguration.Configure(WebApiConfig.Register);
FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);
RouteConfig.RegisterRoutes(RouteTable.Routes);
BundleConfig.RegisterBundles(BundleTable.Bundles);
}
Next steps
To enable usage experiences, start sending custom events or page views.
If you already send custom events or page views, explore the Usage tools to learn how users use your service.
Usage overview
Users, Sessions, and Events
Funnels
Retention
Workbooks
Users, sessions, and events analysis in Application
Insights
12/13/2019 • 2 minutes to read • Edit Online
Find out when people use your web app, what pages they're most interested in, where your users are located, and
what browsers and operating systems they use. Analyze business and usage telemetry by using Azure
Application Insights.
Get started
If you don't yet see data in the users, sessions, or events blades in the Application Insights portal, learn how to get
started with the usage tools.
Next steps
To enable usage experiences, start sending custom events or page views.
If you already send custom events or page views, explore the Usage tools to learn how users use your service.
Funnels
Retention
User Flows
Workbooks
Add user context
Discover how customers are using your application
with Application Insights Funnels
10/24/2019 • 2 minutes to read • Edit Online
Understanding the customer experience is of the utmost importance to your business. If your application involves
multiple stages, you need to know if most customers are progressing through the entire process, or if they are
ending the process at some point. The progression through a series of steps in a web application is known as a
funnel. You can use Azure Application Insights Funnels to gain insights into your users, and monitor step-by-step
conversion rates.
Next steps
Usage overview
Users, Sessions, and Events
Retention
Workbooks
Add user context
Export to Power BI
Application Insights cohorts
12/13/2019 • 4 minutes to read • Edit Online
A cohort is a set of users, sessions, events, or operations that have something in common. In Azure Application
Insights, cohorts are defined by an analytics query. In cases where you have to analyze a specific set of users or
events repeatedly, cohorts can give you more flexibility to express exactly the set you’re interested in.
NOTE
After they're created, cohorts are available from the Users, Sessions, Events, and User Flows tools.
Now this cohort represents all user IDs sent with any custom event or page view on 5 separate days in the
past 28.
5. Select Save.
TIP
Give your cohort a name, like “Engaged Users (5+ Days).” Save it to “My reports” or “Shared reports,” depending on
whether you want other people who have access to this Application Insights resource to see this cohort.
1. Open the Cohorts tool, select the Template Gallery tab, and select Blank Users cohort.
There are three sections:
A Markdown text section, where you describe the cohort in more detail for others on your team.
A parameters section, where you make your own parameters, like Activities and other drop-down
boxes from the previous two examples.
A query section, where you define the cohort by using an analytics query.
In the query section, you write an analytics query. The query selects the certain set of rows that
describe the cohort you want to define. The Cohorts tool then implicitly adds a “| summarize by
user_Id” clause to the query. This data is previewed below the query in a table, so you can make sure
your query is returning results.
NOTE
If you don’t see the query, try resizing the section to make it taller and reveal the query. The animated .gif at
the beginning of this section illustrates the resizing behavior.
2. Copy and paste the following text into the query editor:
3. Select Run Query. If you don't see user IDs appear in the table, change to a country/region where your
application has users.
4. Save and name the cohort.
Learn more
Analytics query language
Users, sessions, events
User flows
Usage overview
Impact analysis with Application Insights
12/13/2019 • 3 minutes to read • Edit Online
Impact analyzes how load times and other properties influence conversion rates for various parts of your app. To
put it more precisely, it discovers how any dimension of a page view, custom event, or request affects the
usage of a different page view or custom event.
1. Select a page view from the For the page view dropdown.
2. Leave the analyze how its dropdown on the default selection of Duration (In this context Duration is an alias
for Page Load Time.)
3. For the impacts the usage of dropdown, select a custom event. This event should correspond to a UI element
on the page view you selected in step 1.
In this instance as Product Page load time increases the conversion rate to Purchase Product clicked goes
down. Based on the distribution above, an optimal page load duration of 3.5 seconds could be targeted to achieve
a potential 55% conversion rate. Further performance improvements to reduce load time below 3.5 seconds do not
currently correlate with additional conversion benefits.
Next steps
To enable usage experiences, start sending custom events or page views.
If you already send custom events or page views, explore the Usage tools to learn how users use your service.
Funnels
Retention
User Flows
Workbooks
Add user context
User retention analysis for web applications with
Application Insights
10/24/2019 • 2 minutes to read • Edit Online
The retention feature in Azure Application Insights helps you analyze how many users return to your app, and
how often they perform particular tasks or achieve goals. For example, if you run a game site, you could compare
the numbers of users who return to the site after losing a game with the number who return after winning. This
knowledge can help you improve both your user experience and your business strategy.
Get started
If you don't yet see data in the retention tool in the Application Insights portal, learn how to get started with the
usage tools.
1. The toolbar allows users to create new retention reports, open existing retention reports, save current
retention report or save as, revert changes made to saved reports, refresh data on the report, share report via
email or direct link, and access the documentation page.
2. By default, retention shows all users who did anything then came back and did anything else over a period.
You can select different combination of events to narrow the focus on specific user activities.
3. Add one or more filters on properties. For example, you can focus on users in a particular country or region.
Click Update after setting the filters.
4. The overall retention chart shows a summary of user retention across the selected time period.
5. The grid shows the number of users retained according to the query builder in #2. Each row represents a
cohort of users who performed any event in the time period shown. Each cell in the row shows how many of
that cohort returned at least once in a later period. Some users may return in more than one period.
6. The insights cards show top five initiating events, and top five returned events to give users a better
understanding of their retention report.
Users can hover over cells on the retention tool to access the analytics button and tool tips explaining what the
cell means. The Analytics button takes users to the Analytics tool with a pre-populated query to generate users
from the cell.
appinsights.trackEvent("won game");
telemetry.TrackEvent("won game");
Next steps
To enable usage experiences, start sending custom events or page views.
If you already send custom events or page views, explore the Usage tools to learn how users use your service.
Users, Sessions, Events
Funnels
User Flows
Workbooks
Add user context
Analyze user navigation patterns with User Flows in
Application Insights
12/13/2019 • 5 minutes to read • Edit Online
The User Flows tool visualizes how users navigate between the pages and features of your site. It's great for
answering questions like:
How do users navigate away from a page on your site?
What do users click on a page on your site?
Where are the places that users churn most from your site?
Are there places where users repeat the same action over and over?
The User Flows tool starts from an initial page view, custom event, or exception that you specify. Given this initial
event, User Flows shows the events that happened before and afterwards during user sessions. Lines of varying
thickness show how many times each path was followed by users. Special Session Started nodes show where
the subsequent nodes began a session. Session Ended nodes show how many users sent no page views or
custom events after the preceding node, highlighting where users probably left your site.
NOTE
Your Application Insights resource must contain page views or custom events to use the User Flows tool. Learn how to set
up your app to collect page views automatically with the Application Insights JavaScript SDK.
Where are the places that users churn most from your site?
Watch for Session Ended nodes that appear high-up in a column in the visualization, especially early in a flow.
This means many users probably churned from your site after following the preceding path of pages and UI
interactions. Sometimes churn is expected - after completing a purchase on an eCommerce site, for example -
but usually churn is a sign of design problems, poor performance, or other issues with your site that can be
improved.
Keep in mind, that Session Ended nodes are based only on telemetry collected by this Application Insights
resource. If Application Insights doesn't receive telemetry for certain user interactions, users could still have
interacted with your site in those ways after the User Flows tool says the session ended.
Are there places where users repeat the same action over and over?
Look for a page view or custom event that is repeated by many users across subsequent steps in the
visualization. This usually means that users are performing repetitive actions on your site. If you find repetition,
think about changing the design of your site or adding new functionality to reduce repetition. For example,
adding bulk edit functionality if you find users performing repetitive actions on each row of a table element.
Common questions
Does the initial event represent the first time the event appears in a session, or any time it appears in a
session?
The initial event on the visualization only represents the first time a user sent that page view or custom event
during a session. If users can send the initial event multiple times in a session, then the "Step 1" column only
shows how users behave after the first instance of initial event, not all instances.
Some of the nodes in my visualization are too high-level. For example, a node that just says “Button Clicked.”
How can I break it down into more detailed nodes?
Use the Split by options in the Edit menu:
1. Choose the event you want to break down in the Event menu.
2. Choose a dimension in the Dimension menu. For example, if you have an event called “Button Clicked,” try a
custom property called “Button Name.”
Next steps
Usage overview
Users, Sessions, and Events
Retention
Adding custom events to your app
Troubleshoot user behavior analytics tools in
Application Insights
12/16/2019 • 3 minutes to read • Edit Online
Have questions about the user behavior analytics tools in Application Insights: Users, Sessions, Events, Funnels,
User Flows, Retention, or Cohorts? Here are some answers.
Counting Users
The user behavior analytics tools show that my app has one user/session, but I know my app has many
users/sessions. How can I fix these incorrect counts?
All telemetry events in Application Insights have an anonymous user ID and a session ID as two of their standard
properties. By default, all of the usage analytics tools count users and sessions based on these IDs. If these
standard properties aren't being populated with unique IDs for each user and session of your app, you'll see an
incorrect count of users and sessions in the usage analytics tools.
If you're monitoring a web app, the easiest solution is to add the Application Insights JavaScript SDK to your app,
and make sure the script snippet is loaded on each page you want to monitor. The JavaScript SDK automatically
generates anonymous user and session IDs, then populates telemetry events with these IDs as they're sent from
your app.
If you're monitoring a web service (no user interface), create a telemetry initializer that populates the anonymous
user ID and session ID properties according to your service's notions of unique users and sessions.
If your app is sending authenticated user IDs, you can count based on authenticated user IDs in the Users tool. In
the "Show" dropdown, choose "Authenticated users."
The user behavior analytics tools don't currently support counting users or sessions based on properties other than
anonymous user ID, authenticated user ID, or session ID.
Naming Events
My app has thousands of different page view and custom event names. It's hard to distinguish between
them, and the user behavior analytics tools often become unresponsive. How can I fix these naming
issues?
Page view and custom event names are used throughout the user behavior analytics tools. Naming events well is
critical to getting value from these tools. The goal is a balance between having too few, overly generic names
("Button clicked") and having too many, overly specific names ("Edit button clicked on
http://www.contoso.com/index").
To make any changes to the page view and custom event names your app is sending, you need to change your
app's source code and redeploy. All telemetry data in Application Insights is stored for 90 days and cannot
be deleted, so changes you make to event names will take 90 days to fully manifest. For the 90 days after making
name changes, both the old and new event names will show up in your telemetry, so adjust queries and
communicate within your teams, accordingly.
If your app is sending too many page view names, check whether these page view names are specified manually in
code or if they're being sent automatically by the Application Insights JavaScript SDK:
If the page view names are manually specified in code using the trackPageView API, change the name to be
less specific. Avoid common mistakes like putting the URL in the name of the page view. Instead, use the
URL parameter of the trackPageView API. Move other details from the page view name into custom
properties.
If the Application Insights JavaScript SDK is automatically sending page view names, you can either change
your pages' titles or switch to manually sending page view names. The SDK sends the title of each page as
the page view name, by default. You could change your titles to be more general, but be mindful of SEO and
other impacts this change could have. Manually specifying page view names with the trackPageView API
overrides the automatically collected names, so you could send more general names in telemetry without
changing page titles.
If your app is sending too many custom event names, change the name in the code to be less specific. Again, avoid
putting URLs and other per-page or dynamic information in the custom event names directly. Instead, move these
details into custom properties of the custom event with the trackEvent API. For example, instead of
appInsights.trackEvent("Edit button clicked on http://www.contoso.com/index") , we suggest something like
appInsights.trackEvent("Edit button clicked", { "Source URL": "http://www.contoso.com/index" }) .
Next steps
User behavior analytics tools overview
Get help
Stack Overflow
Annotations on metric charts in Application Insights
11/6/2019 • 2 minutes to read • Edit Online
Annotations on Metrics Explorer charts show where you deployed a new build, or other significant events.
Annotations make it easy to see whether your changes had any effect on your application's performance. They can
be automatically created by the Azure Pipelines build system. You can also create annotations to flag any event you
like by creating them from PowerShell.
NOTE
This article reflects the deprecated classic metrics experience. Annotations are only currently available in the classic
experience and in workbooks. To learn more about the current metrics experience, see Advanced features of Azure Metrics
Explorer.
3. In a separate browser window, open or create the release template that manages your Azure Pipelines
deployments.
4. Select Add task, and then select the Application Insights Release Annotation task from the menu.
NOTE
The Release Annotation task currently supports only Windows-based agents; it won't run on Linux, macOS, or other
types of agents.
5. Under Application ID, paste the Application Insights ID you copied from the API Access tab.
6. Back in the Application Insights API Access window, select Create API Key.
7. In the Create API key window, type a description, select Write annotations, and then select Generate
key. Copy the new key.
8. In the release template window, on the Variables tab, select Add to create a variable definition for the new
API key.
9. Under Name, enter ApiKey , and under Value, paste the API key you copied from the API Access tab.
10. Select Save in the main release template window to save the template.
View annotations
Now, whenever you use the release template to deploy a new release, an annotation is sent to Application Insights.
The annotations appear on charts in Metrics Explorer.
Select any annotation marker (light gray arrow ) to open details about the release, including requestor, source
control branch, release pipeline, and environment.
You can modify the script, for example to create annotations for the past.
Next steps
Create work items
Automation with PowerShell
Add continuous monitoring to your release pipeline
10/20/2019 • 3 minutes to read • Edit Online
Azure Pipelines integrates with Azure Application Insights to allow continuous monitoring of your DevOps release
pipeline throughout the software development lifecycle.
With continuous monitoring, release pipelines can incorporate monitoring data from Application Insights and
other Azure resources. When the release pipeline detects an Application Insights alert, the pipeline can gate or roll
back the deployment until the alert is resolved. If all checks pass, deployments can proceed automatically from test
all the way to production, without the need for manual intervention.
Azure subscription Drop down and select the linked Azure subscription you
want to use.
App Service name Enter the name of your Azure App Service.
Resource Group name for Application Insights Drop down and select the resource group you want to
use.
Application Insights resource name Drop down and select the Application Insights resource
for the resource group you selected.
7. To save the pipeline with default alert rule settings, select Save at upper right in the Azure DevOps window.
Enter a descriptive comment, and then select OK.
5. Select OK, and then select Save at upper right in the Azure DevOps window. Enter a descriptive comment,
and then select OK.
Add deployment conditions
When you add deployment gates to your release pipeline, an alert that exceeds the thresholds you set prevents
unwanted release promotion. Once you resolve the alert, the deployment can proceed automatically.
To add deployment gates:
1. On the main pipeline page, under Stages, select the Pre-deployment conditions or Post-deployment
conditions symbol, depending on which stage needs a continuous monitoring gate.
5. Under Evaluation options, enter the values you want for settings like The time between re-evaluation
of gates and The timeout after which gates fail.
You can control who has read and update access to your data in Azure Application Insights, by using Role-based
access control in Microsoft Azure.
IMPORTANT
Assign access to users in the resource group or subscription to which your application resource belongs - not in the
resource itself. Assign the Application Insights component contributor role. This ensures uniform control of access to
web tests and alerts along with your application resource. Learn more.
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
The Add permissions view below is primarily specific to Application Insights resources, if you were viewing the
access control permissions from a higher level like resource groups, you would see additional non-Application
Insights-centric roles.
To view information on all Azure role-based access control built-in roles use the official reference content.
Select a role
Where applicable we link to the associated official reference documentation.
Application Insights Component contributor Can edit Application Insights resources, web tests and alerts.
ROLE IN THE RESOURCE GROUP
Application Insights Snapshot Debugger Gives the user permission to use Application Insights
Snapshot Debugger features. Note that this role is included in
neither the Owner nor Contributor roles.
Azure Service Deploy Release Management Contributor Contributor role for services deploying through Azure Service
Deploy.
Data Purger Special role for purging personal data. See our guidance for
personal data for more information.
Log Analytics Contributor Log Analytics Contributor can read all monitoring data and
edit monitoring settings. Editing monitoring settings includes
adding the VM extension to VMs; reading storage account
keys to be able to configure collection of logs from Azure
Storage; creating and configuring Automation accounts;
adding solutions; and configuring Azure diagnostics on all
Azure resources.
Log Analytics Reader Log Analytics Reader can view and search all monitoring data
as well as and view monitoring settings, including viewing the
configuration of Azure diagnostics on all Azure resources.
Monitoring Contributor Can read all monitoring data and update monitoring settings.
Resource Policy Contributor (Preview) Backfilled users from EA, with rights to create/modify resource
policy, create support ticket and read resource/hierarchy.
User Access Administrator Allows a user to manage access for other users to Azure
resources.
Website Contributor Lets you manage websites (not web plans), but not access to
them..
Query within the context of a specific Application Insights resource for owners and contributors
$resourceGroup = "RGNAME"
$resourceName = "AppInsightsName"
$resourceType = "microsoft.insights/components"
(Get-AzRoleAssignment -ResourceGroup $resourceGroup -ResourceType $resourceType -ResourceName $resourceName |
Where-Object {$_.RoleDefinitionName -in @('Owner', 'Contributor') } | Select -ExpandProperty SignInName |
Sort-Object -Unique) -Join ", "
Query within the context of a specific resource group for owners and contributors
$resourceGroup = "RGNAME"
(Get-AzRoleAssignment -ResourceGroup $resourceGroup | Where-Object {$_.RoleDefinitionName -in @('Owner',
'Contributor') } | Select -ExpandProperty SignInName | Sort-Object -Unique) -Join ", "
Geolocation and IP address handling
12/11/2019 • 6 minutes to read • Edit Online
This article explains how geolocation lookup and IP address handling occur in Application Insights along with how
to modify the default behavior.
Default behavior
By default IP addresses are temporarily collected, but not stored in Application Insights. The basic process is as
follows:
IP addresses are sent to Application Insights as part of telemetry data. Upon reaching the ingestion endpoint in
Azure, the IP address is used to perform a geolocation lookup using GeoLite2 from MaxMind. The results of this
lookup are used to populate the following fields client_City , client_StateOrProvince , client_CountryOrRegion . At
this point, the IP address is discarded and 0.0.0.0 is written to the client_IP field.
Browser telemetry: We temporarily collect the sender's IP address. IP address is calculated by the ingestion
endpoint.
Server telemetry: The Application Insights module temporarily collects the client IP address. It is not collected if
X-Forwarded-For is set.
This behavior is by design to help avoid unnecessary collection of personal data. Whenever possible, we
recommend avoiding the collection of personal data.
},
"kind": "web",
"properties": {
"Application_Type": "web",
"Flow_Type": "Redfield",
"Request_Source": "IbizaAIExtension",
// ...
"DisableIpMasking": true
}
}
Portal
If you only need to modify the behavior for a single Application Insights resource the easiest way to accomplish
this is via the Azure portal.
1. Go your Application Insights resource > Settings > Export Template
2. Select Deploy
3. Select Edit Template. (If your template has additional properties or resources that do not appear in this
example template, proceed with caution to ensure that all resources will accept the template deployment as
an incremental change/update.)
4. Make the following changes to the json for your resource and then click Save:
WARNING
If you experience an error that says: The resource group is in a location that is not supported by one or more
resources in the template. Please choose a different resource group. Temporarily select a different resource group
from the dropdown and then re-select your original resource group to resolve the error.
In this case nothing new is being purchased, we are just updating the config of the existing Application
Insights resource.
6. Once the deployment is complete new telemetry data will be recorded.
If you were to select and edit template again you would only see the default template and would not see
your newly added property and its associated value. If you aren't seeing IP address data and want to confirm
that "DisableIpMasking": true is set. Run the following PowerShell: (Replace Fabrikam-dev with the
appropriate resource and resource group name.)
# If you aren't using the cloud shell you will need to connect to your Azure account
# Connect-AzAccount
$AppInsights = Get-AzResource -Name 'Fabrikam-dev' -ResourceType 'microsoft.insights/components' -
ResourceGroupName 'Fabrikam-dev'
$AppInsights.Properties
A list of properties will be returned as a result. One of the properties should read DisableIpMasking: true . If
you run the PowerShell prior to deploying the new property with Azure Resource Manager, the property
will not exist.
Rest API
The Rest API payload to make the same modifications is as follows:
PATCH https://management.azure.com/subscriptions/<sub-id>/resourceGroups/<rg-
name>/providers/microsoft.insights/components/<resource-name>?api-version=2018-05-01-preview HTTP/1.1
Host: management.azure.com
Authorization: AUTH_TOKEN
Content-Type: application/json
Content-Length: 54
{
"location": "<resource location>",
"kind": "web",
"properties": {
"Application_Type": "web",
"DisableIpMasking": true
}
}
Telemetry initializer
If you need a more flexible alternative than DisableIpMasking to record all or part of IP addresses, you can use a
telemetry initializer to copy all or part the IP to a custom field.
ASP.NET / ASP.NET Core
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.DataContracts;
using Microsoft.ApplicationInsights.Extensibility;
namespace MyWebApp
{
public class CloneIPAddress : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
ISupportProperties propTelemetry = telemetry as ISupportProperties;
NOTE
If you are unable to access ISupportProperties , check and make sure you are running the latest stable release of the
Application Insights SDK. ISupportProperties are intended for high cardinality values, whereas GlobalProperties are
more appropriate for low cardinality values like region name, environment name, etc.
namespace MyWebApp
{
public class MvcApplication : System.Web.HttpApplication
{
protected void Application_Start()
{
//Enable your telemetry initializer:
TelemetryConfiguration.Active.TelemetryInitializers.Add(new CloneIPAddress());
}
}
}
using Microsoft.ApplicationInsights.Extensibility;
using CustomInitializer.Telemetry;
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<ITelemetryInitializer, CloneIPAddress>();
}
Node.js
appInsights.defaultClient.addTelemetryProcessor((envelope) => {
const baseData = envelope.data.baseData;
if (appInsights.Contracts.domainSupportsProperties(baseData)) {
const ipAddress = envelope.tags[appInsights.defaultClient.context.keys.locationIp];
if (ipAddress) {
baseData.properties["client-ip"] = ipAddress;
}
}
});
Client-side JavaScript
Unlike the server-side SDKs, the client-side Javascript SDK does not calculate IP address. By default IP address
calculation for client-side telemetry is performed at the ingestion endpoint in Azure upon telemetry arrival. This
means that if you were sending client-side data to a proxy and then forwarding to the ingestion endpoint, IP
address calculation may show the IP address of the proxy and not the client. If no proxy is used this should not be
an issue.
If you wish to calculate IP address directly on the client-side you would need to add your own custom logic to
perform this calculation and use the result to set the ai.location.ip tag. When ai.location.ip is set, IP address
calculation is not performed by the ingestion endpoint and the provided IP address is honored and used for
performing the geo lookup. In this scenario, IP address will still be zeroed out by default.
To retain the entire IP address calculated from your custom logic, you could use a telemetry initializer that would
copy the IP address data you provided in ai.location.ip to a separate custom field. But again unlike the server-
side SDKs, without relying on 3rd party libraries or your own custom client-side IP collection logic the client-side
SDK will not calculate the IP for you.
appInsights.addTelemetryInitializer((item) => {
const ipAddress = item.tags && item.tags["ai.location.ip"];
if (ipAddress) {
item.baseData.properties = {
...item.baseData.properties,
"client-ip": ipAddress
};
}
});
requests
| where timestamp > ago(1h)
| project appName, operation_Name, url, resultCode, client_IP, customDimensions.["client-ip"]
Newly collected IP addresses should appear in the customDimensions_client-ip column. The default client-ip
column will still have all 4 octets either zeroed out or only displaying the first three octets depending on how you
have configured IP address collection at the component level. If you are testing locally after implementing the
telemetry initializer and the value you see for customDimensions_client-ip is ::1 this is expected behavior. ::1
represents the loopback address in IPv6. It is equivalent to 127.0.01 in IPv4 and is the result you will see when
testing from localhost.
Next Steps
Learn more about personal data collection in Application Insights.
Learn more about how IP address collection in Application Insights works. (This is an older external blog
post written by one of our engineers. It predates the current default behavior where IP address is recorded
as 0.0.0.0 , but it goes into greater depth on the mechanics of the built-in
ClientIpHeaderTelemetryInitializer .)
Sampling in Application Insights
1/20/2020 • 21 minutes to read • Edit Online
Sampling is a feature in Azure Application Insights. It is the recommended way to reduce telemetry
traffic, data costs, and storage costs, while preserving a statistically correct analysis of application data.
Sampling also helps you avoid Application Insights throttling your telemetry. The sampling filter selects
items that are related, so that you can navigate between items when you are doing diagnostic
investigations.
When metric counts are presented in the portal, they are renormalized to take into account sampling.
Doing so minimizes any effect on the statistics.
Brief summary
There are three different types of sampling: adaptive sampling, fixed-rate sampling, and ingestion
sampling.
Adaptive sampling is enabled by default in all the latest versions of the Application Insights ASP.NET
and ASP.NET Core Software Development Kits (SDKs). It is also used by Azure Functions.
Fixed-rate sampling is available in recent versions of the Application Insights SDKs for ASP.NET,
ASP.NET Core, Java, and Python.
Ingestion sampling works on the Application Insights service endpoint. It only applies when no other
sampling is in effect. If the SDK samples your telemetry, ingestion sampling is disabled.
For web applications, if you log custom events and need to ensure that a set of events is retained or
discarded together, the events must have the same OperationId value.
If you write Analytics queries, you should take account of sampling. In particular, instead of simply
counting records, you should use summarize sum(itemCount) .
Some telemetry types, including performance metrics and custom metrics, are always kept regardless
of whether sampling is enabled or not.
The following table summarizes the sampling types available for each SDK and type of application:
Types of sampling
There are three different sampling methods:
Adaptive sampling automatically adjusts the volume of telemetry sent from the SDK in your
ASP.NET/ASP.NET Core app, and from Azure Functions. This is the default sampling when you
use the ASP.NET or ASP.NET Core SDK. Adaptive sampling is currently only available for
ASP.NET server-side telemetry, and for Azure Functions.
Fixed-rate sampling reduces the volume of telemetry sent from both your ASP.NET or ASP.NET
Core or Java server and from your users' browsers. You set the rate. The client and server will
synchronize their sampling so that, in Search, you can navigate between related page views and
requests.
Ingestion sampling happens at the Application Insights service endpoint. It discards some of the
telemetry that arrives from your app, at a sampling rate that you set. It doesn't reduce telemetry
traffic sent from your app, but helps you keep within your monthly quota. The main advantage of
ingestion sampling is that you can set the sampling rate without redeploying your app. Ingestion
sampling works uniformly for all servers and clients, but it does not apply when any other types of
sampling are in operation.
IMPORTANT
If adaptive or fixed rate sampling methods are in operation, ingestion sampling is disabled.
Adaptive sampling
Adaptive sampling affects the volume of telemetry sent from your web server app to the Application
Insights service endpoint.
TIP
Adaptive sampling is enabled by default when you use the ASP.NET SDK or the ASP.NET Core SDK, and is also
enabled by default for Azure Functions.
The volume is adjusted automatically to keep within a specified maximum rate of traffic, and is controlled
via the setting MaxTelemetryItemsPerSecond . If the application produces a low amount of telemetry, such as
when debugging or due to low usage, items won't be dropped by the sampling processor as long as
volume is below MaxTelemetryItemsPerSecond . As the volume of telemetry increases, the sampling rate is
adjusted so as to achieve the target volume. The adjustment is recalculated at regular intervals, and is
based on a moving average of the outgoing transmission rate.
To achieve the target volume, some of the generated telemetry is discarded. But like other types of
sampling, the algorithm retains related telemetry items. For example, when you're inspecting the
telemetry in Search, you'll be able to find the request related to a particular exception.
Metric counts such as request rate and exception rate are adjusted to compensate for the sampling rate,
so that they show approximately correct values in Metric Explorer.
Configuring adaptive sampling for ASP.NET applications
NOTE
This section applies to ASP.NET applications, not to ASP.NET Core applications. Learn about configuring adaptive
sampling for ASP.NET Core applications later in this document.
<MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
The target rate that the adaptive algorithm aims for on each server host. If your web app runs on
many hosts, reduce this value so as to remain within your target rate of traffic at the Application
Insights portal.
<EvaluationInterval>00:00:15</EvaluationInterval>
The interval at which the current rate of telemetry is reevaluated. Evaluation is performed as a
moving average. You might want to shorten this interval if your telemetry is liable to sudden
bursts.
<SamplingPercentageDecreaseTimeout>00:02:00</SamplingPercentageDecreaseTimeout>
When sampling percentage value changes, how soon after are we allowed to lower the sampling
percentage again to capture less data?
<SamplingPercentageIncreaseTimeout>00:15:00</SamplingPercentageIncreaseTimeout>
When sampling percentage value changes, how soon after are we allowed to increase the
sampling percentage again to capture more data?
<MinSamplingPercentage>0.1</MinSamplingPercentage>
As sampling percentage varies, what is the minimum value we're allowed to set?
<MaxSamplingPercentage>100.0</MaxSamplingPercentage>
As sampling percentage varies, what is the maximum value we're allowed to set?
<MovingAverageRatio>0.25</MovingAverageRatio>
In the calculation of the moving average, this specifies the weight that should be assigned to the
most recent value. Use a value equal to or less than 1. Smaller values make the algorithm less
reactive to sudden changes.
<InitialSamplingPercentage>100</InitialSamplingPercentage>
The amount of telemetry to sample when the app has just started. Don't reduce this value while
you're debugging.
<ExcludedTypes>Trace;Exception</ExcludedTypes>
A semi-colon delimited list of types that you do not want to be subject to sampling. Recognized
types are: Dependency , Event , Exception , PageView , Request , Trace . All telemetry of the
specified types is transmitted; the types that are not specified will be sampled.
<IncludedTypes>Request;Dependency</IncludedTypes>
A semi-colon delimited list of types that you do want to subject to sampling. Recognized types are:
Dependency , Event , Exception , PageView , Request , Trace . The specified types will be sampled;
all telemetry of the other types will always be transmitted.
To switch off adaptive sampling, remove the AdaptiveSamplingTelemetryProcessor node(s) from
ApplicationInsights.config .
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights.WindowsServer.Channel.Implementation;
using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;
// ...
var builder =
TelemetryConfiguration.Active.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
// For older versions of the Application Insights SDK, use the following line instead:
// var builder = TelemetryConfiguration.Active.TelemetryProcessorChainBuilder;
builder.Build();
You can also adjust the sampling rate for each telemetry type individually, or can even exclude certain
types from being sampled at all:
// The following configures adaptive sampling with 5 items per second, and also excludes Dependency
telemetry from being subjected to sampling.
builder.UseAdaptiveSampling(maxTelemetryItemsPerSecond:5, excludedTypes: "Dependency");
//...
}
The above code will disable adaptive sampling. Follow the steps below to add sampling with more
customization options.
Configure sampling settings
Use extension methods of TelemetryProcessorChainBuilder as shown below to customize sampling
behavior.
IMPORTANT
If you use this method to configure sampling, please make sure to set the aiOptions.EnableAdaptiveSampling
property to false when calling AddApplicationInsightsTelemetry() .
// Alternately, the following configures adaptive sampling with 5 items per second, and also
excludes DependencyTelemetry from being subject to sampling.
// builder.UseAdaptiveSampling(maxTelemetryItemsPerSecond:5, excludedTypes: "Dependency");
builder.Build();
// ...
}
Fixed-rate sampling
Fixed-rate sampling reduces the traffic sent from your web server and web browsers. Unlike adaptive
sampling, it reduces telemetry at a fixed rate decided by you. Fixed-rate sampling is available for
ASP.NET, ASP.NET Core, Java and Python applications.
Like other sampling techniques, this also retains related items. It also synchronizes the client and server
sampling so that related items are retained - for example, when you look at a page view in Search, you
can find its related server requests.
In Metrics Explorer, rates such as request and exception counts are multiplied by a factor to compensate
for the sampling rate, so that they are approximately correct.
Configuring fixed-rate sampling for ASP.NET applications
1. Disable adaptive sampling: In ApplicationInsights.config , remove or comment out the
AdaptiveSamplingTelemetryProcessor node.
<TelemetryProcessors>
<!-- Disabled adaptive sampling:
<Add
Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryPro
cessor, Microsoft.AI.ServerTelemetryChannel">
<MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
</Add>
-->
<TelemetryProcessors>
<Add
Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.SamplingTelemetryProcessor,
Microsoft.AI.ServerTelemetryChannel">
<!-- Set a percentage close to 100/N where N is an integer. -->
<!-- E.g. 50 (=100/2), 33.33 (=100/3), 25 (=100/4), 20, 1 (=100/100), 0.1 (=100/1000) -
->
<SamplingPercentage>10</SamplingPercentage>
</Add>
</TelemetryProcessors>
Alternatively, instead of setting the sampling parameter in the ApplicationInsights.config file, you
can programmatically set these values:
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;
// ...
var builder =
TelemetryConfiguration.Active.DefaultTelemetrySink.TelemetryProcessorChainBuilder;
// For older versions of the Application Insights SDK, use the following line instead:
// var builder = TelemetryConfiguration.Active.TelemetryProcessorChainBuilder;
builder.UseSampling(10.0); // percentage
builder.Build();
//...
}
2. Enable the fixed-rate sampling module. Changes can be made in the Configure method as
shown in the below snippet:
builder.Build();
// ...
}
<TelemetryProcessors>
<BuiltInProcessors>
<Processor type="FixedRateSamplingTelemetryProcessor">
<!-- Set a percentage close to 100/N where N is an integer. -->
<!-- E.g. 50 (=100/2), 33.33 (=100/3), 25 (=100/4), 20, 1 (=100/100), 0.1
(=100/1000) -->
<Add name="SamplingPercentage" value="50" />
</Processor>
</BuiltInProcessors>
</TelemetryProcessors>
3. You can include or exclude specific types of telemetry from sampling using the following tags
inside the Processor tag's FixedRateSamplingTelemetryProcessor :
<ExcludedTypes>
<ExcludedType>Request</ExcludedType>
</ExcludedTypes>
<IncludedTypes>
<IncludedType>Exception</IncludedType>
</IncludedTypes>
The telemetry types that can be included or excluded from sampling are: Dependency , Event , Exception ,
PageView , Request , and Trace .
NOTE
For the sampling percentage, choose a percentage that is close to 100/N where N is an integer. Currently sampling
doesn't support other values.
NOTE
Fixed-rate sampling is only available using the trace exporter. This means incoming and outgoing requests are the
only types of telemetry where sampling can be configured.
2. You may specify a sampler as part of your Tracer configuration. If no explicit sampler is provided,
the ProbabilitySampler will be used by default. The ProbabilitySampler would use a rate of 1/10000
by default, meaning one out of every 10000 requests will be sent to Application Insights. If you want
to specify a sampling rate, see below.
To specify the sampling rate, make sure your Tracer specifies a sampler with a sampling rate between
0.0 and 1.0 inclusive. A sampling rate of 1.0 represents 100%, meaning all of your requests will be sent as
telemetry to Application Insights.
tracer = Tracer(
exporter=AzureExporter(
instrumentation_key='00000000-0000-0000-0000-000000000000',
),
sampler=ProbabilitySampler(1.0),
)
TIP
In ASP.NET apps with JavaScript included, the snippet typically goes in _Layout.cshtml .
instrumentationKey: ...
});
window.appInsights = appInsights;
appInsights.trackPageView();
</script>
For the sampling percentage, choose a percentage that is close to 100/N where N is an integer. Currently
sampling doesn't support other values.
Coordinating server-side and client-side sampling
The client-side JavaScript SDK participates in fixed-rate sampling in conjunction with the server-side
SDK. The instrumented pages will only send client-side telemetry from the same user for which the
server-side SDK made its decision to include in the sampling. This logic is designed to maintain the
integrity of user sessions across client- and server-side applications. As a result, from any particular
telemetry item in Application Insights you can find all other telemetry items for this user or session and
in Search, you can navigate between related page views and requests.
If your client and server-side telemetry don't show coordinated samples:
Verify that you enabled sampling both on the server and client.
Check that you set the same sampling percentage in both the client and server.
Make sure that the SDK version is 2.0 or above.
Ingestion sampling
Ingestion sampling operates at the point where the telemetry from your web server, browsers, and
devices reaches the Application Insights service endpoint. Although it doesn't reduce the telemetry traffic
sent from your app, it does reduce the amount processed and retained (and charged for) by Application
Insights.
Use this type of sampling if your app often goes over its monthly quota and you don't have the option of
using either of the SDK-based types of sampling.
Set the sampling rate in the Usage and estimated costs page:
Like other types of sampling, the algorithm retains related telemetry items. For example, when you're
inspecting the telemetry in Search, you'll be able to find the request related to a particular exception.
Metric counts such as request rate and exception rate are correctly retained.
Data points that are discarded by sampling are not available in any Application Insights feature such as
Continuous Export.
Ingestion sampling doesn't operate while adaptive or fixed-rate sampling is in operation. Adaptive
sampling is enabled by default when the ASP.NET SDK or the ASP.NET Core SDK is being used, or when
Application Insights is enabled in Azure App Service or by using Status Monitor. When telemetry is
received by the Application Insights service endpoint, it examines the telemetry and if the sampling rate is
reported to be less than 100% (which indicates that telemetry is being sampled) then the ingestion
sampling rate that you set is ignored.
WARNING
The value shown on the portal tile indicates the value that you set for ingestion sampling. It doesn't represent the
actual sampling rate if any sort of SDK sampling (adaptive or fixed-rate sampling) is in operation.
If you see that RetainedPercentage for any type is less than 100, then that type of telemetry is being
sampled.
IMPORTANT
Application Insights does not sample session, metrics (including custom metrics), or performance counter
telemetry types in any of the sampling techniques. These types are always excluded from sampling as a reduction
in precision can be highly undesirable for these telemetry types.
<TelemetryProcessors>
<Add
Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryPro
cessor, Microsoft.AI.ServerTelemetryChannel">
<MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
<ExcludedTypes>Event</ExcludedTypes>
</Add>
<Add
Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryPro
cessor, Microsoft.AI.ServerTelemetryChannel">
<MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
<IncludedTypes>Event</IncludedTypes>
</Add>
</TelemetryProcessors>
Next steps
Filtering can provide more strict control of what your SDK sends.
Read the Developer Network article Optimize Telemetry with Application Insights.
Telemetry channels in Application Insights
12/17/2019 • 8 minutes to read • Edit Online
Telemetry channels are an integral part of the Azure Application Insights SDKs. They manage buffering and
transmission of telemetry to the Application Insights service. The .NET and .NET Core versions of the SDKs have
two built-in telemetry channels: InMemoryChannel and ServerTelemetryChannel . This article describes each channel
in detail, including how to customize channel behavior.
The Send(ITelemetry item) method of a telemetry channel is called after all telemetry initializers and telemetry
processors are called. So, any items dropped by a telemetry processor won't reach the channel. Send() doesn't
typically send the items to the back end instantly. Typically, it buffers them in memory and sends them in batches,
for efficient transmission.
Live Metrics Stream also has a custom channel that powers the live streaming of telemetry. This channel is
independent of the regular telemetry channel, and this document doesn't apply to it.
<TelemetrySinks>
<Add Name="default">
<TelemetryProcessors>
<!-- Telemetry processors omitted for brevity -->
</TelemetryProcessors>
<TelemetryChannel
Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel,
Microsoft.AI.ServerTelemetryChannel">
<StorageFolder>d:\temp\applicationinsights</StorageFolder>
</TelemetryChannel>
</Add>
</TelemetrySinks>
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;
protected void Application_Start()
{
var serverTelemetryChannel = new ServerTelemetryChannel();
serverTelemetryChannel.StorageFolder = @"d:\temp\applicationinsights";
serverTelemetryChannel.Initialize(TelemetryConfiguration.Active);
TelemetryConfiguration.Active.TelemetryChannel = serverTelemetryChannel;
}
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel;
services.AddApplicationInsightsTelemetry();
}
IMPORTANT
Configuring the channel by using TelemetryConfiguration.Active is not recommended for ASP.NET Core applications.
Open-source SDK
Like every SDK for Application Insights, channels are open source. Read and contribute to the code, or report
problems, at the official GitHub repo.
Next steps
Sampling
SDK Troubleshooting
Filtering and preprocessing telemetry in the
Application Insights SDK
11/8/2019 • 9 minutes to read • Edit Online
You can write and configure plug-ins for the Application Insights SDK to customize how telemetry can be
enriched and processed before it's sent to the Application Insights service.
Sampling reduces the volume of telemetry without affecting your statistics. It keeps together related data
points so that you can navigate between them when diagnosing a problem. In the portal, the total counts
are multiplied to compensate for the sampling.
Filtering with Telemetry Processors lets you filter out telemetry in the SDK before it is sent to the server. For
example, you could reduce the volume of telemetry by excluding requests from robots. Filtering is a more
basic approach to reducing traffic than sampling. It allows you more control over what is transmitted, but
you have to be aware that it affects your statistics - for example, if you filter out all successful requests.
Telemetry Initializers add or modify properties to any telemetry sent from your app, including telemetry
from the standard modules. For example, you could add calculated values; or version numbers by which to
filter the data in the portal.
The SDK API is used to send custom events and metrics.
Before you start:
Install the appropriate SDK for your application: ASP.NET, ASP.NET Core, Non HTTP/Worker for
.NET/.NET Core, Java or JavaScript
Filtering
This technique gives you direct control over what is included or excluded from the telemetry stream. Filtering
can be used to drop telemetry items from being sent to Application Insights. You can use it in conjunction with
Sampling, or separately.
To filter telemetry, you write a telemetry processor and register it with the TelemetryConfiguration . All
telemetry goes through your processor, and you can choose to drop it from the stream or give it to the next
processor in the chain. This includes telemetry from the standard modules such as the HTTP request collector
and the dependency collector, and telemetry you have tracked yourself. You can, for example, filter out
telemetry about requests from robots, or successful dependency calls.
WARNING
Filtering the telemetry sent from the SDK using processors can skew the statistics that you see in the portal, and make it
difficult to follow related items.
Instead, consider using sampling.
this.Next.Process(item);
}
<TelemetryProcessors>
<Add Type="WebApplication9.SuccessfulDependencyFilter, WebApplication9">
<!-- Set public property -->
<MyParamFromConfigFile>2-beta</MyParamFromConfigFile>
</Add>
</TelemetryProcessors>
You can pass string values from the .config file by providing public named properties in your class.
WARNING
Take care to match the type name and any property names in the .config file to the class and property names in the
code. If the .config file references a non-existent type or property, the SDK may silently fail to send any telemetry.
Alternatively, you can initialize the filter in code. In a suitable initialization class - for example AppStart in
Global.asax.cs - insert your processor into the chain:
builder.Build();
TelemetryClients created after this point will use your processors.
ASP.NET Core/ Worker Service apps
NOTE
Adding processor using ApplicationInsights.config or using TelemetryConfiguration.Active is not valid for
ASP.NET Core applications or if you are using Microsoft.ApplicationInsights.WorkerService SDK.
For apps written using ASP.NET Core or WorkerService, adding a new TelemetryProcessor is done by using
AddApplicationInsightsTelemetryProcessor extension method on IServiceCollection , as shown below. This
method is called in ConfigureServices method of your Startup.cs class.
Example filters
Synthetic requests
Filter out bots and web tests. Although Metrics Explorer gives you the option to filter out synthetic sources, this
option reduces traffic and ingestion size by filtering them at the SDK itself.
Failed authentication
Filter out requests with a "401" response.
return true;
};
appInsights.addTelemetryInitializer(filteringFunction);
namespace MvcWebRole.Telemetry
{
/*
* Custom TelemetryInitializer that overrides the default SDK
* behavior of treating response codes >= 400 as failed requests
*
*/
public class MyTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
var requestTelemetry = telemetry as RequestTelemetry;
// Is this a TrackRequest() ?
if (requestTelemetry == null) return;
int code;
bool parsed = Int32.TryParse(requestTelemetry.ResponseCode, out code);
if (!parsed) return;
if (code >= 400 && code < 500)
{
// If we set the Success property, the SDK won't change it:
requestTelemetry.Success = true;
<ApplicationInsights>
<TelemetryInitializers>
<!-- Fully qualified type name, assembly name: -->
<Add Type="MvcWebRole.Telemetry.MyTelemetryInitializer, MvcWebRole"/>
...
</TelemetryInitializers>
</ApplicationInsights>
Alternatively, you can instantiate the initializer in code, for example in Global.aspx.cs:
For apps written using ASP.NET Core or WorkerService, adding a new TelemetryInitializer is done by
adding it to the Dependency Injection container, as shown below. This is done in Startup.ConfigureServices
method.
using Microsoft.ApplicationInsights.Extensibility;
using CustomInitializer.Telemetry;
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton<ITelemetryInitializer, MyTelemetryInitializer>();
}
<Add type="mypackage.MyConfigurableContextInitializer">
<Param name="some_config_property" value="some_value" />
</Add>
appInsights.queue.push(function () {
appInsights.context.addTelemetryInitializer(function (envelope) {
var telemetryItem = envelope.data.baseData;
appInsights.trackPageView();
</script>
For a summary of the non-custom properties available on the telemetryItem, see Application Insights Export
Data Model.
You can add as many initializers as you like, and they are called in the order they are added.
Example TelemetryInitializers
Add custom property
The following sample initializer adds a custom property to every tracked telemetry.
Troubleshooting ApplicationInsights.config
Confirm that the fully qualified type name and assembly name are correct.
Confirm that the applicationinsights.config file is in your output directory and contains any recent changes.
Reference docs
API Overview
ASP.NET reference
SDK Code
ASP.NET Core SDK
ASP.NET SDK
JavaScript SDK
Next steps
Search events and logs
Sampling
Troubleshooting
Telemetry correlation in Application Insights
12/9/2019 • 12 minutes to read • Edit Online
In the world of microservices, every logical operation requires work to be done in various components of the
service. You can monitor each of these components separately by using Application Insights. Application Insights
supports distributed telemetry correlation, which you use to detect which component is responsible for failures or
performance degradation.
This article explains the data model used by Application Insights to correlate telemetry sent by multiple
components. It covers context-propagation techniques and protocols. It also covers the implementation of
correlation tactics on different languages and platforms.
In a microservices environment, traces from components can go to different storage items. Every component can
have its own instrumentation key in Application Insights. To get telemetry for the logical operation, Application
Insights queries data from every storage item. When the number of storage items is large, you'll need a hint about
where to look next. The Application Insights data model defines two fields to solve this problem: request.source
and dependency.target . The first field identifies the component that initiated the dependency request. The second
field identifies which component returned the response of the dependency call.
Example
Let's look at an example. An application called Stock Prices shows the current market price of a stock by using an
external API called Stock. The Stock Prices application has a page called Stock page that the client web browser
opens by using GET /Home/Stock . The application queries the Stock API by using the HTTP call
GET /api/stock/value .
When the call GET /api/stock/value is made to an external service, you need to know the identity of that server so
you can set the dependency.target field appropriately. When the external service doesn't support monitoring,
target is set to the host name of the service (for example, stock-prices-api.com ). But if the service identifies itself
by returning a predefined HTTP header, target contains the service identity that allows Application Insights to
build a distributed trace by querying telemetry from that service.
Correlation headers
Application Insights is transitioning to W3C Trace-Context, which defines:
traceparent : Carries the globally unique operation ID and unique identifier of the call.
tracestate : Carries system -specific tracing context.
The latest version of the Application Insights SDK supports the Trace-Context protocol, but you might need to opt
in to it. (Backward compatibility with the previous correlation protocol supported by the Application Insights SDK
will be maintained.)
The correlation HTTP protocol, also called Request-Id, is being deprecated. This protocol defines two headers:
Request-Id : Carries the globally unique ID of the call.
Correlation-Context : Carries the name-value pairs collection of the distributed trace properties.
Application Insights also defines the extension for the correlation HTTP protocol. It uses Request-Context name-
value pairs to propagate the collection of properties used by the immediate caller or callee. The Application Insights
SDK uses this header to set the dependency.target and request.source fields.
Enable W3C distributed tracing support for classic ASP.NET apps
NOTE
Starting with Microsoft.ApplicationInsights.Web and Microsoft.ApplicationInsights.DependencyCollector , no
configuration is needed.
W3C Trace-Context support is implemented in a backward-compatible way. Correlation is expected to work with
applications that are instrumented with previous versions of the SDK (without W3C support).
If you want to keep using the legacy Request-Id protocol, you can disable Trace-Context by using this
configuration:
Activity.DefaultIdFormat = ActivityIdFormat.Hierarchical;
Activity.ForceDefaultIdFormat = true;
If you run an older version of the SDK, we recommend that you update it or apply the following configuration to
enable Trace-Context. This feature is available in the Microsoft.ApplicationInsights.Web and
Microsoft.ApplicationInsights.DependencyCollector packages, starting with version 2.8.0 -beta1. It's disabled by
default. To enable it, make these changes to ApplicationInsights.config :
Under RequestTrackingTelemetryModule , add the EnableW3CHeadersExtraction element and set its value to true .
Under DependencyTrackingTelemetryModule , add the EnableW3CHeadersInjection element and set its value to
true .
Add W3COperationCorrelationTelemetryInitializer under TelemetryInitializers . It will look similar to this
example:
<TelemetryInitializers>
<Add Type="Microsoft.ApplicationInsights.Extensibility.W3C.W3COperationCorrelationTelemetryInitializer,
Microsoft.ApplicationInsights"/>
...
</TelemetryInitializers>
NOTE
Starting with Microsoft.ApplicationInsights.AspNetCore version 2.8.0, no configuration is needed.
W3C Trace-Context support is implemented in a backward-compatible way. Correlation is expected to work with
applications that are instrumented with previous versions of the SDK (without W3C support).
If you want to keep using the legacy Request-Id protocol, you can disable Trace-Context by using this
configuration:
Activity.DefaultIdFormat = ActivityIdFormat.Hierarchical;
Activity.ForceDefaultIdFormat = true;
If you run an older version of the SDK, we recommend that you update it or apply the following configuration to
enable Trace-Context.
This feature is in version 2.5.0-beta1 and in
Microsoft.ApplicationInsights.AspNetCore
Microsoft.ApplicationInsights.DependencyCollector version 2.8.0 -beta1. It's disabled by default. To enable it, set
ApplicationInsightsServiceOptions.RequestCollectionOptions.EnableW3CDistributedTracing to true :
<Instrumentation>
<BuiltIn enabled="true">
<HTTP enabled="true" W3C="true" enableW3CBackCompat="true"/>
</BuiltIn>
</Instrumentation>
NOTE
Backward compatibility mode is enabled by default, and the enableW3CBackCompat parameter is optional. Use it only
when you want to turn backward compatibility off.
Ideally, you would turn this off when all your services have been updated to newer versions of SDKs that support the
W3C protocol. We highly recommend that you move to these newer SDKs as soon as possible.
IMPORTANT
Make sure the incoming and outgoing configurations are exactly the same.
Operation_Id TraceId
app = Flask(__name__)
middleware = FlaskMiddleware(
app,
exporter=AzureExporter(),
sampler=ProbabilitySampler(rate=1.0),
)
@app.route('/')
def hello():
return 'Hello World!'
if __name__ == '__main__':
app.run(host='localhost', port=8080, threaded=True)
This code runs a sample Flask application on your local machine, listening to port 8080 . To correlate trace context,
you send a request to the endpoint. In this example, you can use a curl command:
By looking at the Trace-Context header format, you can derive the following information:
version : 00
trace-id : 4bf92f3577b34da6a3ce929d0e0e4736
parent-id/span-id : 00f067aa0ba902b7
trace-flags : 01
If you look at the request entry that was sent to Azure Monitor, you can see fields populated with the trace header
information. You can find this data under Logs (Analytics) in the Azure Monitor Application Insights resource.
The id field is in the format <trace-id>.<span-id> , where the trace-id is taken from the trace header that was
passed in the request and the span-id is a generated 8-byte array for this span.
The operation_ParentId field is in the format <trace-id>.<parent-id> , where both the trace-id and the
parent-id are taken from the trace header that was passed in the request.
Log correlation
OpenCensus Python enables you to correlate logs by adding a trace ID, a span ID, and a sampling flag to log
records. You add these attributes by installing OpenCensus logging integration. The following attributes will be
added to Python LogRecord objects: traceId , spanId , and traceSampled . Note that this takes effect only for
loggers that are created after the integration.
Here's a sample application that demonstrates this:
import logging
config_integration.trace_integrations(['logging'])
logging.basicConfig(format='%(asctime)s traceId=%(traceId)s spanId=%(spanId)s %(message)s')
tracer = Tracer(sampler=AlwaysOnSampler())
logger = logging.getLogger(__name__)
logger.warning('Before the span')
with tracer.span(name='hello'):
logger.warning('In the span')
logger.warning('After the span')
Notice that there's a spanId present for the log message that's within the span. This is the same spanId that
belongs to the span named hello .
You can export the log data by using AzureLogHandler . For more information, see this article.
NOTE
Only calls made via Apache HttpClient are supported for the correlation feature. Both Spring RestTemplate and Feign can be
used with Apache HttpClient under the hood.
Currently, automatic context propagation across messaging technologies (like Kafka, RabbitMQ, and Azure Service
Bus) isn't supported. It is possible to code such scenarios manually by using the trackDependency and
trackRequest methods. In these methods, a dependency telemetry represents a message being enqueued by a
producer. The request represents a message being processed by a consumer. In this case, both operation_id and
operation_parentId should be propagated in the message's properties.
Role name
You might want to customize the way component names are displayed in the Application Map. To do so, you can
manually set the cloud_RoleName by taking one of the following actions:
With Application Insights Java SDK 2.5.0 and later, you can specify the cloud_RoleName by adding
<RoleName> to your ApplicationInsights.xml file:
If you use Spring Boot with the Application Insights Spring Boot Starter, you just need to set your custom
name for the application in the application.properties file:
spring.application.name=<name-of-app>
The Spring Boot Starter automatically assigns cloudRoleName to the value you enter for the
spring.application.name property.
Next steps
Write custom telemetry.
For advanced correlation scenarios in ASP.NET Core and ASP.NET, see Track custom operations.
Learn more about setting cloud_RoleName for other SDKs.
Onboard all components of your microservice on Application Insights. Check out the supported platforms.
See the data model for Application Insights types.
Learn how to extend and filter telemetry.
Review the Application Insights config reference.
Track custom operations with Application Insights
.NET SDK
12/9/2019 • 14 minutes to read • Edit Online
Azure Application Insights SDKs automatically track incoming HTTP requests and calls to dependent services,
such as HTTP requests and SQL queries. Tracking and correlation of requests and dependencies give you
visibility into the whole application's responsiveness and reliability across all microservices that combine this
application.
There is a class of application patterns that can't be supported generically. Proper monitoring of such patterns
requires manual code instrumentation. This article covers a few patterns that might require manual
instrumentation, such as custom queue processing and running long-running background tasks.
This document provides guidance on how to track custom operations with the Application Insights SDK. This
documentation is relevant for:
Application Insights for .NET (also known as Base SDK) version 2.4+.
Application Insights for web applications (running ASP.NET) version 2.4+.
Application Insights for ASP.NET Core version 2.1+.
Overview
An operation is a logical piece of work run by an application. It has a name, start time, duration, result, and a
context of execution like user name, properties, and result. If operation A was initiated by operation B, then
operation B is set as a parent for A. An operation can have only one parent, but it can have many child operations.
For more information on operations and telemetry correlation, see Azure Application Insights telemetry
correlation.
In the Application Insights .NET SDK, the operation is described by the abstract class OperationTelemetry and its
descendants RequestTelemetry and DependencyTelemetry.
// If there is a Request-Id received from the upstream service, set the telemetry context
accordingly.
if (context.Request.Headers.ContainsKey("Request-Id"))
{
var requestId = context.Request.Headers.Get("Request-Id");
// Get the operation ID from the Request-Id (if you follow the HTTP Protocol for Correlation).
requestTelemetry.Context.Operation.Id = GetOperationId(requestId);
requestTelemetry.Context.Operation.ParentId = requestId;
}
The HTTP Protocol for Correlation also declares the Correlation-Context header. However, it's omitted here for
simplicity.
Queue instrumentation
While there are W3C Trace Context and HTTP Protocol for Correlation to pass correlation details with HTTP
request, every queue protocol has to define how the same details are passed along the queue message. Some
queue protocols (such as AMQP ) allow passing additional metadata and some others (such Azure Storage
Queue) require the context to be encoded into the message payload.
NOTE
Cross-component tracing is not supported for queues yet With HTTP, if your producer and consumer send
telemetry to different Application Insights resources, Transaction Diagnostics Experience and Application Map show
transactions and map end-to-end. In case of queues this is not supported yet.
try
{
await queue.SendAsync(message);
Process
public async Task Process(BrokeredMessage message)
{
// After the message is taken from the queue, create RequestTelemetry to track its processing.
// It might also make sense to get the name from the message.
RequestTelemetry requestTelemetry = new RequestTelemetry { Name = "process " + queueName };
try
{
await ProcessMessage();
}
catch (Exception e)
{
telemetryClient.TrackException(e);
throw;
}
finally
{
// Update status code and success as appropriate.
telemetryClient.StopOperation(operation);
}
}
// MessagePayload represents your custom message and also serializes correlation identifiers into
payload.
// For example, if you choose to pass payload serialized to JSON, it might look like
// {'RootId' : 'some-id', 'ParentId' : '|some-id.1.2.3.', 'message' : 'your message to process'}
var jsonPayload = JsonConvert.SerializeObject(new MessagePayload
{
RootId = operation.Telemetry.Context.Operation.Id,
ParentId = operation.Telemetry.Id,
Payload = message
});
// Add operation.Telemetry.Id to the OperationContext to correlate Storage logs and Application Insights
telemetry.
OperationContext context = new OperationContext { ClientRequestID = operation.Telemetry.Id};
try
{
await queue.AddMessageAsync(queueMessage, null, null, new QueueRequestOptions(), context);
}
catch (StorageException e)
{
operation.Telemetry.Properties.Add("AzureServiceRequestID", e.RequestInformation.ServiceRequestID);
operation.Telemetry.Success = false;
operation.Telemetry.ResultCode = e.RequestInformation.HttpStatusCode.ToString();
telemetryClient.TrackException(e);
}
finally
{
// Update status code and success as appropriate.
telemetryClient.StopOperation(operation);
}
}
To reduce the amount of telemetry your application reports or if you don't want to track the Enqueue operation
for other reasons, use the Activity API directly:
Create (and start) a new Activity instead of starting the Application Insights operation. You do not need to
assign any properties on it except the operation name.
Serialize yourActivity.Id into the message payload instead of operation.Telemetry.Id . You can also use
Activity.Current.Id .
Dequeue
Similarly to Enqueue , an actual HTTP request to the Storage queue is automatically tracked by Application
Insights. However, the Enqueue operation presumably happens in the parent context, such as an incoming
request context. Application Insights SDKs automatically correlate such an operation (and its HTTP part) with the
parent request and other telemetry reported in the same scope.
The Dequeue operation is tricky. The Application Insights SDK automatically tracks HTTP requests. However, it
doesn't know the correlation context until the message is parsed. It's not possible to correlate the HTTP request
to get the message with the rest of the telemetry especially when more than one message is received.
public async Task<MessagePayload> Dequeue(CloudQueue queue)
{
var operation = telemetryClient.StartOperation<DependencyTelemetry>("dequeue " + queue.Name);
operation.Telemetry.Type = "Azure queue";
operation.Telemetry.Data = "Dequeue " + queue.Name;
try
{
var message = await queue.GetMessageAsync();
}
catch (StorageException e)
{
operation.telemetry.Properties.Add("AzureServiceRequestID", e.RequestInformation.ServiceRequestID);
operation.telemetry.Success = false;
operation.telemetry.ResultCode = e.RequestInformation.HttpStatusCode.ToString();
telemetryClient.TrackException(e);
}
finally
{
// Update status code and success as appropriate.
telemetryClient.StopOperation(operation);
}
return null;
}
Process
In the following example, an incoming message is tracked in a manner similarly to incoming HTTP request:
// It might also make sense to get the name from the message.
requestTelemetry.Context.Operation.Id = message.RootId;
requestTelemetry.Context.Operation.ParentId = message.ParentId;
try
{
await ProcessMessage();
}
catch (Exception e)
{
telemetryClient.TrackException(e);
throw;
}
finally
{
// Update status code and success as appropriate.
telemetryClient.StopOperation(operation);
}
}
Similarly, other queue operations can be instrumented. A peek operation should be instrumented in a similar way
as a dequeue operation. Instrumenting queue management operations isn't necessary. Application Insights tracks
operations such as HTTP, and in most cases, it's enough.
When you instrument message deletion, make sure you set the operation (correlation) identifiers. Alternatively,
you can use the Activity API. Then you don't need to set operation identifiers on the telemetry items because
Application Insights SDK does it for you:
Create a new Activity after you've got an item from the queue.
Use Activity.SetParentId(message.ParentId) to correlate consumer and producer logs.
Start the Activity .
Track dequeue, process, and delete operations by using Start/StopOperation helpers. Do it from the same
asynchronous control flow (execution context). In this way, they're correlated properly.
Stop the Activity .
Use Start/StopOperation , or call Track telemetry manually.
Dependency Types
Application Insights uses dependency type to cusomize UI experiences. For queues it recognizes following types
of DependencyTelemetry that improve Transaction diagnostics experience:
Azure queue for Azure Storage Queues
Azure Event Hubs for Azure Event Hubs
Azure Service Bus for Azure Service Bus
Batch processing
With some queues, you can dequeue multiple messages with one request. Processing such messages is
presumably independent and belongs to the different logical operations. It's not possible to correlate the
Dequeue operation to a particular message being processed.
Each message should be processed in its own asynchronous control flow. For more information, see the
Outgoing dependencies tracking section.
In this example, telemetryClient.StartOperation creates DependencyTelemetry and fills the correlation context.
Let's say you have a parent operation that was created by incoming requests that scheduled the operation. As
long as BackgroundTask starts in the same asynchronous control flow as an incoming request, it's correlated with
that parent operation. BackgroundTask and all nested telemetry items are automatically correlated with the
request that caused it, even after the request ends.
When the task starts from the background thread that doesn't have any operation ( Activity ) associated with it,
BackgroundTask doesn't have any parent. However, it can have nested operations. All telemetry items reported
from the task are correlated to the DependencyTelemetry created in BackgroundTask .
Disposing operation causes operation to be stopped, so you may do it instead of calling StopOperation .
Warning: in some cases unhanded exception may prevent finally to be called so operations may not be
tracked.
Parallel operations processing and tracking
StopOperation only stops the operation that was started. If the current running operation doesn't match the one
you want to stop, StopOperation does nothing. This situation might happen if you start multiple operations in
parallel in the same execution context:
var firstOperation = telemetryClient.StartOperation<DependencyTelemetry>("task 1");
var firstTask = RunMyTaskAsync();
await firstTask;
// FAILURE!!! This will do nothing and will not report telemetry for the first operation
// as currently secondOperation is active.
telemetryClient.StopOperation(firstOperation);
await secondTask;
Make sure you always call StartOperation and process operation in the same async method to isolate
operations running in parallel. If operation is synchronous (or not async), wrap process and track with Task.Run :
Next steps
Learn the basics of telemetry correlation in Application Insights.
Check out how correlated data powers Transaction Diagnostics Experience and Application Map.
See the data model for Application Insights types and data model.
Report custom events and metrics to Application Insights.
Check out standard configuration for context properties collection.
Check the System.Diagnostics.Activity User Guide to see how we correlate telemetry.
Automate custom reports with Azure Application
Insights data
10/20/2019 • 4 minutes to read • Edit Online
Periodical reports help keep a team informed on how their business critical services are doing. Developers,
DevOps/SRE teams, and their managers can be productive with automated reports reliably delivering insights
without requiring everyone to sign in the portal. Such reports can also help identify gradual increases in latencies,
load or failure rates that may not trigger any alert rules.
Each enterprise has its unique reporting needs, such as:
Specific percentile aggregations of metrics, or custom metrics in a report.
Have different reports for daily, weekly, and monthly roll-ups of data for different audiences.
Segmentation by custom attributes like region, or environment.
Group some AI resources together in a single report, even if they may be in different subscriptions or resource
groups etc.
Separate reports containing sensitive metrics sent to selective audience.
Reports to stakeholders who may not have access to the portal resources.
NOTE
The weekly Application Insights digest email did not allow any customization, and will be discontinued in favor of the custom
options listed below. The last weekly digest email will be sent on June 11, 2018. Please configure one of the following options
to get similar custom reports (use the query suggested below).
let period=7d;
requests
| where timestamp > ago(period)
| summarize Row = 1, TotalRequests = sum(itemCount), FailedRequests = sum(toint(success == 'False')),
RequestsDuration = iff(isnan(avg(duration)), '------', tostring(toint(avg(duration) * 100) / 100.0))
| join (
dependencies
| where timestamp > ago(period)
| summarize Row = 1, TotalDependencies = sum(itemCount), FailedDependencies = sum(success == 'False'),
DependenciesDuration = iff(isnan(avg(duration)), '------', tostring(toint(avg(duration) * 100) / 100.0))
) on Row | join (
pageViews
| where timestamp > ago(period)
| summarize Row = 1, TotalViews = sum(itemCount)
) on Row | join (
exceptions
| where timestamp > ago(period)
| summarize Row = 1, TotalExceptions = sum(itemCount)
) on Row | join (
availabilityResults
| where timestamp > ago(period)
| summarize Row = 1, OverallAvailability = iff(isnan(avg(toint(success))), '------',
tostring(toint(avg(toint(success)) * 10000) / 100.0)),
AvailabilityDuration = iff(isnan(avg(duration)), '------', tostring(toint(avg(duration) * 100) / 100.0))
) on Row
| project TotalRequests, FailedRequests, RequestsDuration, TotalDependencies, FailedDependencies,
DependenciesDuration, TotalViews, TotalExceptions, OverallAvailability, AvailabilityDuration
NOTE
By default, function apps are created with runtime version 2.x. You must target Azure Functions runtime version 1.x
to use the Application Insights scheduled digest template.
6. Enter an appropriate recipient e-mail address for your report and select Create.
7. Select your Function App > Platform features > Application settings.
8. Create three new application settings with appropriate corresponding values AI_APP_ID , AI_APP_KEY , and
SendGridAPI . Select Save.
(The AI_ values can be found under API Access for the Application Insights Resource you want to report on.
If you don't have an Application Insights API Key, there is the option to Create API Key.)
AI_APP_ID = Application ID
AI_APP_KEY = API Key
SendGridAPI =SendGrid API Key
NOTE
If you don't have a SendGrid account you can create one. SendGrid's documentation for Azure Functions is
here. If just want a minimal explanation of how to setup SendGrid and generate an API key one is provided at
the end of this article.
10. Under the SendGridAPI Key App Setting, select your newly created App Setting for SendGridAPI.
11. Run and test your Function App.
12. Check your e-mail to confirm that the message was sent/received successfully.
3. This will launch SendGrid's site. Select Settings > API Keys.
4. Create an API Key > choose Create & View (Please review SendGrid's documentation on restricted access
to determine what level of permissions is appropriate for your API Key. Full Access is selected here for
example purposes only.)
5. Copy the entire key, this value is what you need in your Function App settings as the value for SendGridAPI
Next steps
Learn more about creating Analytics queries.
Learn more about programmatically querying Application Insights data
Learn more about Logic Apps.
Learn more about Microsoft Flow.
Diagnose exceptions in your web apps with
Application Insights
12/13/2019 • 11 minutes to read • Edit Online
Exceptions in your live web app are reported by Application Insights. You can correlate failed requests with
exceptions and other events at both the client and server, so that you can quickly diagnose the causes.
Alternatively, instead of looking at exceptions of a specific failing operation, you can start from the overall
view of exceptions, by switching to the Exceptions tab at the top. Here you can see all the exceptions collected
for your monitored app.
No exceptions showing? See Capture exceptions.
NOTE
If your app generates a lot of telemetry, the adaptive sampling module will automatically reduce the volume that is sent
to the portal by sending only a representative fraction of events. Events that are part of the same operation will be
selected or deselected as a group, so that you can navigate between related events. Learn about sampling.
try
{ ...
}
catch (ex)
{
appInsights.trackException(ex, "handler loc",
{Game: currentGame.Name,
State: currentGame.State.ToString()});
}
The properties and measurements parameters are optional, but are useful for filtering and adding extra
information. For example, if you have an app that can run several games, you could find all the exception
reports related to a particular game. You can add as many items as you like to each dictionary.
Browser exceptions
Most browser exceptions are reported.
If your web page includes script files from content delivery networks or other domains, ensure your script tag
has the attribute crossorigin="anonymous" , and that the server sends CORS headers. This will allow you to get
a stack trace and detail for unhandled JavaScript exceptions from these resources.
static GoodController()
{
telemetryClient = new TelemetryClient();
}
}
Web forms
For web forms, the HTTP Module will be able to collect the exceptions when there are no redirects configured
with CustomErrors.
But if you have active redirects, add the following lines to the Application_Error function in Global.asax.cs.
(Add a Global.asax file if you don't already have one.)
ai.TrackException(Server.GetLastError());
}
}
MVC
Starting with Application Insights Web SDK version 2.6 (beta3 and later), Application Insights collects
unhandled exceptions thrown in the MVC 5+ controllers methods automatically. If you have previously added
a custom handler to track such exceptions (as described in following examples), you may remove it to prevent
double tracking of exceptions.
There are a number of cases that the exception filters cannot handle. For example:
Exceptions thrown from controller constructors.
Exceptions thrown from message handlers.
Exceptions thrown during routing.
Exceptions thrown during response content serialization.
Exception thrown during application start-up.
Exception thrown in background tasks.
All exceptions handled by application still need to be tracked manually. Unhandled exceptions originating
from controllers typically result in 500 "Internal Server Error" response. If such response is manually
constructed as a result of handled exception (or no exception at all) it is tracked in corresponding request
telemetry with ResultCode 500, however Application Insights SDK is unable to track corresponding
exception.
Prior versions support
If you use MVC 4 (and prior) of Application Insights Web SDK 2.5 (and prior), refer to the following examples
to track exceptions.
If the CustomErrors configuration is Off , then exceptions will be available for the HTTP Module to collect.
However, if it is RemoteOnly (default), or On , then the exception will be cleared and not available for
Application Insights to automatically collect. You can fix that by overriding the
System.Web.Mvc.HandleErrorAttribute class, and applying the overridden class as shown for the different
MVC versions below (GitHub source):
using System;
using System.Web.Mvc;
using Microsoft.ApplicationInsights;
namespace MVC2App.Controllers
{
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, Inherited = true, AllowMultiple =
true)]
public class AiHandleErrorAttribute : HandleErrorAttribute
{
public override void OnException(ExceptionContext filterContext)
{
if (filterContext != null && filterContext.HttpContext != null && filterContext.Exception !=
null)
{
//If customError is Off, then AI HTTPModule will report the exception
if (filterContext.HttpContext.IsCustomErrorEnabled)
{ //or reuse instance (recommended!). see note above
var ai = new TelemetryClient();
ai.TrackException(filterContext.Exception);
}
}
base.OnException(filterContext);
}
}
}
MVC 2
Replace the HandleError attribute with your new attribute in your controllers.
namespace MVC2App.Controllers
{
[AiHandleError]
public class HomeController : Controller
{
...
Sample
MVC 3
Register AiHandleErrorAttribute as a global filter in Global.asax.cs:
Sample
MVC 4, MVC5
Register AiHandleErrorAttribute as a global filter in FilterConfig.cs:
Sample
Web API
Starting with Application Insights Web SDK version 2.6 (beta3 and later), Application Insights collects
unhandled exceptions thrown in the controller methods automatically for WebAPI 2+. If you have previously
added a custom handler to track such exceptions (as described in following examples), you may remove it to
prevent double tracking of exceptions.
There are a number of cases that the exception filters cannot handle. For example:
Exceptions thrown from controller constructors.
Exceptions thrown from message handlers.
Exceptions thrown during routing.
Exceptions thrown during response content serialization.
Exception thrown during application start-up.
Exception thrown in background tasks.
All exceptions handled by application still need to be tracked manually. Unhandled exceptions originating
from controllers typically result in 500 "Internal Server Error" response. If such response is manually
constructed as a result of handled exception (or no exception at all) it is tracked in a corresponding request
telemetry with ResultCode 500, however Application Insights SDK is unable to track corresponding
exception.
Prior versions support
If you use WebAPI 1 (and prior) of Application Insights Web SDK 2.5 (and prior), refer to the following
examples to track exceptions.
Web API 1.x
Override System.Web.Http.Filters.ExceptionFilterAttribute:
using System.Web.Http.Filters;
using Microsoft.ApplicationInsights;
namespace WebAPI.App_Start
{
public class AiExceptionFilterAttribute : ExceptionFilterAttribute
{
public override void OnException(HttpActionExecutedContext actionExecutedContext)
{
if (actionExecutedContext != null && actionExecutedContext.Exception != null)
{ //or reuse instance (recommended!). see note above
var ai = new TelemetryClient();
ai.TrackException(actionExecutedContext.Exception);
}
base.OnException(actionExecutedContext);
}
}
}
You could add this overridden attribute to specific controllers, or add it to the global filter configuration in the
WebApiConfig class:
using System.Web.Http;
using WebApi1.x.App_Start;
namespace WebApi1.x
{
public static class WebApiConfig
{
public static void Register(HttpConfiguration config)
{
config.Routes.MapHttpRoute(name: "DefaultApi", routeTemplate: "api/{controller}/{id}",
defaults: new { id = RouteParameter.Optional });
...
config.EnableSystemDiagnosticsTracing();
Sample
Web API 2.x
Add an implementation of IExceptionLogger:
using System.Web.Http.ExceptionHandling;
using Microsoft.ApplicationInsights;
namespace ProductsAppPureWebAPI.App_Start
{
public class AiExceptionLogger : ExceptionLogger
{
public override void Log(ExceptionLoggerContext context)
{
if (context !=null && context.Exception != null)
{//or reuse instance (recommended!). see note above
var ai = new TelemetryClient();
ai.TrackException(context.Exception);
}
base.Log(context);
}
}
}
using System.Web.Http;
using System.Web.Http.ExceptionHandling;
using ProductsAppPureWebAPI.App_Start;
namespace WebApi2WithMVC
{
public static class WebApiConfig
{
public static void Register(HttpConfiguration config)
{
// Web API configuration and services
config.Routes.MapHttpRoute(
name: "DefaultApi",
routeTemplate: "api/{controller}/{id}",
defaults: new { id = RouteParameter.Optional }
);
config.Services.Add(typeof(IExceptionLogger), new AiExceptionLogger());
}
}
}
Sample
As alternatives, you could:
1. Replace the only ExceptionHandler with a custom implementation of IExceptionHandler. This is only called
when the framework is still able to choose which response message to send (not when the connection is
aborted for instance)
2. Exception Filters (as described in the section on Web API 1.x controllers above) - not called in all cases.
WCF
Add a class that extends Attribute and implements IErrorHandler and IServiceBehavior.
using System;
using System.Collections.Generic;
using System.Linq;
using System.ServiceModel.Description;
using System.ServiceModel.Dispatcher;
using System.Web;
using Microsoft.ApplicationInsights;
namespace WcfService4.ErrorHandling
{
public class AiLogExceptionAttribute : Attribute, IErrorHandler, IServiceBehavior
{
public void AddBindingParameters(ServiceDescription serviceDescription,
System.ServiceModel.ServiceHostBase serviceHostBase,
System.Collections.ObjectModel.Collection<ServiceEndpoint> endpoints,
System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
{
}
ai.TrackException(error);
return false;
}
namespace WcfService4
{
[AiLogException]
public class Service1 : IService1
{
...
Sample
Next steps
Monitor REST, SQL, and other calls to dependencies
Monitor page load times, browser exceptions, and AJAX calls
Monitor performance counters
Explore .NET/.NET Core and Python trace logs in
Application Insights
12/13/2019 • 6 minutes to read • Edit Online
Send diagnostic tracing logs for your ASP.NET/ASP.NET Core application from ILogger, NLog, log4Net, or
System.Diagnostics.Trace to Azure Application Insights. For Python applications, send diagnostic tracing logs
using AzureLogHandler in OpenCensus Python for Azure Monitor. You can then explore and search them.
Those logs are merged with the other log files from your application, so you can identify traces that are
associated with each user request and correlate them with other events and exception reports.
NOTE
Do you need the log-capture module? It's a useful adapter for third-party loggers. But if you aren't already using NLog,
log4Net, or System.Diagnostics.Trace, consider just calling Application Insights TrackTrace() directly.
<configuration>
<system.diagnostics>
<trace>
<listeners>
<add name="myAppInsightsListener"
type="Microsoft.ApplicationInsights.TraceListener.ApplicationInsightsTraceListener,
Microsoft.ApplicationInsights.TraceListener" />
</listeners>
</trace>
</system.diagnostics>
</configuration>
NOTE
No Application Insights menu or log collector option? Try Troubleshooting.
Manual installation
Use this method if your project type isn't supported by the Application Insights installer (for example a
Windows desktop project).
1. If you plan to use log4Net or NLog, install it in your project.
2. In Solution Explorer, right-click your project, and select Manage NuGet Packages.
3. Search for "Application Insights."
4. Select one of the following packages:
For ILogger: Microsoft.Extensions.Logging.ApplicationInsights
For NLog: Microsoft.ApplicationInsights.NLogTarget
For Log4Net: Microsoft.ApplicationInsights.Log4NetAppender nuget v2.13.0-beta1
ILogger
For examples of using the Application Insights ILogger implementation with console applications and
ASP.NET Core, see ApplicationInsightsLoggerProvider for .NET Core ILogger logs.
<Add Type="Microsoft.ApplicationInsights.EventSourceListener.EventSourceTelemetryModule,
Microsoft.ApplicationInsights.EventSourceListener">
<Sources>
<Add Name="MyCompany" Level="Verbose" />
</Sources>
</Add>
<Add Type="Microsoft.ApplicationInsights.DiagnosticSourceListener.DiagnosticSourceTelemetryModule,
Microsoft.ApplicationInsights.DiagnosticSourceListener">
<Sources>
<Add Name="MyDiagnosticSourceName" />
</Sources>
</Add>
For each DiagnosticSource you want to trace, add an entry with the Name attribute set to the name of your
DiagnosticSource.
NOTE
ETW events can only be collected if the process that hosts the SDK runs under an identity that's a member of
Performance Log Users or Administrators.
<Add Type="Microsoft.ApplicationInsights.EtwCollector.EtwCollectorTelemetryModule,
Microsoft.ApplicationInsights.EtwCollector">
<Sources>
<Add ProviderName="MyCompanyEventSourceName" Level="Verbose" />
</Sources>
</Add>
An advantage of TrackTrace is that you can put relatively long data in the message. For example, you can
encode POST data there.
You can also add a severity level to your message. And, like other telemetry, you can add property values to
help filter or search for different sets of traces. For example:
var telemetry = new Microsoft.ApplicationInsights.TelemetryClient();
telemetry.TrackTrace("Slow database response",
SeverityLevel.Warning,
new Dictionary<string,string> { {"database", db.ID} });
This would enable you to easily filter out in Search all the messages of a particular severity level that relate to
a particular database.
import logging
logger = logging.getLogger(__name__)
logger.addHandler(AzureLogHandler(connection_string='InstrumentationKey=<your-instrumentation_key-here>'))
logger.warning('Hello, World!')
NOTE
If your application sends a lot of data and you're using the Application Insights SDK for ASP.NET version 2.0.0-beta3 or
later, the adaptive sampling feature may operate and send only a portion of your telemetry. Learn more about
sampling.
Troubleshooting
How do I do this for Java?
Use the Java log adapters.
There's no Application Insights option on the project context menu
Make sure that Developer Analytics Tools is installed on the development machine. At Visual Studio Tools
> Extensions and Updates, look for Developer Analytics Tools. If it isn't on the Installed tab, open the
Online tab and install it.
This might be a project type that Devloper Analytics Tools doesn't support. Use manual installation.
There's no log adapter option in the configuration tool
Install the logging framework first.
If you're using System.Diagnostics.Trace, make sure that you have it configured in web.config.
Make sure that you have the latest version of Application Insights. In Visual Studio, go to Tools >
Extensions and Updates, and open the Updates tab. If Developer Analytics Tools is there, select it to
update it.
I get the "Instrumentation key cannot be empty" error message
You probably installed the logging adapter Nuget package without installing Application Insights. In Solution
Explorer, right-click ApplicationInsights.config, and select Update Application Insights. You'll be prompted
to sign in to Azure and create an Application Insights resource or reuse an existing one. That should fix the
problem.
I can see traces but not other events in diagnostic search
It can take a while for all the events and requests to get through the pipeline.
How much data is retained?
Several factors affect the amount of data that's retained. For more information, see the limits section of the
customer event metrics page.
I don't see some log entries that I expected
If your application sends voluminous amounts of data and you're using the Application Insights SDK for
ASP.NET version 2.0.0-beta3 or later, the adaptive sampling feature may operate and send only a portion of
your telemetry. Learn more about sampling.
Next steps
Diagnose failures and exceptions in ASP.NET
Learn more about Search
Set up availability and responsiveness tests
Troubleshooting
System performance counters in Application Insights
12/2/2019 • 4 minutes to read • Edit Online
Windows provides a wide variety of performance counters such as CPU occupancy, memory, disk, and network
usage. You can also define your own performance counters. Performance counters collection is supported as long
as your application is running under IIS on an on-premises host, or virtual machine to which you have
administrative access. Though applications running as Azure Web Apps don't have direct access to performance
counters, a subset of available counters are collected by Application Insights.
View counters
The Metrics pane shows the default set of performance counters.
The current default counters that are configured to be collected for ASP.NET/ASP.NET Core web applications
are:
% Process\Processor Time
% Process\Processor Time Normalized
Memory\Available Bytes
ASP.NET Requests/Sec
.NET CLR Exceptions Thrown / sec
ASP.NET ApplicationsRequest Execution Time
Process\Private Bytes
Process\IO Data Bytes/sec
ASP.NET Applications\Requests In Application Queue
Processor(_Total)\% Processor Time
Add counters
If the performance counter you want isn't included in the list of metrics, you can add it.
1. Find out what counters are available in your server by using this PowerShell command on the local server:
Get-Counter -ListSet *
(See Get-Counter .)
2. Open ApplicationInsights.config.
If you added Application Insights to your app during development, edit ApplicationInsights.config in
your project, and then redeploy it to your servers.
3. Edit the performance collector directive:
<Add
Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.PerformanceCollectorModule,
Microsoft.AI.PerfCounterCollector">
<Counters>
<Add PerformanceCounter="\Objects\Processes"/>
<Add PerformanceCounter="\Sales(photo)\# Items Sold" ReportAs="Photo sales"/>
</Counters>
</Add>
NOTE
ASP.NET Core applications do not have ApplicationInsights.config , and hence the above method is not valid for
ASP.NET Core Applications.
You can capture both standard counters and those you've implemented yourself. \Objects\Processes is an
example of a standard counter that is available on all Windows systems. \Sales(photo)\# Items Sold is an
example of a custom counter that might be implemented in a web service.
The format is \Category(instance)\Counter" , or for categories that don't have instances, just \Category\Counter .
ReportAs is required for counter names that do not match [a-zA-Z()/-_ \.]+ - that is, they contain characters
that are not in the following sets: letters, round brackets, forward slash, hyphen, underscore, space, dot.
If you specify an instance, it will be collected as a dimension "CounterInstanceName" of the reported metric.
Collecting performance counters in code for ASP.NET Web Applications or .NET/.NET Core Console
Applications
To collect system performance counters and send them to Application Insights, you can adapt the snippet below:
Or you can do the same thing with custom metrics you created:
('Instance' here refers to the performance counter instance, not the role, or server machine instance. The
performance counter instance name typically segments counters such as processor time by the name of the
process or application.)
To get a chart of available memory over the recent period:
Like other telemetry, performanceCounters also has a column cloud_RoleInstance that indicates the identity of
the host server instance on which your app is running. For example, to compare the performance of your app on
the different machines:
Alerts
Like other metrics, you can set an alert to warn you if a performance counter goes outside a limit you specify.
Open the Alerts pane and click Add Alert.
Next steps
Dependency tracking
Exception tracking
EventCounters introduction
12/17/2019 • 3 minutes to read • Edit Online
EventCounter is .NET/.NET Core mechanism to publish and consume counters or statistics. This document gives
an overview of EventCounters and examples on how to publish and consume them. EventCounters are supported
in all OS platforms - Windows, Linux, and macOS. It can be thought of as a cross-platform equivalent for the
PerformanceCounters that is only supported in Windows systems.
While users can publish any custom EventCounters to meet their needs, the .NET Core 3.0 runtime publishes a set
of these counters by default. The document will walk through the steps required to collect and view
EventCounters (system defined or user defined) in Azure Application Insights.
CATEGORY COUNTER
System.Runtime cpu-usage
System.Runtime working-set
System.Runtime gc-heap-size
System.Runtime gen-0-gc-count
System.Runtime gen-1-gc-count
System.Runtime gen-2-gc-count
System.Runtime time-in-gc
System.Runtime gen-0-size
System.Runtime gen-1-size
System.Runtime gen-2-size
System.Runtime loh-size
CATEGORY COUNTER
System.Runtime alloc-rate
System.Runtime assembly-count
System.Runtime exception-count
System.Runtime threadpool-thread-count
System.Runtime monitor-lock-contention-count
System.Runtime threadpool-queue-length
System.Runtime threadpool-completed-items-count
System.Runtime active-timer-count
Microsoft.AspNetCore.Hosting requests-per-second
Microsoft.AspNetCore.Hosting total-requests
Microsoft.AspNetCore.Hosting current-requests
Microsoft.AspNetCore.Hosting failed-requests
NOTE
Counters of category Microsoft.AspNetCore.Hosting are only added in ASP.NET Core Applications.
// This adds a user defined counter "MyCounter" from EventSource named "MyEventSource"
module.Counters.Add(new EventCounterCollectionRequest("MyEventSource", "MyCounter"));
customMetrics
| where name contains "System.Runtime|ThreadPool Completed Work Item Count"
| where timestamp >= ago(1h)
| summarize avg(value) by cloud_RoleInstance, bin(timestamp, 1m)
| render timechart
Like other telemetry, customMetrics also has a column cloud_RoleInstance that indicates the identity of the host
server instance on which your app is running. The above query shows the counter value per instance, and can be
used to compare performance of different server instances.
Alerts
Like other metrics, you can set an alert to warn you if an event counter goes outside a limit you specify. Open the
Alerts pane and click Add Alert.
Next steps
Dependency tracking
Dependency Tracking in Azure Application Insights
1/19/2020 • 5 minutes to read • Edit Online
A dependency is an external component that is called by your application. It's typically a service called using
HTTP, or a database, or a file system. Application Insights measures the duration of dependency calls, whether
its failing or not, along with additional information like name of dependency and so on. You can investigate
specific dependency calls, and correlate them to requests and exceptions.
DEPENDENCIES DETAILS
SQL Calls made with SqlClient . See this for capturing SQL
query.
Azure storage (Blob, Table, Queue ) Calls made with Azure Storage Client.
If you're missing a dependency, or using a different SDK make sure it's in the list ofauto-collected
dependencies. If the dependency isn't auto-collected, you can still track it manually with a track dependency
call.
Alternatively, TelemetryClient provides extension methods StartOperation and StopOperation which can be
used to manually track dependencies, as shown here
If you want to switch off the standard dependency tracking module, remove the reference to
DependencyTrackingTelemetryModule in ApplicationInsights.config for ASP.NET applications. For ASP.NET
Core applications, follow instructions here.
For ASP.NET Core applications, there's no additional step required to get the full SQL Query.
For ASP.NET applications, full SQL query is collected with the help of byte code instrumentation, which
requires instrumentation engine. Additional platform-specific steps, as described below, are required.
Azure Web App In your web app control panel, open the Application
Insights blade and enable SQL Commands under .NET
IIS Server (Azure VM, on-prem, and so on.) Use the Status Monitor PowerShell Module to install the
Instrumentation Engine and restart IIS.
In the above cases, the correct way of validating that instrumentation engine is correctly installed is by
validating that the SDK version of collected DependencyTelemetry is 'rddp'. 'rdddsd' or 'rddf' indicates
dependencies are collected via DiagnosticSource or EventSource callbacks, and hence full SQL query won't be
captured.
Failed requests
Failed requests might also be associated with failed calls to dependencies.
We can go to the Failures tab on the left and then click on the dependencies tab at the top.
Here you will be able to see the failed dependency count. To get more details about a failed occurrence trying
clicking on a dependency name in the bottom table. You can click on the blue Dependencies button at the
bottom right to get the end-to-end transaction details.
Logs (Analytics)
You can track dependencies in the Kusto query language. Here are some examples.
Find any failed dependency calls:
dependencies
| where timestamp > ago(1d) and client_Type != "Browser"
| join (requests | where timestamp > ago(1d))
on operation_Id
Open-source SDK
Like every Application Insights SDK, dependency collection module is also open-source. Read and contribute
to the code, or report issues at the official GitHub repo.
Next steps
Exceptions
User & page data
Availability
Configuring the Application Insights SDK with
ApplicationInsights.config or .xml
12/19/2019 • 10 minutes to read • Edit Online
The Application Insights .NET SDK consists of a number of NuGet packages. The core package provides the
API for sending telemetry to the Application Insights. Additional packages provide telemetry modules and
initializers for automatically tracking telemetry from your application and its context. By adjusting the
configuration file, you can enable or disable Telemetry Modules and initializers, and set parameters for some
of them.
The configuration file is named ApplicationInsights.config or ApplicationInsights.xml , depending on the
type of your application. It is automatically added to your project when you install most versions of the SDK.
By default, when using the automated experience from the Visual Studio template projects that support Add
> Application Insights Telemetry, the ApplicationInsights.config file is created in the project root folder
and when complied is copied to the bin folder. It is also added to a web app by Status Monitor on an IIS
server. The configuration file is ignored if extension for Azure website or extension for Azure VM and virtual
machine scale set is used.
There isn't an equivalent file to control the SDK in a web page.
This document describes the sections you see in the configuration file, how they control the components of
the SDK, and which NuGet packages load those components.
NOTE
ApplicationInsights.config and .xml instructions do not apply to the .NET Core SDK. For configuring .NET Core
applications, follow this guide.
* `Microsoft.ApplicationInsights.Extensibility.Implementation.Tracing.DiagnosticsTelemetryModule`
* [Microsoft.ApplicationInsights](https://www.nuget.org/packages/Microsoft.ApplicationInsights) NuGet
package. If you only install this package, the ApplicationInsights.config file is not automatically
created.
Developer Mode
DeveloperModeWithDebuggerAttachedTelemetryModule forces the Application Insights TelemetryChannel to send
data immediately, one telemetry item at a time, when a debugger is attached to the application process. This
reduces the amount of time between the moment when your application tracks telemetry and when it
appears on the Application Insights portal. It causes significant overhead in CPU and network bandwidth.
Microsoft.ApplicationInsights.WindowsServer.DeveloperModeWithDebuggerAttachedTelemetryModule
Application Insights Windows Server NuGet package
Web Request Tracking
Reports the response time and result code of HTTP requests.
Microsoft.ApplicationInsights.Web.RequestTrackingTelemetryModule
Microsoft.ApplicationInsights.Web NuGet package
Exception tracking
ExceptionTrackingTelemetryModule tracks unhandled exceptions in your web app. See Failures and exceptions.
Microsoft.ApplicationInsights.Web.ExceptionTrackingTelemetryModule
Microsoft.ApplicationInsights.Web NuGet package
Microsoft.ApplicationInsights.WindowsServer.UnobservedExceptionTelemetryModule - tracks unobserved task
exceptions.
Microsoft.ApplicationInsights.WindowsServer.UnhandledExceptionTelemetryModule - tracks unhandled
exceptions for worker roles, windows services, and console applications.
Application Insights Windows Server NuGet package.
EventSource Tracking
EventSourceTelemetryModule allows you to configure EventSource events to be sent to Application Insights as
traces. For information on tracking EventSource events, see Using EventSource Events.
Microsoft.ApplicationInsights.EventSourceListener.EventSourceTelemetryModule
Microsoft.ApplicationInsights.EventSourceListener
ETW Event Tracking
EtwCollectorTelemetryModule allows you to configure events from ETW providers to be sent to Application
Insights as traces. For information on tracking ETW events, see Using ETW Events.
Microsoft.ApplicationInsights.EtwCollector.EtwCollectorTelemetryModule
Microsoft.ApplicationInsights.EtwCollector
Microsoft.ApplicationInsights
The Microsoft.ApplicationInsights package provides the core API of the SDK. The other Telemetry Modules
use this, and you can also use it to define your own telemetry.
No entry in ApplicationInsights.config.
Microsoft.ApplicationInsights NuGet package. If you just install this NuGet, no .config file is generated.
Telemetry Channel
The telemetry channel manages buffering and transmission of telemetry to the Application Insights service.
Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.ServerTelemetryChannel is the default
channel for web applications. It buffers data in memory, and employs retry mechanisms and local disk
storage for more reliable telemetry delivery.
Microsoft.ApplicationInsights.InMemoryChannel is a lightweight telemetry channel, which is used if no
other channel is configured.
For .NET applications running in Service Fabric, you can include the
Microsoft.ApplicationInsights.ServiceFabric NuGet package. This package includes a
FabricTelemetryInitializer , which adds Service Fabric properties to telemetry items. For more information,
see the GitHub page about the properties added by this NuGet package.
<TelemetryProcessors>
<Add
Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.AdaptiveSamplingTelemetryProcessor,
Microsoft.AI.ServerTelemetryChannel">
<MaxTelemetryItemsPerSecond>5</MaxTelemetryItemsPerSecond>
</Add>
</TelemetryProcessors>
The parameter provides the target that the algorithm tries to achieve. Each instance of the SDK works
independently, so if your server is a cluster of several machines, the actual volume of telemetry will be
multiplied accordingly.
Learn more about sampling.
Fixed-rate sampling Telemetry Processor (from 2.0.0-beta1)
There is also a standard sampling Telemetry Processor (from 2.0.1):
<TelemetryProcessors>
<Add Type="Microsoft.ApplicationInsights.WindowsServer.TelemetryChannel.SamplingTelemetryProcessor,
Microsoft.AI.ServerTelemetryChannel">
<ApplicationInsights>
...
<Channel>
<MaxTelemetryBufferCapacity>100</MaxTelemetryBufferCapacity>
</Channel>
...
</ApplicationInsights>
FlushIntervalInSeconds
Determines how often the data that is stored in the in-memory storage should be flushed (sent to
Application Insights).
Min: 1
Max: 300
Default: 5
<ApplicationInsights>
...
<Channel>
<FlushIntervalInSeconds>100</FlushIntervalInSeconds>
</Channel>
...
</ApplicationInsights>
MaxTransmissionStorageCapacityInMB
Determines the maximum size in MB that is allotted to the persistent storage on the local disk. This storage is
used for persisting telemetry items that failed to be transmitted to the Application Insights endpoint. When
the storage size has been met, new telemetry items will be discarded.
Min: 1
Max: 100
Default: 10
<ApplicationInsights>
...
<Channel>
<MaxTransmissionStorageCapacityInMB>50</MaxTransmissionStorageCapacityInMB>
</Channel>
...
</ApplicationInsights>
InstrumentationKey
This determines the Application Insights resource in which your data appears. Typically you create a separate
resource, with a separate key, for each of your applications.
If you want to set the key dynamically - for example if you want to send results from your application to
different resources - you can omit the key from the configuration file, and set it in code instead.
To set the key for all instances of TelemetryClient, including standard Telemetry Modules. Do this in an
initialization method, such as global.aspx.cs in an ASP.NET service:
using Microsoft.ApplicationInsights.Extensibility;
using Microsoft.ApplicationInsights;
If you just want to send a specific set of events to a different resource, you can set the key for a specific
TelemetryClient:
To get a new key, create a new resource in the Application Insights portal.
ApplicationId Provider
Available starting in v2.6.0
The purpose of this provider is to lookup an Application ID based on an Instrumentation Key. The
Application ID is included in RequestTelemetry and DependencyTelemetry and used to determine Correlation
in the Portal.
This is available by setting TelemetryConfiguration.ApplicationIdProvider either in code or in config.
Interface: IApplicationIdProvider
public interface IApplicationIdProvider
{
bool TryGetApplicationId(string instrumentationKey, out string applicationId);
}
ApplicationInsightsApplicationIdProvider
This is a wrapper around our Profile API. It will throttle requests and cache results.
This provider is added to your config file when you install either
Microsoft.ApplicationInsights.DependencyCollector or Microsoft.ApplicationInsights.Web
This class has an optional property . By default this is set to
ProfileQueryEndpoint
https://dc.services.visualstudio.com/api/profiles/{0}/appId . If you need to configure a proxy for this
configuration, we recommend proxying the base address and including "/api/profiles/{0}/appId". Note that
'{0}' is substituted at runtime per request with the Instrumentation Key.
Example Configuration via ApplicationInsights.config:
<ApplicationInsights>
...
<ApplicationIdProvider
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplica
tionIdProvider, Microsoft.ApplicationInsights">
<ProfileQueryEndpoint>https://dc.services.visualstudio.com/api/profiles/{0}/appId</ProfileQueryEndpoint>
</ApplicationIdProvider>
...
</ApplicationInsights>
DictionaryApplicationIdProvider
This is a static provider, which will rely on your configured Instrumentation Key / Application ID pairs.
This class has a property Defined , which is a Dictionary<string,string> of Instrumentation Key to
Application ID pairs.
This class has an optional property Next which can be used to configure another provider to use when an
Instrumentation Key is requested that does not exist in your configuration.
Example Configuration via ApplicationInsights.config:
<ApplicationInsights>
...
<ApplicationIdProvider
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.DictionaryApplicationIdPro
vider, Microsoft.ApplicationInsights">
<Defined>
<Type key="InstrumentationKey_1" value="ApplicationId_1"/>
<Type key="InstrumentationKey_2" value="ApplicationId_2"/>
</Defined>
<Next
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplica
tionIdProvider, Microsoft.ApplicationInsights" />
</ApplicationIdProvider>
...
</ApplicationInsights>
Next steps
Learn more about the API.
Correlating Application Insights data with custom
data sources
10/20/2019 • 2 minutes to read • Edit Online
Application Insights collects several different data types: exceptions, traces, page views, and others. While this is
often sufficient to investigate your application’s performance, reliability, and usage, there are cases when it is useful
to correlate the data stored in Application Insights to other completely custom datasets.
Some situations where you might want custom data include:
Data enrichment or lookup tables: for example, supplement a server name with the owner of the server and the
lab location in which it can be found
Correlation with non-Application Insights data sources: for example, correlate data about a purchase on a web-
store with information from your purchase-fulfillment service to determine how accurate your shipping time
estimates were
Completely custom data: many of our customers love the query language and performance of the Azure
Monitor log platform that backs Application Insights, and want to use it to query data that is not at all related to
Application Insights. For example, to track the solar panel performance as part of a smart home installation as
outlined here.
Ingesting data
In this section, we will review how to get your data into Azure Monitor logs.
If you don’t already have one, provision a new Log Analytics workspace by following these instructions through
and including the “create a workspace” step.
To start sending log data into Azure Monitor. Several options exist:
For a synchronous mechanism, you can either directly call the data collector API or use our Logic App
connector – simply look for “Azure Log Analytics” and pick the “Send Data” option:
For an asynchronous option, use the Data Collector API to build a processing pipeline. See this article for
details.
Correlating data
Application Insights is based on the Azure Monitor log platform. We can therefore use cross-resource joins to
correlate any data we ingested into Azure Monitor with our Application Insights data.
For example, we can ingest our lab inventory and locations into a table called “LabLocations_CL” in a Log Analytics
workspace called “myLA”. If we then wanted to review our requests tracked in Application Insights app called
“myAI” and correlate the machine names that served the requests to the locations of these machines stored in the
previously mentioned custom table, we can run the following query from either the Application Insights or Azure
Monitor context:
app('myAI').requests
| join kind= leftouter (
workspace('myLA').LabLocations_CL
| project Computer_S, Owner_S, Lab_S
) on $left.cloud_RoleInstance == $right.Computer
Next Steps
Check out the Data Collector API reference.
For more information on cross-resource joins.
Deep diagnostics for web apps and services with
Application Insights
12/8/2019 • 10 minutes to read • Edit Online
It's essential to monitor a modern application while it is running. Most importantly, you want to detect failures
before most of your customers do. You also want to discover and fix performance issues that, while not
catastrophic, perhaps slow things down or cause some inconvenience to your users. And when the system is
performing to your satisfaction, you want to know what the users are doing with it: Are they using the latest
feature? Are they succeeding with it?
Modern web applications are developed in a cycle of continuous delivery: release a new feature or improvement;
observe how well it works for the users; plan the next increment of development based on that knowledge. A key
part of this cycle is the observation phase. Application Insights provides the tools to monitor a web application for
performance and usage.
The most important aspect of this process is diagnostics and diagnosis. If the application fails, then business is
being lost. The prime role of a monitoring framework is therefore to detect failures reliably, notify you immediately,
and to present you with the information needed to diagnose the problem. This is exactly what Application Insights
does.
Where do bugs come from?
Failures in web systems typically arise from configuration issues or bad interactions between their many
components. The first task when tackling a live site incident is therefore to identify the locus of the problem: which
component or relationship is the cause?
Some of us, those with gray hair, can remember a simpler era in which a computer program ran in one computer.
The developers would test it thoroughly before shipping it; and having shipped it, would rarely see or think about it
again. The users would have to put up with the residual bugs for many years.
Things are so very different now. Your app has a plethora of different devices to run on, and it can be difficult to
guarantee the exact same behavior on each one. Hosting apps in the cloud means bugs can be fixed fast, but it also
means continuous competition and the expectation of new features at frequent intervals.
In these conditions, the only way to keep a firm control on the bug count is automated unit testing. It would be
impossible to manually re-test everything on every delivery. Unit test is now a commonplace part of the build
process. Tools such as the Xamarin Test Cloud help by providing automated UI testing on multiple browser
versions. These testing regimes allow us to hope that the rate of bugs found inside an app can be kept to a
minimum.
Typical web applications have many live components. In addition to the client (in a browser or device app) and the
web server, there's likely to be substantial backend processing. Perhaps the backend is a pipeline of components, or
a looser collection of collaborating pieces. And many of them won't be in your control - they are external services
on which you depend.
In configurations like these, it can be difficult and uneconomical to test for, or foresee, every possible failure mode,
other than in the live system itself.
Questions ...
Some questions we ask when we're developing a web system:
Is my app crashing?
What exactly happened? - If it failed a request, I want to know how it got there. We need a trace of events...
Is my app fast enough? How long does it take to respond to typical requests?
Can the server handle the load? When the rate of requests rises, does the response time hold steady?
How responsive are my dependencies - the REST APIs, databases and other components that my app calls. In
particular, if the system is slow, is it my component, or am I getting slow responses from someone else?
Is my app Up or Down? Can it be seen from around the world? Let me know if it stops....
What is the root cause? Was the failure in my component or a dependency? Is it a communication issue?
How many users are impacted? If I have more than one issue to tackle, which is the most important?
Smart detection
Proactive diagnostics is a recent feature. Without any special configuration by you, Application Insights
automatically detects and alerts you about unusual rises in failure rates in your app. It's smart enough to ignore a
background of occasional failures, and also rises that are simply proportionate to a rise in requests. So for
example, if there's a failure in one of the services you depend on, or if the new build you just deployed isn't
working so well, then you'll know about it as soon as you look at your email. (And there are webhooks so that you
can trigger other apps.)
Another aspect of this feature performs a daily in-depth analysis of your telemetry, looking for unusual patterns of
performance that are hard to discover. For example, it can find slow performance associated with a particular
geographical area, or with a particular browser version.
In both cases, the alert not only tells you the symptoms it's discovered, but also gives you data you need to help
diagnose the problem, such as relevant exception reports.
Customer Samtec said: "During a recent feature cutover, we found an under-scaled database that was hitting its
resource limits and causing timeouts. Proactive detection alerts came through literally as we were triaging the
issue, very near real time as advertised. This alert coupled with the Azure platform alerts helped us almost
instantly fix the issue. Total downtime < 10 minutes."
Application Map
Application Map automatically discovers your application topology, laying the performance information on top of
it, to let you easily identify performance bottlenecks and problematic flows across your distributed environment. It
allows you to discover application dependencies on Azure Services. You can triage the problem by understanding
if it is code-related or dependency related and from a single place drill into related diagnostics experience. For
example, your application may be failing due to performance degradation in SQL tier. With application map, you
can see it immediately and drill into the SQL Index Advisor or Query Insights experience.
Video
Next steps
Getting started with Application Insights is easy. The main options are:
IIS servers, and also for Azure App Service.
Instrument your project during development. You can do this for ASP.NET or Java apps, as well as Node.js and
a host of other types.
Instrument any web page by adding a short code snippet.
Monitor performance in web applications
10/24/2019 • 6 minutes to read • Edit Online
Make sure your application is performing well, and find out quickly about any failures. Application Insights will tell
you about any performance issues and exceptions, and help you find and diagnose the root causes.
Application Insights can monitor both Java and ASP.NET web applications and services, WCF services. They can
be hosted on-premises, on virtual machines, or as Microsoft Azure websites.
On the client side, Application Insights can take telemetry from web pages and a wide variety of devices including
iOS, Android, and Windows Store apps.
NOTE
Uncheck all the metrics to see the full selection that is available. The metrics fall into groups; when any member of a group
is selected, only the other members of that group appear.
Slowest requests
Selecting any metric disables the others that can't appear on the same chart.
Set alerts
To be notified by email of unusual values of any metric, add an alert. You can choose either to send the email to the
account administrators, or to specific email addresses.
Set the resource before the other properties. Don't choose the webtest resources if you want to set alerts on
performance or usage metrics.
Be careful to note the units in which you're asked to enter the threshold value.
I don't see the Add Alert button. - Is this a group account to which you have read-only access? Check with the
account administrator.
Diagnosing issues
Here are a few tips for finding and diagnosing performance issues:
Set up web tests to be alerted if your web site goes down or responds incorrectly or slowly.
Compare the Request count with other metrics to see if failures or slow response are related to load.
Insert and search trace statements in your code to help pinpoint problems.
Monitor your Web app in operation with Live Metrics Stream.
Capture the state of your .NET application with Snapshot Debugger.
To get a better sense of the user experiences for this operation, we can select a larger time range. We can then also
narrow down in time on a specific time window where the operation was slow. In the following example, we've
switched from the default 24 hours time range to the 7 days time range and then zoomed into the 9:47 to 12:47
time window between Tue the 12th and Wed the 13th. Both the duration distribution and the number of sample
and profiler traces have been updated on the right.
To narrow in on the slow experiences, we next zoom into the durations that fall between 95th and the 99th
percentile. These represent the 4% of user interactions that were slow.
We can now either look at the representative samples, by clicking on the Samples button, or at the representative
profiler traces, by clicking on the Profiler traces button. In this example, there are four traces that have been
collected for GET Customers/Details in the time window and range duration of interest.
Sometimes the issue will not be in your code, but rather in a dependency your code calls. You can switch to the
Dependencies tab in the performance triage view to investigate such slow dependencies. By default the
performance view is trending averages, but what you really want to look at is the 95th percentile (or the 99th, in
case you are monitoring a mature service). In the following example we have focused on the slow Azure BLOB
dependency, where we call PUT fabrikamaccount. The good experiences cluster around 40 ms, while the slow calls
to the same dependency are three times slower, clustering around 120 ms. It doesn't take many of these calls to
add up to cause the respective operation to noticeably slow down. You can drill into the representative samples and
profiler traces, just like you can with the Operations tab.
The performance investigation experience shows relevant insights along side the sample set you decided to focus
on. The best way to look at all of the available insights is to switch to a 30 days time range and then select Overall
to see insights across all operations for the past month.
Next steps
Web tests - Have web requests sent to your application at regular intervals from around the world.
Capture and search diagnostic traces - Insert trace calls and sift through the results to pinpoint issues.
Usage tracking - Find out how people use your application.
Troubleshooting - and Q & A
Separating telemetry from Development, Test, and
Production
12/12/2019 • 4 minutes to read • Edit Online
When you are developing the next version of a web application, you don't want to mix up the Application Insights
telemetry from the new version and the already released version. To avoid confusion, send the telemetry from
different development stages to separate Application Insights resources, with separate instrumentation keys
(ikeys). To make it easier to change the instrumentation key as a version moves from one stage to another, it can
be useful to set the ikey in code instead of in the configuration file.
(If your system is an Azure Cloud Service, there's another method of setting separate ikeys.)
In this example, the ikeys for the different resources are placed in different versions of the web configuration file.
Swapping the web configuration file - which you can do as part of the release script - will swap the target
resource.
Web pages
The iKey is also used in your app's web pages, in the script that you got from the quickstart blade. Instead of
coding it literally into the script, generate it from the server state. For example, in an ASP.NET app:
JavaScript in Razor
<script type="text/javascript">
// Standard Application Insights web page script:
var appInsights = window.appInsights || function(config){ ...
// Modify this part:
}({instrumentationKey:
// Generate from server property:
"@Microsoft.ApplicationInsights.Extensibility.
TelemetryConfiguration.Active.InstrumentationKey"
}) // ...
Application type affects what you see on the overview blade and the properties available in metric explorer.
If you don't see your type of app, choose one of the web types for web pages.
Resource group is a convenience for managing properties like access control. You could use separate
resource groups for development, test, and production.
Subscription is your payment account in Azure.
Location is where we keep your data. Currently it can't be changed.
Add to dashboard puts a quick-access tile for your resource on your Azure Home page.
Creating the resource takes a few seconds. You'll see an alert when it's done.
(You can write a PowerShell script to create a resource automatically.)
Getting the instrumentation key
The instrumentation key identifies the resource that you created.
You need the instrumentation keys of all the resources to which your app will send data.
There are several different methods of setting the Application Version property.
Set directly:
telemetryClient.Context.Component.Version = typeof(MyProject.MyClass).Assembly.GetName().Version;
Wrap that line in a telemetry initializer to ensure that all TelemetryClient instances are set consistently.
[ASP.NET] Set the version in BuildInfo.config . The web module will pick up the version from the
BuildLabel node. Include this file in your project and remember to set the Copy Always property in
Solution Explorer.
[ASP.NET] Generate BuildInfo.config automatically in MSBuild. To do this, add a few lines to your
.csproj file:
<PropertyGroup>
<GenerateBuildInfoConfigFile>true</GenerateBuildInfoConfigFile>
<IncludeServerNameInBuildInfo>true</IncludeServerNameInBuildInfo>
</PropertyGroup>
<PropertyGroup>
<GenerateBuildInfoConfigFile>true</GenerateBuildInfoConfigFile>
<IncludeServerNameInBuildInfo>true</IncludeServerNameInBuildInfo>
</PropertyGroup>
When it has the build info, the Application Insights web module automatically adds Application version as a
property to every item of telemetry. That allows you to filter by version when you perform diagnostic searches,
or when you explore metrics.
However, notice that the build version number is generated only by the Microsoft Build Engine, not by the
developer build from Visual Studio.
Release annotations
If you use Azure DevOps, you can get an annotation marker added to your charts whenever you release a new
version. The following image shows how this marker appears.
Next steps
Shared resources for multiple roles
Create a Telemetry Initializer to distinguish A|B variants
How do I ... in Application Insights?
11/12/2019 • 5 minutes to read • Edit Online
Your app might also show signs of strain by returning failure codes. Set an alert on Failed requests.
If you want to set an alert on Server exceptions, you might have to do some additional setup in order to see data.
Email on exceptions
1. Set up exception monitoring
2. Set an alert on the Exception count metric
Email on an event in my app
Let's suppose you'd like to get an email when a specific event occurs. Application Insights doesn't provide this
facility directly, but it can send an alert when a metric crosses a threshold.
Alerts can be set on custom metrics, though not custom events. Write some code to increase a metric when the
event occurs:
telemetry.TrackMetric("Alarm", 10);
or:
Because alerts have two states, you have to send a low value when you consider the alert to have ended:
telemetry.TrackMetric("Alarm", 0.5);
Now set an alert to fire when the metric goes above a mid value for a short period:
Visualize data
Dashboard with metrics from multiple apps
In Metric Explorer, customize your chart and save it as a favorite. Pin it to the Azure dashboard.
Dashboard with data from other sources and Application Insights
Export telemetry to Power BI.
Or
Use SharePoint as your dashboard, displaying data in SharePoint web parts. Use continuous export and Stream
Analytics to export to SQL. Use PowerView to examine the database, and create a SharePoint web part for
PowerView.
Filter out anonymous or authenticated users
If your users sign in, you can set the authenticated user ID. (It doesn't happen automatically.)
You can then:
Search on specific user IDs
Filter metrics to either anonymous or authenticated users
Disable telemetry
To dynamically stop and start the collection and transmission of telemetry from the server:
ASP.NET Classic applications
using Microsoft.ApplicationInsights.Extensibility;
TelemetryConfiguration.Active.DisableTelemetry = true;
Other applications
It is not recommended to use TelemetryConfiguration.Active singleton on console or ASP.NET Core applications.
if you created TelemetryConfiguration instance yourself - set DisableTelemetry to true .
For ASP.NET Core applications you may access TelemetryConfiguration instance using ASP.NET Core
dependency injection. Please find more details in ApplicationInsights for ASP.NET Core applications article.
Want to keep your telemetry for longer than the standard retention period? Or process it in some specialized
way? Continuous Export is ideal for this. The events you see in the Application Insights portal can be exported
to storage in Microsoft Azure in JSON format. From there, you can download your data and write whatever
code you need to process it.
Before you set up continuous export, there are some alternatives you might want to consider:
The Export button at the top of a metrics or search tab lets you transfer tables and charts to an Excel
spreadsheet.
Analytics provides a powerful query language for telemetry. It can also export results.
If you're looking to explore your data in Power BI, you can do that without using Continuous Export.
The Data access REST API lets you access your telemetry programmatically.
You can also access setup continuous export via Powershell.
After Continuous Export copies your data to storage (where it can stay for as long as you like), it's still available
in Application Insights for the usual retention period.
WARNING
By default, the storage location will be set to the same geographical region as your Application Insights resource.
If you store in a different region, you may incur transfer charges.
NOTE
Sampling. If your application sends a lot of data, the sampling feature may operate and send only a fraction of the
generated telemetry. Learn more about sampling.
$"{applicationName}_{instrumentationKey}/{type}/{blobDeliveryTimeUtc:yyyy-MM-dd}/{
blobDeliveryTimeUtc:HH}/{blobId}_{blobCreationTimeUtc:yyyyMMdd_HHmmss}.blob"
Where
blobCreationTimeUtc is time when blob was created in the internal staging storage
blobDeliveryTimeUtc is the time when blob is copied to the export destination storage
Data format
Each blob is a text file that contains multiple '\n'-separated rows. It contains the telemetry processed over a
time period of roughly half a minute.
Each row represents a telemetry data point such as a request or page view.
Each row is an unformatted JSON document. If you want to sit and stare at it, open it in Visual Studio and
choose Edit, Advanced, Format File:
Time durations are in ticks, where 10 000 ticks = 1 ms. For example, these values show a time of 1 ms to send a
request from the browser, 3 ms to receive it, and 1.8 s to process the page in the browser:
Detailed data model reference for the property types and values.
Export samples
Export to SQL using Stream Analytics
Stream Analytics sample 2
On larger scales, consider HDInsight - Hadoop clusters in the cloud. HDInsight provides a variety of
technologies for managing and analyzing big data, and you could use it to process data that has been exported
from Application Insights.
Q&A
But all I want is a one-time download of a chart.
Yes, you can do that. At the top of the tab, click Export Data.
I set up an export, but there's no data in my store.
Did Application Insights receive any telemetry from your app since you set up the export? You'll only
receive new data.
I tried to set up an export, but was denied access
If the account is owned by your organization, you have to be a member of the owners or contributors
groups.
Can I export straight to my own on-premises store?
No, sorry. Our export engine currently only works with Azure storage at this time.
Is there any limit to the amount of data you put in my store?
No. We'll keep pushing data in until you delete the export. We'll stop if we hit the outer limits for blob
storage, but that's pretty huge. It's up to you to control how much storage you use.
How many blobs should I see in the storage?
For every data type you selected to export, a new blob is created every minute (if data is available).
In addition, for applications with high traffic, additional partition units are allocated. In this case, each
unit creates a blob every minute.
I regenerated the key to my storage or changed the name of the container, and now the export doesn't
work.
Edit the export and open the export destination tab. Leave the same storage selected as before, and click
OK to confirm. Export will restart. If the change was within the past few days, you won't lose data.
Can I pause the export?
Yes. Click Disable.
Code samples
Stream Analytics sample
Export to SQL using Stream Analytics
Detailed data model reference for the property types and values.
Use Stream Analytics to process exported data from
Application Insights
10/20/2019 • 5 minutes to read • Edit Online
Azure Stream Analytics is the ideal tool for processing data exported from Application Insights. Stream Analytics
can pull data from a variety of sources. It can transform and filter the data, and then route it to a variety of sinks.
In this example, we'll create an adaptor that takes data from Application Insights, renames and processes some of
the fields, and pipes it into Power BI.
WARNING
There are much better and easier recommended ways to display Application Insights data in Power BI. The path illustrated
here is just an example to illustrate how to process exported data.
Make a note of the common part of the path name, which is derived from the application name and
instrumentation key.
The events are written to blob files in JSON format. Each file may contain one or more events. So we'd like to read
the event data and filter out the fields we want. There are all kinds of things we could do with the data, but our
plan today is to use Stream Analytics to pipe the data to Power BI.
webapplication27_12345678123412341234123456789abcdef0/PageViews/{date}/{time}
In this example:
webapplication27 is the name of the Application Insights resource all lower case.
1234... is the instrumentation key of the Application Insights resource, omitting dashes.
PageViews is the type of data you want to analyze. The available types depend on the filter you set in
Continuous Export. Examine the exported data to see the other available types, and see the export data model.
/{date}/{time} is a pattern written literally.
NOTE
Inspect the storage to make sure you get the path right.
Provide your work or school account to authorize Stream Analytics to access your Power BI resource. Then
invent a name for the output, and for the target Power BI dataset and table.
SELECT
flat.ArrayValue.name,
count(*)
INTO
[pbi-output]
FROM
[export-input] A
OUTER APPLY GetElements(A.[event]) as flat
GROUP BY TumblingWindow(minute, 1), flat.ArrayValue.name
SELECT
A.context.data.eventtime,
avg(CASE WHEN flat.arrayvalue.myMetric.value IS NULL THEN 0 ELSE flat.arrayvalue.myMetric.value END) as
myValue
INTO
[pbi-output]
FROM
[export-input] A
OUTER APPLY GetElements(A.context.custom.metrics) as flat
GROUP BY TumblingWindow(minute, 1), A.context.data.eventtime
This query drills into the metrics telemetry to get the event time and the metric value. The metric values are
inside an array, so we use the OUTER APPLY GetElements pattern to extract the rows. "myMetric" is the name
of the metric in this case.
Query to include values of dimension properties
WITH flat AS (
SELECT
MySource.context.data.eventTime as eventTime,
InstanceId = MyDimension.ArrayValue.InstanceId.value,
BusinessUnitId = MyDimension.ArrayValue.BusinessUnitId.value
FROM MySource
OUTER APPLY GetArrayElements(MySource.context.custom.dimensions) MyDimension
)
SELECT
eventTime,
InstanceId,
BusinessUnitId
INTO AIOutput
FROM flat
This query includes values of the dimension properties without depending on a particular dimension being at a
fixed index in the dimension array.
Run the job
You can select a date in the past to start the job from.
Open Power BI with your work or school account, and select the dataset and table that you defined as the output
of the Stream Analytics job.
Now you can use this dataset in reports and dashboards in Power BI.
No data?
Check that you set the date format correctly to YYYY -MM -DD (with dashes).
Video
Noam Ben Zeev shows how to process exported data using Stream Analytics.
Next steps
Continuous export
Detailed data model reference for the property types and values.
Application Insights
Walkthrough: Export to SQL from Application
Insights using Stream Analytics
10/20/2019 • 6 minutes to read • Edit Online
This article shows how to move your telemetry data from Azure Application Insights into an Azure SQL database
by using Continuous Export and Azure Stream Analytics.
Continuous export moves your telemetry data into Azure Storage in JSON format. We'll parse the JSON objects
using Azure Stream Analytics and create rows in a database table.
(More generally, Continuous Export is the way to do your own analysis of the telemetry your apps send to
Application Insights. You could adapt this code sample to do other things with the exported telemetry, such as
aggregation of data.)
We'll start with the assumption that you already have the app you want to monitor.
In this example, we will be using the page view data, but the same pattern can easily be extended to other data
types such as custom events and exceptions.
3. Let some data accumulate. Sit back and let people use your application for a while. Telemetry will come in
and you'll see statistical charts in metric explorer and individual events in diagnostic search.
And also, the data will export to your storage.
4. Inspect the exported data, either in the portal - choose Browse, select your storage account, and then
Containers - or in Visual Studio. In Visual Studio, choose View / Cloud Explorer, and open Azure /
Storage. (If you don't have this menu option, you need to install the Azure SDK: Open the New Project
dialog and open Visual C# / Cloud / Get Microsoft Azure SDK for .NET.)
Make a note of the common part of the path name, which is derived from the application name and
instrumentation key.
The events are written to blob files in JSON format. Each file may contain one or more events. So we'd like to
read the event data and filter out the fields we want. There are all kinds of things we could do with the data, but
our plan today is to use Stream Analytics to move the data to a SQL database. That will make it easy to run lots of
interesting queries.
Make sure that the database server allows access to Azure services:
webapplication27_12345678123412341234123456789abcdef0/PageViews/{date}/{time}
In this example:
webapplication27 is the name of the Application Insights resource, all in lower case.
1234... is the instrumentation key of the Application Insights resource with dashes removed.
PageViews is the type of data we want to analyze. The available types depend on the filter you set in
Continuous Export. Examine the exported data to see the other available types, and see the export data model.
/{date}/{time} is a pattern written literally.
To get the name and iKey of your Application Insights resource, open Essentials on its overview page, or open
Settings.
TIP
Use the Sample function to check that you have set the input path correctly. If it fails: Check that there is data in the
storage for the sample time range you chose. Edit the input definition and check you set the storage account, path prefix
and date format correctly.
Set query
Open the query section:
Replace the default query with:
Notice that the first few properties are specific to page view data. Exports of other telemetry types will have
different properties. See the detailed data model reference for the property types and values.
Close the wizard and wait for a notification that the output has been set up.
Start processing
Start the job from the action bar:
You can choose whether to start processing the data starting from now, or to start with earlier data. The latter is
useful if you have had Continuous Export already running for a while.
After a few minutes, go back to SQL Server Management Tools and watch the data flowing in. For example, use a
query like this:
Related articles
Export to PowerBI using Stream Analytics
Detailed data model reference for the property types and values.
Continuous Export in Application Insights
Application Insights
Application Insights Export Data Model
10/20/2019 • 7 minutes to read • Edit Online
This table lists the properties of telemetry sent from the Application Insights SDKs to the portal. You'll see these
properties in data output from Continuous Export. They also appear in property filters in Metric Explorer and
Diagnostic Search.
Points to note:
[0] in these tables denotes a point in the path where you have to insert an index; but it isn't always 0.
Time durations are in tenths of a microsecond, so 10000000 == 1 second.
Dates and times are UTC, and are given in the ISO format yyyy-MM-DDThh:mm:ss.sssZ
Example
// A server report about an HTTP request
{
"request": [
{
"urlData": { // derived from 'url'
"host": "contoso.org",
"base": "/",
"hashTag": ""
},
"responseCode": 200, // Sent to client
"success": true, // Default == responseCode<400
// Request id becomes the operation id of child events
"id": "fCOhCdCnZ9I=",
"name": "GET Home/Index",
"count": 1, // 100% / sampling rate
"durationMetric": {
"value": 1046804.0, // 10000000 == 1 second
// Currently the following fields are redundant:
"count": 1.0,
"min": 1046804.0,
"max": 1046804.0,
"stdDev": 0.0,
"sampledValue": 1046804.0
},
"url": "/"
}
],
"internal": {
"data": {
"id": "7f156650-ef4c-11e5-8453-3f984b167d05",
"documentVersion": "1.61"
}
},
"context": {
"device": { // client browser
"type": "PC",
"screenResolution": { },
"roleInstance": "WFWEB14B.fabrikam.net"
},
"application": { },
"location": { // derived from client ip
"continent": "North America",
"country": "United States",
// last octagon is anonymized to 0 at portal:
"clientip": "168.62.177.0",
"clientip": "168.62.177.0",
"province": "",
"city": ""
},
"data": {
"isSynthetic": true, // we identified source as a bot
// percentage of generated data sent to portal:
"samplingRate": 100.0,
"eventTime": "2016-03-21T10:05:45.7334717Z" // UTC
},
"user": {
"isAuthenticated": false,
"anonId": "us-tx-sn1-azr", // bot agent id
"anonAcquisitionDate": "0001-01-01T00:00:00Z",
"authAcquisitionDate": "0001-01-01T00:00:00Z",
"accountAcquisitionDate": "0001-01-01T00:00:00Z"
},
"operation": {
"id": "fCOhCdCnZ9I=",
"parentId": "fCOhCdCnZ9I=",
"name": "GET Home/Index"
},
"cloud": { },
"serverDevice": { },
"custom": { // set by custom fields of track calls
"dimensions": [ ],
"metrics": [ ]
},
"session": {
"id": "65504c10-44a6-489e-b9dc-94184eb00d86",
"isFirst": true
}
}
Context
All types of telemetry are accompanied by a context section. Not all of these fields are transmitted with every
data point.
context.device.deviceModel string
context.device.deviceName string
context.device.id string
context.device.network string
context.device.oemName string
context.device.os string
context.device.roleName string
context.device.screenResolution string
context.location.continent string
context.location.country string
context.session.isFirst boolean
context.user.accountAcquisitionDate string
context.user.accountId string
context.user.anonAcquisitionDate string
context.user.anonId string
context.user.authId string
context.user.isAuthenticated boolean
context.user.storeRegion string
internal.data.documentVersion string
Events
Custom events generated by TrackEvent().
Exceptions
Reports exceptions in the server and in the browser.
PATH TYPE NOTES
Trace Messages
Sent by TrackTrace, and by the logging adapters.
message [0] raw string The log message, max length 10k.
Remote dependency
Sent by TrackDependency. Used to report performance and usage of calls to dependencies in the server, and
AJAX calls in the browser.
Requests
Sent by TrackRequest. The standard modules use this to reports server response time, measured at the server.
request [0] name string GET/POST + url base. Max length 250
clientPerformance [0] total.value integer Time from starting to send the request
to displaying the page.
Page Views
Sent by trackPageView () or stopTrackPage
Availability
Reports availability web tests.
Metrics
Generated by TrackMetric().
The metric value is found in context.custom.metrics[0]
For example:
{
"metric": [ ],
"context": {
...
"custom": {
"dimensions": [
{ "ProcessId": "4068" }
],
"metrics": [
{
"dispatchRate": {
"value": 0.001295,
"count": 1.0,
"min": 0.001295,
"max": 0.001295,
"stdDev": 0.0,
"sampledValue": 0.001295,
"sum": 0.001295
}
}
} ] }
}
About metric values
Metric values, both in metric reports and elsewhere, are reported with a standard object structure. For example:
"durationMetric": {
"name": "contoso.org",
"type": "Aggregation",
"value": 468.71603053650279,
"count": 1.0,
"min": 468.71603053650279,
"max": 468.71603053650279,
"stdDev": 0.0,
"sampledValue": 468.71603053650279
}
Currently - though this might change in the future - in all values reported from the standard SDK modules,
count==1 and only the name and value fields are useful. The only case where they would be different would be
if you write your own TrackMetric calls in which you set the other parameters.
The purpose of the other fields is to allow metrics to be aggregated in the SDK, to reduce traffic to the portal. For
example, you could average several successive readings before sending each metric report. Then you would
calculate the min, max, standard deviation and aggregate value (sum or average) and set count to the number of
readings represented by the report.
In the tables above, we have omitted the rarely-used fields count, min, max, stdDev and sampledValue.
Instead of pre-aggregating metrics, you can use sampling if you need to reduce the volume of telemetry.
Durations
Except where otherwise noted, durations are represented in tenths of a microsecond, so that 10000000.0 means
1 second.
See also
Application Insights
Continuous Export
Code samples
Debug your applications with Azure Application
Insights in Visual Studio
12/8/2019 • 3 minutes to read • Edit Online
In Visual Studio (2015 and later), you can analyze performance and diagnose issues in your ASP.NET web app
both in debugging and in production, using telemetry from Azure Application Insights.
If you created your ASP.NET web app using Visual Studio 2017 or later, it already has the Application Insights
SDK. Otherwise, if you haven't done so already, add Application Insights to your app.
To monitor your app when it's in live production, you normally view the Application Insights telemetry in the
Azure portal, where you can set alerts and apply powerful monitoring tools. But for debugging, you can also
search and analyze the telemetry in Visual Studio. You can use Visual Studio to analyze telemetry both from your
production site and from debugging runs on your development machine. In the latter case, you can analyze
debugging runs even if you haven't yet configured the SDK to send telemetry to the Azure portal.
The free text search works on any fields in the events. For example, search for part of the URL of a page; or the
value of a property such as client city; or specific words in a trace log.
Click any event to see its detailed properties.
For requests to your web app, you can click through to the code.
You can also open related items to help diagnose failed requests or exceptions.
View exceptions and failed requests
Exception reports show in the Search window. (In some older types of ASP.NET application, you have to set up
exception monitoring to see exceptions that are handled by the framework.)
Click an exception to get a stack trace. If the code of the app is open in Visual Studio, you can click through from
the stack trace to the relevant line of the code.
Trends
Trends is a tool for visualizing how your app behaves over time.
Choose Explore Telemetry Trends from the Application Insights toolbar button or Application Insights Search
window. Choose one of five common queries to get started. You can analyze different datasets based on telemetry
types, time ranges, and other properties.
To find anomalies in your data, choose one of the anomaly options under the "View Type" dropdown. The filtering
options at the bottom of the window make it easy to hone in on specific subsets of your telemetry.
Local monitoring
(From Visual Studio 2015 Update 2) If you haven't configured the SDK to send telemetry to the Application
Insights portal (so that there is no instrumentation key in ApplicationInsights.config) then the diagnostics window
displays telemetry from your latest debugging session.
This is desirable if you have already published a previous version of your app. You don't want the telemetry from
your debugging sessions to be mixed up with the telemetry on the Application Insights portal from the published
app.
It's also useful if you have some custom telemetry that you want to debug before sending telemetry to the portal.
At first, I fully configured Application Insights to send telemetry to the portal. But now I'd like to see the
telemetry only in Visual Studio.
In the Search window's Settings, there's an option to search local diagnostics even if your app sends
telemetry to the portal.
To stop telemetry being sent to the portal, comment out the line <instrumentationkey>... from
ApplicationInsights.config. When you're ready to send telemetry to the portal again, uncomment it.
Next steps
The Application Insights Trends tool visualizes how your web application's important telemetry events change
over time, helping you quickly identify problems and anomalies. By linking you to more detailed diagnostic
information, Trends can help you improve your app's performance, track down the causes of exceptions, and
uncover insights from your custom events.
Filter
Discover more specific trends with the filter controls at the bottom of the window. To apply a filter, click on its
name. You can quickly switch between different filters to discover trends that may be hiding in a particular
dimension of your telemetry. If you apply a filter in one dimension, like Exception Type, filters in other dimensions
remain clickable even though they appear grayed-out. To un-apply a filter, click it again. Ctrl-click to select multiple
filters in the same dimension.
Find anomalies
The Trends tool can highlight bubbles of events that are anomalous compared to other bubbles in the same time
series. In the View Type dropdown, choose Counts in time bucket (highlight anomalies) or Percentages in
time bucket (highlight anomalies). Red bubbles are anomalous. Anomalies are defined as bubbles with
counts/percentages exceeding 2.1 times the standard deviation of the counts/percentages that occurred in the past
two time periods (48 hours if you're viewing the last 24 hours, etc.).
TIP
Highlighting anomalies is especially helpful for finding outliers in time series of small bubbles that may otherwise look
similarly sized.
Next steps
Methods in the code of your web app can be annotated with telemetry about run-time exceptions and request
response times. If you install Azure Application Insights in your application, the telemetry appears in Visual Studio
CodeLens - the notes at the top of each function where you're used to seeing useful information such as the
number of places the function is referenced or the last person who edited it.
NOTE
Application Insights in CodeLens is available in Visual Studio 2015 Update 3 and later, or with the latest version of Developer
Analytics Tools extension. CodeLens is available in the Enterprise and Professional editions of Visual Studio.
TIP
Application Insights request and exception indicators may take a few extra seconds to load after other CodeLens indicators
appear.
Exceptions in CodeLens
The exception CodeLens indicator shows the number of exceptions that have occurred in the past 24 hours from
the 15 most frequently occurring exceptions in your application during that period, while processing the request
served by the method.
To see more details, click the exceptions CodeLens indicator:
The percentage change in number of exceptions from the most recent 24 hours relative to the prior 24 hours
Choose Go to code to navigate to the source code for the function throwing the exception
Choose Search to query all instances of this exception that have occurred in the past 24 hours
Choose Trend to view a trend visualization for occurrences of this exception in the past 24 hours
Choose View all exceptions in this app to query all exceptions that have occurred in the past 24 hours
Choose Explore exception trends to view a trend visualization for all exceptions that have occurred in the
past 24 hours.
TIP
If you see "0 exceptions" in CodeLens but you know there should be exceptions, check to make sure the right Application
Insights resource is selected in CodeLens. To select another resource, right-click on your project in the Solution Explorer and
choose Application Insights > Choose Telemetry Source. CodeLens is only shown for the 15 most frequently occurring
exceptions in your application in the past 24 hours, so if an exception is the 16th most frequently or less, you'll see "0
exceptions." Exceptions from ASP.NET views may not appear on the controller methods that generated those views.
TIP
If you see "? exceptions" in CodeLens, you need to associate your Azure account with Visual Studio or your Azure account
credential may have expired. In either case, click "? exceptions" and choose Add an account... to enter your credentials.
Requests in CodeLens
The request CodeLens indicator shows the number of HTTP requests that been serviced by a method in the past
24 hours, plus the percentage of those requests that failed.
To see more details, click the requests CodeLens indicator:
The absolute and percentage changes in number of requests, failed requests, and average response times over
the past 24 hours compared to the prior 24 hours
The reliability of the method, calculated as the percentage of requests that did not fail in the past 24 hours
Choose Search for requests or failed requests to query all the (failed) requests that occurred in the past 24
hours
Choose Trend to view a trend visualization for requests, failed requests, or average response times in the past
24 hours.
Choose the name of the Application Insights resource in the upper left corner of the CodeLens details view to
change which resource is the source for CodeLens data.
Next steps
Working with Application Insights in Visual Studio
Search telemetry, see data in CodeLens, and configure
Application Insights. All within Visual Studio.
Azure Application Insights monitors the availability, performance and usage of your apps. Here you'll learn how to
set it up for a SharePoint site.
The window that opens is the place where you'll see performance and usage data about your app. To get back to it
next time you login to Azure, you should find a tile for it on the start screen. Alternatively click Browse to find it.
window.appInsights=appInsights,appInsights.queue&&0===appInsights.queue.length&&appInsights.trackPageView();
</script>
Insert the script just before the </head> tag of every page you want to track. If your website has a master page,
you can put the script there. For example, in an ASP.NET MVC project, you'd put it in View\Shared_Layout.cshtml
The script contains the instrumentation key that directs the telemetry to your Application Insights resource.
Add the code to your site pages
On the master page
If you can edit the site's master page, that will provide monitoring for every page in the site.
Check out the master page and edit it using SharePoint Designer or any other editor.
Capturing User Id
The standard web page code snippet doesn't capture the user id from SharePoint, but you can do that with a small
modification.
1. Copy your app's instrumentation key from the Essentials drop-down in Application Insights.
<script type="text/javascript">
var personProperties;
// Ensure that the SP.UserProfiles.js file is loaded before the custom code runs.
SP.SOD.executeOrDelayUntilScriptLoaded(getUserProperties, 'SP.UserProfiles.js');
function getUserProperties() {
// Get the current client context and PeopleManager instance.
var clientContext = new SP.ClientContext.get_current();
var peopleManager = new SP.UserProfiles.PeopleManager(clientContext);
personProperties = peopleManager.getMyProperties();
Next Steps
Web tests to monitor the availability of your site.
Application Insights for other types of app.
Monitoring usage and performance in Classic
Windows Desktop apps
10/29/2019 • 2 minutes to read • Edit Online
Applications hosted on premises, in Azure, and in other clouds can all take advantage of Application Insights. The
only limitation is the need to allow communication to the Application Insights service. For monitoring Universal
Windows Platform (UWP ) applications, we recommend Visual Studio App Center.
If you use ApplicationInsights.config, make sure its properties in Solution Explorer are set to Build Action
= Content, Copy to Output Directory = Copy.
5. Use the API to send telemetry.
6. Run your app, and see the telemetry in the resource you created in the Azure portal.
Example code
using Microsoft.ApplicationInsights;
if (tc != null)
{
tc.Flush(); // only for desktop apps
using Microsoft.ApplicationInsights.Channel;
using Microsoft.ApplicationInsights.Extensibility;
namespace CustomInitializer.Telemetry
{
public class MyTelemetryInitializer : ITelemetryInitializer
{
public void Initialize(ITelemetry telemetry)
{
if (string.IsNullOrEmpty(telemetry.Context.Cloud.RoleName))
{
//set custom role name here, you can pass an empty string if needed.
telemetry.Context.Cloud.RoleInstance = "Custom RoleInstance";
}
}
}
}
Instantiate the initializer in the Program.cs Main() method below setting the instrumentation key:
using Microsoft.ApplicationInsights.Extensibility;
using CustomInitializer.Telemetry;
Next steps
Create a dashboard
Diagnostic Search
Explore metrics
Write Analytics queries
Monitoring Azure resources with Azure Monitor
1/8/2020 • 9 minutes to read • Edit Online
When you have critical applications and business processes relying on Azure resources, you want to monitor those
resources for their availability, performance, and operation. This article describes the monitoring data generated by
Azure resources and how you can use the features of Azure Monitor to analyze and alert on this data.
IMPORTANT
This article applies to all services in Azure that use Azure Monitor. Compute resources, including VMs and App Service,
generate the same monitoring data described here but also have a guest operating system that may also generate logs and
metrics. See the monitoring documentation for these services for details on how to collect and analyze this data.
Monitoring data
Resources in Azure generate logs and metrics shown the following diagram. Refer to the documentation for each
Azure services for the specific data they generate and any additional solutions or insights they provide.
Platform metrics - Numerical values that are automatically collected at regular intervals and describe some
aspect of a resource at a particular time.
Resource logs - Provide insight into operations that were performed within an Azure resource (the data plane),
for example getting a secret from a Key Vault or making a request to a database. The content and structure of
resource logs varies by the Azure service and resource type.
Activity log - Provides insight into the operations on each Azure resource in the subscription from the outside
(the management plane), for example creating a new resource or starting a virtual machine. This is information
about the what, who, and when for any write operations (PUT, POST, DELETE ) taken on the resources in your
subscription.
Configuration requirements
Configure monitoring
Some monitoring data is collected automatically, but you may need to perform some configuration depending on
your requirements. See the information below for specific information for each type of monitoring data.
Platform metrics - Platform metrics are collected automatically into Azure Monitor Metrics with no
configuration required. Create a diagnostic setting to send entries to Azure Monitor Logs or to forward them
outside of Azure.
Resource logs - Resource logs are automatically generated by Azure resources but not collected without a
diagnostic setting. Create a diagnostic setting to send entries to Azure Monitor Logs or to forward them
outside of Azure.
Activity log - The Activity log is collected automatically with no configuration required and can be view in the
Azure portal. Create a diagnostic setting to copy them to Azure Monitor Logs or to forward them outside of
Azure.
Log Analytics workspace
Collecting data into Azure Monitor Logs requires a Log Analytics workspace. You can start monitoring your
service quickly by creating a new workspace, but there may be value in using a workspace that's collecting data
from other services. See Create a Log Analytics workspace in the Azure portal for details on creating a workspace
and Designing your Azure Monitor Logs deployment to help determine the best workspace design for your
requirements. If you use an existing workspace in your organization, then you will require appropriate permissions
as described in Manage access to log data and workspaces in Azure Monitor.
Diagnostic settings
Diagnostic settings define where resource logs and metrics for a particular resource should be sent. Possible
destinations are:
Log Analytics workspace which allows you to analyze data with other monitoring data collected by Azure
Monitor using powerful log queries and also to leverage other Azure Monitor features such as log alerts and
visualizations.
Event hubs to stream data to external systems such as third-party SIEMs and other log analytics solutions.
Azure storage account which is useful for audit, static analysis, or backup.
Follow the procedure in Create diagnostic setting to collect platform logs and metrics in Azure to create and
manage diagnostic settings through the Azure portal. See Create diagnostic setting in Azure using a Resource
Manager template to define them in a template and enable complete monitoring for a resource when it's created.
Metrics
Analyze metrics in the Azure portal using metrics explorer which is available from the Metrics menu item for most
services. This tool allows you to work with individual metrics or combine multiple to identify correlations and
trends.
See Getting started with Azure Metrics Explorer for the basics of using metrics explorer.
See Advanced features of Azure Metrics Explorer for advanced features of metrics explorer such as using
multiple metrics and applying filters and splitting.
Activity log
View entries in the activity log in the Azure portal with the initial filter set to the current resource. Copy the activity
log to a Log Analytics workspace to access it to use it in log queries and workbooks.
See View and retrieve Azure Activity log events for details on viewing the Activity log and retrieving entries
using a variety of methods.
See the documentation for your Azure service for the specific events that get logged.
Next steps
See Supported services, schemas, and categories for Azure Resource Logs for details of resource logs for
different Azure services.
Monitor resource groups with Azure Monitor
(preview)
10/17/2019 • 4 minutes to read • Edit Online
Modern applications are often complex and highly distributed with many discrete parts working together to
deliver a service. Recognizing this complexity, Azure Monitor provides monitoring insights for resource groups.
This makes it easy to triage and diagnose any problems your individual resources encounter, while offering
context as to the health and performance of the resource group—and your application—as a whole.
By default, the resources are grouped by app layer and resource type. App layer is a simple categorization of
resource types, that only exists within the context of the resource group insights overview page. There are
resource types related to application code, compute infrastructure, networking, storage + databases. Management
tools get their own app layers, and every other resource is categorized as belonging to the Other app layer. This
grouping can help you see at-a-glance what subsystems of your application are healthy and unhealthy.
When App Service is chosen, you are presented with a gallery of Azure Monitor Workbook templates.
Choosing the template for Failure Insights will open the workbook.
You can select any of the rows. The selection is then displayed in a graphical details view.
Workbooks abstract away the difficult work of creating custom reports and visualizations into an easily
consumable format. While some users may only want to adjust the prebuilt parameters, workbooks are
completely customizable.
To get a sense of how this workbook functions internally, select Edit in the top bar.
A number of Edit boxes appear near the various elements of the workbook. Select the Edit box below the table of
operations.
This reveals the underlying log query that is driving the table visualization.
You can modify the query directly. Or you can use it as a reference and borrow from it when designing your own
custom parameterized workbook.
Investigate performance
Performance offers its own gallery of workbooks. For App Service the prebuilt Application Performance workbook
offers the following view:
In this case, if you select edit you will see that this set of visualizations is powered by Azure Monitor Metrics.
Troubleshooting
Enabling access to alerts
To see alerts in Azure Monitor for Resource Groups, someone with an Owner or Contributor role for this
subscription needs to open Azure Monitor for Resource Groups for any resource group in the subscription. This
will enable anyone with read access to see alerts in Azure Monitor for Resource Groups for all of the resource
groups in the subscription. If you have an Owner or Contributor role, refresh this page in a few minutes.
Azure Monitor for Resource Groups relies on the Azure Monitor Alerts Management system to retrieve alert
status. Alerts Management isn't configured for every resource group and subscription by default, and it can only
be enabled by someone with an Owner or Contributor role. It can be enabled either by:
Opening Azure Monitor for Resource Groups for any resource group in the subscription.
Or by going to the subscription, clicking Resource Providers, then clicking Register for
Alerts.Management.
Next steps
Azure Monitor Workbooks
Azure Resource Health
Azure Monitor Alerts
Overview of Azure platform logs
12/27/2019 • 2 minutes to read • Edit Online
Platform logs provide detailed diagnostic and auditing information for Azure resources and the Azure
platform they depend on. They are automatically generated although you need to configure certain platform
logs to be forwarded to one or more destinations to be retained. This article provides an overview of
platform logs including what information they provide and how you can configure them for collection and
analysis.
Azure Active Directory logs Azure Tenant Contains the history of sign-in
activity and audit trail of changes
made in the Azure Active Directory
for a particular tenant. See What are
Azure Active Directory reports? for a
complete description of Azure Active
Directory Logs.
NOTE
The Azure Activity Log is primarily for activities that occur in Azure Resource Manager. It does not track resources
using the Classic/RDFE model. Some Classic resource types have a proxy resource provider in Azure Resource
Manager (for example, Microsoft.ClassicCompute). If you interact with a Classic resource type through Azure
Resource Manager using these proxy resource providers, the operations appear in the Activity Log. If you interact
with a Classic resource type outside of the Azure Resource Manager proxies, your actions are only recorded in the
Operation Log. The Operation Log can be browsed in a separate section of the portal.
Destinations
You can send platform logs to one or more of the destinations in the following table depending on your
monitoring requirements. Configure destinations for platform logs by creating a Diagnostic setting.
Log Analytics workspace Analyze the logs with other Activity log and Resource logs
monitoring data and leverage Azure Azure Activity Directory logs
Monitor features such as log queries
and alerts.
Azure storage Archive the logs for audit, static Activity log and Resource logs
analysis, or backup. Azure Activity Directory logs
Event hub Stream the logs to third-party Activity log and Resource logs
logging and telemetry systems. Azure Activity Directory logs
Next steps
View the Activity log schema for different categories
View the resource log schema for different Azure services
View and retrieve Azure Activity log events
1/8/2020 • 7 minutes to read • Edit Online
The Azure Activity Log provides insight into subscription-level events that have occurred in Azure. This article
provides details on different methods for viewing and retrieving Activity Log events.
Azure portal
View the Activity Log for all resources from the Monitor menu in the Azure portal. View the Activity Log for a
particular resource from the Activity Log option in that resource's menu.
Administrative Contains the record of all create, update, delete, and action
operations performed through Resource Manager. Examples
of Administrative events include create virtual machine and
delete network security group.
Service Health Contains the record of any service health incidents that have
occurred in Azure. An example of a Service Health event SQL
Azure in East US is experiencing downtime.
Resource Health Contains the record of any resource health events that have
occurred to your Azure resources. An example of a Resource
Health event is Virtual Machine health status changed to
unavailable.
PowerShell
Use the Get-AzLog cmdlet to retrieve the Activity Log from PowerShell. Following are some common examples.
NOTE
Get-AzLog only provides 15 days of history. Use the -MaxEvents parameter to query the last N events beyond 15 days.
To access events older than 15 days, use the REST API or SDK. If you do not include StartTime, then the default value is
EndTime minus one hour. If you do not include EndTime, then the default value is current time. All times are in UTC.
Get log entries from a specific resource provider between a date time range:
CLI
Use az monitor activity-log to retrieve the Activity Log from CLI. Following are some common examples.
View all available options.
REST API
Use the Azure Monitor REST API to retrieve the Activity Log from a REST client. Following are some common
examples.
Get Activity Logs with filter:
GET https://management.azure.com/subscriptions/089bd33f-d4ec-47fe-8ba5-
0753aa5c5b33/providers/microsoft.insights/eventtypes/management/values?api-version=2015-04-
01&$filter=eventTimestamp ge '2018-01-21T20:00:00Z' and eventTimestamp le '2018-01-23T20:00:00Z' and
resourceGroupName eq 'MSSupportGroup'
GET https://management.azure.com/subscriptions/089bd33f-d4ec-47fe-8ba5-
0753aa5c5b33/providers/microsoft.insights/eventtypes/management/values?api-version=2015-04-
01&$filter=eventTimestamp ge '2015-01-21T20:00:00Z' and eventTimestamp le '2015-01-23T20:00:00Z' and
resourceGroupName eq
'MSSupportGroup'&$select=eventName,id,resourceGroupName,resourceProviderName,operationName,status,eventTimesta
mp,correlationId,submissionTimestamp,level
GET https://management.azure.com/subscriptions/089bd33f-d4ec-47fe-8ba5-
0753aa5c5b33/providers/microsoft.insights/eventtypes/management/values?api-version=2015-04-
01&$select=eventName,id,resourceGroupName,resourceProviderName,operationName,status,eventTimestamp,correlation
Id,submissionTimestamp,level
GET https://management.azure.com/subscriptions/089bd33f-d4ec-47fe-8ba5-
0753aa5c5b33/providers/microsoft.insights/eventtypes/management/values?api-version=2015-04-01
Azure Activity Log Entries Shows a bar chart of the top Azure Activity Log entry record
totals for the date range that you have selected and shows a
list of the top 10 activity callers. Click the bar chart to run a
log search for AzureActivity . Click a caller item to run a log
search returning all Activity Log entries for that item.
Activity Logs by Status Shows a doughnut chart for Azure Activity Log status for the
selected date range and a list of the top ten status records.
Click the chart to run a log query for
AzureActivity | summarize AggregatedValue = count()
by ActivityStatus
. Click a status item to run a log search returning all Activity
Log entries for that status record.
Activity Logs by Resource Shows the total number of resources with Activity Logs and
lists the top ten resources with record counts for each
resource. Click the total area to run a log search for
AzureActivity | summarize AggregatedValue = count()
by Resource
, which shows all Azure resources available to the solution.
Click a resource to run a log query returning all activity
records for that resource.
VISUALIZATION PART DESCRIPTION
Activity Logs by Resource Provider Shows the total number of resource providers that produce
Activity Logs and lists the top ten. Click the total area to run a
log query for
AzureActivity | summarize AggregatedValue = count()
by ResourceProvider
, which shows all Azure resource providers. Click a resource
provider to run a log query returning all activity records for
the provider.
Next steps
Read an overview of platform logs
Create diagnostic setting to send Activity logs to other destinations
Collect Azure platform logs in Log Analytics
workspace in Azure Monitor
1/8/2020 • 5 minutes to read • Edit Online
Platform logs in Azure, including Azure Activity log and resource logs, provide detailed diagnostic and auditing
information for Azure resources and the Azure platform they depend on. This article describes collecting resource
logs in a Log Analytics workspace which allows you to analyze it with other monitoring data collected in Azure
Monitor Logs using powerful log queries and also to leverage other Azure Monitor features such as alerts and
visualizations.
Prerequisites
You need to create a new workspace if you don't already have one. The workspace does not have to be in the
same subscription as the resource sending logs as long as the user who configures the setting has appropriate
RBAC access to both subscriptions.
RESOU
RCEPRO CATEG
VIDER ORY A B C D E F G H I
Micros AuditL x1 y1 z1
oft.Ser ogs
vice1
Micros ErrorLo q1 w1 e1
oft.Ser gs
vice1
Micros AuditL j1 k1 l1
oft.Ser ogs
vice2
Micros ErrorLo q2 w2 e2
oft.Ser gs
vice1
Micros AuditL j3 k3 l3
oft.Ser ogs
vice2
Micros AuditL x5 y5 z5
oft.Ser ogs
vice1
...
Resource -Specific
In this mode, individual tables in the selected workspace are created for each category selected in the diagnostic
setting. This method is recommended since it makes it much easier to work with the data in log queries, provides
better discoverability of schemas and their structure, improves performance across both ingestion latency and
query times, and the ability to grant RBAC rights on a specific table. All Azure services will eventually migrate to
the Resource-Specific mode.
The example above would result in three tables being created:
Table Service1AuditLogs as follows:
RESOURCE PROVIDER CATEGORY A B C
Service1 AuditLogs x1 y1 z1
Service1 AuditLogs x5 y5 z5
...
Service1 ErrorLogs q1 w1 e1
Service1 ErrorLogs q2 w2 e2
...
Service2 AuditLogs j1 k1 l1
Service2 AuditLogs j3 k3 l3
...
You can modify an existing diagnostic setting to resource-specific mode. In this case, data that was already
collected will remain in the AzureDiagnostics table until it's removed according to your retention setting for the
workspace. New data will be collected in to the dedicated table. Use the union operator to query data across both
tables.
Continue to watch Azure Updates blog for announcements about Azure services supporting Resource-Specific
mode.
Column limit in AzureDiagnostics
There is a 500 property limit for any table in Azure Monitor Logs. Once this limit is reached, any rows containing
data with any property outside of the first 500 will be dropped at ingestion time. The AzureDiagnostics table is in
particular susceptible to this limit since it includes properties for all Azure services writing to it.
If you're collecting resource logs from multiple services, AzureDiagnostics may exceed this limit, and data will be
missed. Until all Azure services support resource-specific mode, you should configure resources to write to
multiple workspaces to reduce the possibility of reaching the 500 column limit.
Azure Data Factory
Azure Data Factory, because of a very detailed set of logs, is a service that is known to write a large number of
columns and potentially cause AzureDiagnostics to exceed its limit. For any diagnostic settings configured before
the resource-specific mode was enabled there will be a new column created for every uniquely-named user
parameter against any activity. More columns will be created because of the verbose nature of activity inputs and
outputs.
You should migrate your logs to use the resource-specific mode as soon as possible. If you are unable to do so
immediately, an interim alternative is to isolate Azure Data Factory logs into their own workspace to minimize the
chance of these logs impacting other log types being collected in your workspaces.
Next steps
Read more about resource logs.
Create diagnostic setting to collect logs and metrics in Azure.
Archive Azure resource logs to storage account
1/14/2020 • 2 minutes to read • Edit Online
Platform logs in Azure, including Azure Activity log and resource logs, provide detailed diagnostic and auditing
information for Azure resources and the Azure platform they depend on. This article describes collecting
platform logs to an Azure storage account to retain data for archiving.
Prerequisites
You need to create an Azure storage account if you don't already have one. The storage account does not have to
be in the same subscription as the resource sending logs as long as the user who configures the setting has
appropriate RBAC access to both subscriptions.
IMPORTANT
Azure Data Lake Storage Gen2 accounts are not currently supported as a destination for diagnostic settings even though
they may be listed as a valid option in the Azure portal.
You should not use an existing storage account that has other, non-monitoring data stored in it so that you can
better control access to the data. If you are archiving the Activity log and resource logs together though, you may
choose to use the same storage account to keep all monitoring data in a central location.
For example, the blob for a network security group might have a name similar to the following:
insights-logs-networksecuritygrouprulecounter/resourceId=/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/RESOURCEGROUPS/TESTRESOURCEGROUP/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUP/TESTNSG/y=2016
/m=08/d=22/h=18/m=00/PT1H.json
Each PT1H.json blob contains a JSON blob of events that occurred within the hour specified in the blob URL (for
example, h=12). During the present hour, events are appended to the PT1H.json file as they occur. The minute
value (m=00) is always 00, since resource log events are broken into individual blobs per hour.
Within the PT1H.json file, each event is stored with the following format. This will use a common top level
schema but be unique for each Azure services as described in Resource logs schema and Activity log schema.
NOTE
Platform logs are written to blob storage using JSON lines, where each event is a line and the newline character indicates a
new event. This format was implemented in November 2018. Prior to this date, logs were written to blob storage as a json
array of records as described in Prepare for format change to Azure Monitor platform logs archived to a storage account.
Next steps
Read more about resource logs.
Create diagnostic setting to collect logs and metrics in Azure.
Download blobs for analysis.
Archive Azure Active Directory logs with Azure Monitor.
Stream Azure platform logs to Azure Event Hubs
1/8/2020 • 3 minutes to read • Edit Online
Platform logs in Azure, including Azure Activity log and resource logs, provide detailed diagnostic and auditing
information for Azure resources and the Azure platform they depend on. This article describes streaming
platform logs to event hubs to send data to external systems such as third-party SIEMs and other log analytics
solutions.
SELECT
records.ArrayValue.[Properties you want to track]
INTO
[OutputSourceName – the Power BI source]
FROM
[InputSourceName] AS e
CROSS APPLY GetArrayElements(e.records) AS records
Prerequisites
You need to create an event hub if you don't already have one. If you already have a diagnostic setting using this
Event Hubs namespace, then that event hub will be reused.
The shared access policy for the namespace defines the permissions that the streaming mechanism has.
Streaming to Event Hubs requires Manage, Send, and Listen permissions. You can create or modify shared access
policies in the Azure portal under the Configure tab for your Event Hubs namespace.
To update the diagnostic setting to include streaming, you must have the ListKey permission on that Event Hubs
authorization rule. The Event Hubs namespace does not have to be in the same subscription as the subscription
that's emitting logs, as long as the user who configures the setting has appropriate RBAC access to both
subscriptions and both subscriptions are in the same AAD tenant.
properties Properties of the event. These will vary for each Azure service
as described in .
Following is sample output data from Event Hubs for a resource log:
{
"records": [
{
"time": "2016-07-15T18:00:22.6235064Z",
"workflowId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-
EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA",
"resourceId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-
EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA/RUNS/08587330013509
921957/ACTIONS/SEND_EMAIL",
"category": "WorkflowRuntime",
"level": "Error",
"operationName": "Microsoft.Logic/workflows/workflowActionCompleted",
"properties": {
"$schema": "2016-04-01-preview",
"startTime": "2016-07-15T17:58:55.048482Z",
"endTime": "2016-07-15T18:00:22.4109204Z",
"status": "Failed",
"code": "BadGateway",
"resource": {
"subscriptionId": "df602c9c-7aa0-407d-a6fb-eb20c8bd1192",
"resourceGroupName": "JohnKemTest",
"workflowId": "243aac67fe904cf195d4a28297803785",
"workflowName": "JohnKemTestLA",
"runId": "08587330013509921957",
"location": "westus",
"actionName": "Send_email"
},
"correlation": {
"actionTrackingId": "29a9862f-969b-4c70-90c4-dfbdc814e413",
"clientTrackingId": "08587330013509921958"
}
}
},
{
"time": "2016-07-15T18:01:15.7532989Z",
"workflowId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-
EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA",
"resourceId": "/SUBSCRIPTIONS/DF602C9C-7AA0-407D-A6FB-
EB20C8BD1192/RESOURCEGROUPS/JOHNKEMTEST/PROVIDERS/MICROSOFT.LOGIC/WORKFLOWS/JOHNKEMTESTLA/RUNS/08587330012106
702630/ACTIONS/SEND_EMAIL",
"category": "WorkflowRuntime",
"level": "Information",
"operationName": "Microsoft.Logic/workflows/workflowActionStarted",
"properties": {
"$schema": "2016-04-01-preview",
"startTime": "2016-07-15T18:01:15.5828115Z",
"status": "Running",
"resource": {
"subscriptionId": "df602c9c-7aa0-407d-a6fb-eb20c8bd1192",
"resourceGroupName": "JohnKemTest",
"workflowId": "243aac67fe904cf195d4a28297803785",
"workflowName": "JohnKemTestLA",
"runId": "08587330012106702630",
"location": "westus",
"actionName": "Send_email"
},
"correlation": {
"actionTrackingId": "042fb72c-7bd4-439e-89eb-3cf4409d429e",
"clientTrackingId": "08587330012106702632"
}
}
}
]
}
Next steps
Read more about resource logs.
Create diagnostic setting to collect logs and metrics in Azure.
Stream Azure Active Directory logs with Azure Monitor.
Get started with Event Hubs.
Update to Azure Activity log collection and export
1/23/2020 • 2 minutes to read • Edit Online
The Azure Activity log is a platform log that provides insight into subscription-level events that have occurred in
Azure. The method to send Activity log entries to an event hub or storage account or to a Log Analytics workspace
has changed to use diagnostic settings. This article describes the difference between the methods and how to clear
legacy settings in preparation to change to diagnostic settings.
ActivityStatus ActivityStatusValue
ActivitySubstatus ActivitySubstatusValue
OperationName OperationNameValue
ResourceProvider ResourceProviderValue
See the following articles for details on using the legacy collection methods.
Collect and analyze Azure activity logs in Log Analytics workspace in Azure Monitor
Collect Azure Activity logs into Azure Monitor across Azure Active Directory tenants
Export Azure Activity log to storage or Azure Event Hubs
Next steps
Learn more about the Activity Log
Learn more about diagnostic settings
Collect and analyze Azure activity logs in Log
Analytics workspace in Azure Monitor
1/14/2020 • 4 minutes to read • Edit Online
WARNING
You can now collect the Activity log into a Log Analytics workspace using a diagnostic setting similar to how you collect
resource logs. See Collect and analyze Azure activity logs in Log Analytics workspace in Azure Monitor.
The Azure Activity Log provides insight into subscription-level events that have occurred in your Azure
subscription. This article describes how to collect the Activity Log into a Log Analytics workspace and how to use
the Activity Log Analytics monitoring solution, which provides log queries and views for analyzing this data.
Connecting the Activity Log to a Log Analytics workspace provides the following benefits:
Consolidate the Activity Log from multiple Azure subscriptions into one location for analysis.
Store Activity Log entries for longer than 90 days.
Correlate Activity Log data with other monitoring data collected by Azure Monitor.
Use log queries to perform complex analysis and gain deep insights on Activity Log entries.
IMPORTANT
You may receive an error with the following procedure if the Microsoft.OperationalInsights and
Microsoft.OperationsManagement resource providers aren't registered for your subscription. See Azure resource providers
and types to register these providers.
Use the following procedure to connect the Activity Log to your Log Analytics workspace:
1. From the Log Analytics workspaces menu in the Azure portal, select the workspace to collect the
Activity Log.
2. In the Workspace Data Sources section of the workspace's menu, select Azure Activity log.
3. Click the subscription you want to connect.
4. Click Connect to connect the Activity log in the subscription to the selected workspace. If the subscription
is already connected to another workspace, click Disconnect first to disconnect it.
Azure Activity Log Entries Shows a bar chart of the top Azure Activity Log entry record
totals for the date range that you have selected and shows a
list of the top 10 activity callers. Click the bar chart to run a
log search for AzureActivity . Click a caller item to run a
log search returning all Activity Log entries for that item.
Activity Logs by Status Shows a doughnut chart for Azure Activity Log status for the
selected date range and a list of the top ten status records.
Click the chart to run a log query for
AzureActivity | summarize AggregatedValue = count()
by ActivityStatus
. Click a status item to run a log search returning all Activity
Log entries for that status record.
Activity Logs by Resource Shows the total number of resources with Activity Logs and
lists the top ten resources with record counts for each
resource. Click the total area to run a log search for
AzureActivity | summarize AggregatedValue = count()
by Resource
, which shows all Azure resources available to the solution.
Click a resource to run a log query returning all activity
records for that resource.
VISUALIZATION PART DESCRIPTION
Activity Logs by Resource Provider Shows the total number of resource providers that produce
Activity Logs and lists the top ten. Click the total area to run
a log query for
AzureActivity | summarize AggregatedValue = count()
by ResourceProvider
, which shows all Azure resource providers. Click a resource
provider to run a log query returning all activity records for
the provider.
Next steps
Learn more about the Activity Log.
Learn more about the Azure Monitor data platform.
Use log queries to view detailed information from your Activity Log.
Collect Azure Activity logs into Azure Monitor across
Azure Active Directory tenants (legacy)
12/27/2019 • 10 minutes to read • Edit Online
NOTE
This article describes the legacy method for configuring the Azure Activity log across Azure tenants to be collected in a Log
Analytics workspace. You can now collect the Activity log into a Log Analytics workspace using a diagnostic setting similar to
how you collect resource logs. See Collect and analyze Azure activity logs in Log Analytics workspace in Azure Monitor.
This article steps through a method to collect Azure Activity Logs into a Log Analytics workspace in Azure Monitor
using the Azure Log Analytics Data Collector connector for Logic Apps. Use the process in this article when you
need to send logs to a workspace in a different Azure Active Directory tenant. For example, if you are a managed
service provider, you may want to collect activity logs from a customer's subscription and store them in a Log
Analytics workspace in your own subscription.
If the Log Analytics workspace is in the same Azure subscription, or in a different subscription but in the same
Azure Active Directory, use the steps in the Collect and analyze Azure activity logs in Log Analytics workspace in
Azure Monitor to collect Azure Activity logs.
Overview
The strategy used in this scenario is to have Azure Activity Log send events to an Event Hub where a Logic App
sends them to your Log Analytics workspace.
Requirements
Following are the requirements for the Azure resources used in this scenario.
The Event Hub namespace does not have to be in the same subscription as the subscription emitting logs. The
user who configures the setting must have appropriate access permissions to both subscriptions. If you have
multiple subscriptions in the same Azure Active directory, you can send the activity logs for all subscriptions to
a single event hub.
The Logic App can be in a different subscription from the event hub and does not need to be in the same Azure
Active Directory. The Logic App reads from the Event Hub using the Event Hub's shared access key.
The Log Analytics workspace can be in a different subscription and Azure Active Directory from the Logic App,
but for simplicity we recommend that they are in the same subscription. The Logic App sends to the workspace
using the Log Analytics workspace ID and key.
2. Under Create namespace, either enter a new namespace or selecting an existing one. The system
immediately checks to see if the name is available.
3. Choose the pricing tier (Basic or Standard), an Azure subscription, resource group, and location for the new
resource. Click Create to create the namespace. You may have to wait a few minutes for the system to fully
provision the resources.
4. Click the namespace you just created from the list.
5. Select Shared access policies, and then click RootManageSharedAccessKey.
6. Click the copy button to copy the RootManageSharedAccessKey connection string to the clipboard.
7. In a temporary location, such as Notepad, keep a copy the Event Hub name and either the primary or
secondary Event Hub connection string. The Logic App requires these values. For the Event Hub connection
string, you can use the RootManageSharedAccessKey connection string or create a separate one. The
connection string you use must start with Endpoint=sb:// and be for a policy that has the Manage access
policy.
3. Select the Subscription to export from, and then click Select all in the Regions drop-down to select
events for resources in all regions. Click the Export to an event hub check box.
4. Click Service bus namespace, and then select the Subscription with the event hub, the event hub
namespace, and an event hub policy name.
5. Click OK and then Save to save these settings. The settings are immediately be applied to your
subscription.
Subscription Select the Azure subscription that will contain the logic
app.
Location Select the datacenter region for deploying your logic app.
Log Analytics Select if you want to log the status of each run of your
logic app in a Log Analytics workspace.
3. Select Create. When Deployment Succeeded notification displays, click on Go to resource to open your
Logic App.
4. Under Templates, choose Blank Logic App.
The Logic Apps Designer now shows you available connectors and their triggers, which you use for starting your
logic app workflow.
Add Event Hub trigger
1. In the search box for the Logic App Designer, type event hubs for your filter. Select the trigger Event Hubs
- When events are available in Event Hub.
2. When you're prompted for credentials, connect to your Event Hubs namespace. Enter a name for your
connection and then the connection string that you copied. Select Create.
3. After you create the connection, edit the settings for the trigger. Start by selecting insights-operational-
logs from the Event Hub name drop-down.
TIP
You can get a sample payload by clicking Run and looking at the Raw Output from the Event Hub. You can then use this
output with Use sample payload to generate schema in the Parse JSON activity to generate the schema.
4. After you create the connection, edit the settings in the table below.
SETTING VALUE DESCRIPTION
JSON Request body Output from the Compose action Retrieves the records from the body
of the Compose action.
5. Click Save to save the changes you've made to your Logic App.
To see detailed information on each step, click on the step name to expand it. Click on Show raw inputs and
Show raw outputs to see more information on the data received and sent at each step.
NOTE
The first time a new custom log is sent to the Log Analytics workspace it may take up to an hour for the custom log to be
searchable.
NOTE
The activity logs are written to a custom table and do not appear in the Activity Log solution.
Next steps
In this article, you’ve created a logic app to read Azure Activity Logs from an Event Hub and send them to the Log
Analytics workspace for analysis. To learn more about visualizing data in a workspace, including creating
dashboards, review the tutorial for Visualize data.
Visualize Log Search data tutorial
Export Azure Activity log to storage or Azure Event
Hubs
1/23/2020 • 8 minutes to read • Edit Online
IMPORTANT
The method for sending the Azure Activity log to Azure Storage and Azure Event Hubs has changed to diagnostic settings.
This article describes the legacy method which is in the process of being deprecated. See Update to Azure Activity log
collection and export for a comparison.
The Azure Activity Log provides insight into subscription-level events that have occurred in your Azure
subscription. In addition to viewing the Activity log in the Azure portal or copying it to a Log Analytics workspace
where it can be analyzed with other data collected by Azure Monitor, you can create a log profile to archive the
Activity log to an Azure storage account or stream it to an Event Hub.
Prerequisites
Storage account
If you're archiving your Activity Log, you need to create a storage account if you don't already have one. You
should not use an existing storage account that has other, non-monitoring data stored in it so that you can better
control access to monitoring data. If you are also archiving logs and metrics to a storage account though, you may
choose to use that same storage account to keep all monitoring data in a central location.
The storage account does not have to be in the same subscription as the subscription emitting logs as long as the
user who configures the setting has appropriate RBAC access to both subscriptions.
NOTE
You cannot currently archive data to a storage account that is behind a secured virtual network.
Event Hubs
If you're sending your Activity Log to an event hub, then you need to create an event hub if you don't already have
one. If you previously streamed Activity Log events to this Event Hubs namespace, then that event hub will be
reused.
The shared access policy defines the permissions that the streaming mechanism has. Streaming to Event Hubs
requires Manage, Send, and Listen permissions. You can create or modify shared access policies for the Event
Hubs namespace in the Azure portal under the Configure tab for your Event Hubs namespace.
To update the Activity Log log profile to include streaming, you must have the ListKey permission on that Event
Hubs authorization rule. The Event Hubs namespace does not have to be in the same subscription as the
subscription that's emitting logs, as long as the user who configures the setting has appropriate RBAC access to
both subscriptions and both subscriptions are in the same AAD tenant.
Stream the Activity Log to an Event Hub by creating a Log Profile.
IMPORTANT
You may receive an error when creating a log profile if the Microsoft.Insights resource provider isn't registered. See Azure
resource providers and types to register this provider.
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
If a log profile already exists, you first need to remove the existing log profile and then create a new one.
1. Use Get-AzLogProfile to identify if a log profile exists. If a log profile does exist, note the name property.
2. Use Remove-AzLogProfile to remove the log profile using the value from the name property.
Example script
Following is a sample PowerShell script to create a log profile that writes the Activity Log to both a storage
account and event hub.
az monitor log-profiles create --name "default" --location null --locations "global" "eastus" "westus"
--categories "Delete" "Write" "Action" --enabled false --days 0 --service-bus-rule-id
"/subscriptions/<YOUR SUBSCRIPTION ID>/resourceGroups/<RESOURCE GROUP
NAME>/providers/Microsoft.EventHub/namespaces/<EVENT HUB NAME
SPACE>/authorizationrules/RootManageSharedAccessKey"
Next steps
Learn more about the Activity Log
Collect Activity Log into Azure Monitor Logs
Create diagnostic setting to collect platform logs
and metrics in Azure
1/8/2020 • 5 minutes to read • Edit Online
Platform logs in Azure, including Azure Activity log and resource logs, provide detailed diagnostic and
auditing information for Azure resources and the Azure platform they depend on. This article provides
details on creating and configuring diagnostic settings to send platform logs to different destinations.
IMPORTANT
Before you create a diagnostic setting to collect the Activity log, you should first disable any legacy configuration. See
Collect Azure Activity log with legacy settings for details.
Each Azure resource requires its own diagnostic setting, which defines the following:
Categories of logs and metric data sent to the destinations defined in the setting. The available categories
will vary for different resource types.
One or more destinations to send the logs. Current destinations include Log Analytics workspace, Event
Hubs, and Azure Storage.
A single diagnostic setting can define no more than one of each of the destinations. If you want to send data
to more than one of a particular destination type (for example, two different Log Analytics workspaces), then
create multiple settings. Each resource can have up to 5 diagnostic settings.
NOTE
Platform metrics are collected automatically to Azure Monitor Metrics. Diagnostic settings can be used to collect
metrics for certain Azure services into Azure Monitor Logs for analysis with other monitoring data using log queries.
Destinations
Platform logs can be sent to the destinations in the following table. The configuration for each destination is
performed using the same process for creating diagnostic settings described in this article. Follow each link
in the following table for details on sending data to that destination.
DESTINATION DESCRIPTION
Log Analytics workspace Collecting logs into a Log Analytics workspace allows you
to analyze them with other monitoring data collected by
Azure Monitor using powerful log queries and also to
leverage other Azure Monitor features such as alerts and
visualizations.
Event hubs Sending logs to Event Hubs allows you to stream data to
external systems such as third-party SIEMs and other log
analytics solutions.
Azure storage account Archiving logs to an Azure storage account is useful for
audit, static analysis, or backup.
Create diagnostic settings in Azure portal
You can configure diagnostic settings in the Azure portal either from the Azure Monitor menu or from the
menu for the resource.
1. Where you configure diagnostic settings in the Azure portal depends on the resource.
For a single resource, click Diagnostic settings under Monitor in the resource's menu.
For one or more resources, click Diagnostic settings under Settings in the Azure Monitor
menu and then click on the resource.
For the Activity log, click Activity log in the Azure Monitor menu and then Diagnostic
settings. Make sure you disable any legacy configuration for the Activity log. See Disable
existing settings for details.
2. If no settings exist on the resource you have selected, you are prompted to create a setting. Click Add
diagnostic setting.
If there are existing settings on the resource, you will see a list of settings already configured. Either
click Add diagnostic setting to add a new setting or Edit setting to edit an existing one. Each
setting can have no more than one of each of the destination types.
3. Give your setting a name if it doesn't already have one.
4. Check the box for each destination to send the logs. Click Configure to specify their settings as
described in the following table.
SETTING DESCRIPTION
Event hub namespace The namespace where the event hub is created (if this
is your first time streaming logs) or streamed to (if
there are already resources that are streaming that log
category to this namespace).
Event hub name Optionally specify an event hub name to send all data
in the setting. If you don't specify a name, an event
hub is created for each log category. If you are sending
multiple categories, you may want to specify a name to
limit the number of event hubs created. See Azure
Event Hubs quotas and limits for details.
Event hub policy name Defines the permissions that the streaming mechanism
has.
5. Check the box for each of the categories of data to send to the specified destinations. The list of
categories will vary for each Azure service.
NOTE
Sending multi-dimensional metrics via diagnostic settings is not currently supported. Metrics with dimensions
are exported as flattened single dimensional metrics, aggregated across dimension values.
For example: The 'Incoming Messages' metric on an Event Hub can be explored and charted on a per queue
level. However, when exported via diagnostic settings the metric will be represented as all incoming messages
across all queues in the Event Hub.
6. Click Save.
After a few moments, the new setting appears in your list of settings for this resource, and logs are streamed
to the specified destinations as new event data is generated. Note that there may be up to fifteen minutes
between when an event is emitted and when it appears in a Log Analytics workspace.
IMPORTANT
You cannot use this method for the Azure Activity log. Instead, use Create diagnostic setting in Azure Monitor using a
Resource Manager template to create a Resource Manager template and deploy it with PowerShell.
Following is an example PowerShell cmdlet to create a diagnostic setting using all three destinations.
IMPORTANT
You cannot use this method for the Azure Activity log. Instead, use Create diagnostic setting in Azure Monitor using a
Resource Manager template to create a Resource Manager template and deploy it with CLI.
Following is an example CLI command to create a diagnostic setting using all three destinations.
Next steps
Read more about Azure platform Logs
Create diagnostic setting in Azure using a Resource
Manager template
1/14/2020 • 4 minutes to read • Edit Online
Diagnostic settings in Azure Monitor specify where to send Platform logs that are collected by Azure resources
and the Azure platform they depend on. This article provides details and examples for using an Azure Resource
Manager template to create and configure diagnostic settings to collect platform logs to different destinations.
NOTE
Since you can't create a diagnostic setting for the Azure Activity log using PowerShell or CLI like diagnostic settings for other
Azure resources, create a Resource Manager template for the Activity log using the information in this article and deploy the
template using PowerShell or CLI.
Deployment methods
You can deploy Resource Manager templates using any valid method including PowerShell and CLI. Diagnostic
settings for Activity log must deploy to a subscription using az deployment create for CLI or New-AzDeployment
for PowerShell. Diagnostic settings for resource logs must deploy to a resource group using
az group deployment create for CLI or New-AzResourceGroupDeployment for PowerShell.
See Deploy resources with Resource Manager templates and Azure PowerShell and Deploy resources with
Resource Manager templates and Azure CLI for details.
Resource logs
For resource logs, add a resource of type <resource namespace>/providers/diagnosticSettings to the template. The
properties section follows the format described in Diagnostic Settings - Create Or Update. Provide a category in
the logs section for each of the categories valid for the resource that you want to collect. Add the metrics
property to collect resource metrics to the same destinations if the resource supports metrics.
Following is a template that collects a resource log category for a particular resource to a Log Analytics
workspace, storage account, and event hub.
"resources": [
{
"type": "/<resource namespace>/providers/diagnosticSettings",
"name": "[concat(parameters('resourceName'),'/microsoft.insights/', parameters('settingName'))]",
"dependsOn": [
"[<resource Id for which resource logs will be enabled>]"
],
"apiVersion": "2017-05-01-preview",
"properties": {
"name": "[parameters('settingName')]",
"storageAccountId": "[resourceId('Microsoft.Storage/storageAccounts',
parameters('storageAccountName'))]",
"eventHubAuthorizationRuleId": "[parameters('eventHubAuthorizationRuleId')]",
"eventHubName": "[parameters('eventHubName')]",
"workspaceId": "[parameters('workspaceId')]",
"logs": [
{
"category": "<category name>",
"enabled": true
}
],
"metrics": [
{
"category": "AllMetrics",
"enabled": true
}
]
}
}
]
Example
Following is an example that creates a diagnostic setting for an autoscale setting that enables streaming of
resource logs to an event hub, a storage account, and a Log Analytics workspace.
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"autoscaleSettingName": {
"type": "string",
"metadata": {
"description": "The name of the autoscale setting"
}
},
"settingName": {
"type": "string",
"metadata": {
"description": "The name of the diagnostic setting"
}
},
"workspaceId": {
"type": "string",
"metadata": {
"description": "ResourceIDl of the Log Analytics workspace in which resource logs should be saved."
}
},
"storageAccountId": {
"type": "string",
"metadata": {
"description": "ResourceID of the Storage Account in which resource logs should be saved."
}
},
"eventHubAuthorizationRuleId": {
"type": "string",
"metadata": {
"description": "Resource ID of the event hub authorization rule for the Event Hubs namespace in which the
event hub should be created or streamed to."
}
},
"eventHubName": {
"type": "string",
"metadata": {
"description": "Optional. Name of the event hub within the namespace to which logs are streamed. Without
this, an event hub is created for each log category."
}
}
},
"variables": {},
"resources": [
{
"type": "microsoft.insights/autoscalesettings/providers/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[concat(parameters('autoscaleSettingName'),'/microsoft.insights/',
parameters('settingName'))]",
"dependsOn": [
"[resourceId('Microsoft.Insights/autoscalesettings', parameters('autoscaleSettingName'))]"
],
"properties": {
"workspaceId": "[parameters('workspaceId')]",
"storageAccountId": "[parameters('storageAccountId')]",
"eventHubAuthorizationRuleId": "[parameters('eventHubAuthorizationRuleId')]",
"eventHubName": "[parameters('eventHubName')]",
"logs": [
{
"category": "AutoscaleScaleActions",
"enabled": true
},
{
"category": "AutoscaleEvaluations",
"enabled": true
}
]
}
}
]
}
Activity log
For the Azure Activity log, add a resource of type Microsoft.Insights/diagnosticSettings . The available categories
are listed in Categories in the Activity Log. Following is a template that collects all Activity log categories to a Log
Analytics workspace, storage account, and event hub.
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"settingName": {
"type": "string",
"metadata": {
"description": "The name of the diagnostic setting"
}
},
"workspaceId": {
"type": "string",
"metadata": {
"description": "ResourceID of the Log Analytics workspace in which resource logs should be saved."
}
},
"storageAccountId": {
"storageAccountId": {
"type": "string",
"metadata": {
"description": "ResourceID of the Storage Account in which resource logs should be saved."
}
},
"eventHubAuthorizationRuleId": {
"type": "string",
"metadata": {
"description": "Resource ID of the event hub authorization rule for the Event Hubs namespace in which the
event hub should be created or streamed to."
}
},
"eventHubName": {
"type": "string",
"metadata": {
"description": "Optional. Name of the event hub within the namespace to which logs are streamed. Without
this, an event hub is created for each log category."
}
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.Insights/diagnosticSettings",
"apiVersion": "2017-05-01-preview",
"name": "[parameters('settingName')]",
"location": "global",
"properties": {
"workspaceId": "[parameters('workspaceId')]",
"storageAccountId": "[parameters('storageAccountId')]",
"eventHubAuthorizationRuleId": "[parameters('eventHubAuthorizationRuleId')]",
"eventHubName": "[parameters('eventHubName')]",
"logs": [
{
"category": "Administrative",
"enabled": true
},
{
"category": "Security",
"enabled": true
},
{
"category": "ServiceHealth",
"enabled": true
},
{
"category": "Alert",
"enabled": true
},
{
"category": "Recommendation",
"enabled": true
},
{
"category": "Policy",
"enabled": true
},
{
"category": "Autoscale",
"enabled": true
},
{
"category": "ResourceHealth",
"enabled": true
}
]
}
}
]
]
}
Next steps
Read more about platform logs in Azure.
Learn about diagnostic settings.
Prepare for format change to Azure Monitor platform
logs archived to a storage account
12/6/2019 • 4 minutes to read • Edit Online
WARNING
If you are sending Azure resource logs or metrics to a storage account using diagnostic settings or activity logs to a storage
account using log profiles, the format of the data in the storage account changed to JSON Lines on Nov. 1, 2018. The
instructions below describe the impact and how to update your tooling to handle the new format.
What changed
Azure Monitor offers a capability that enables you to send resource logs and activity logs into an Azure storage
account, Event Hubs namespace, or into a Log Analytics workspace in Azure Monitor. In order to address a system
performance issue, on November 1, 2018 at 12:00 midnight UTC the format of log data send to blob storage
changed. If you have tooling that is reading data out of blob storage, you need to update your tooling to
understand the new data format.
On Thursday, November 1, 2018 at 12:00 midnight UTC, the blob format changed to be JSON Lines. This
means each record will be delimited by a newline, with no outer records array and no commas between JSON
records.
The blob format changed for all diagnostic settings across all subscriptions at once. The first PT1H.json file
emitted for November 1 used this new format. The blob and container names remain the same.
Setting a diagnostic setting between prior to November 1 continued to emit data in the current format until
November 1.
This change occurred at once across all public cloud regions. The change will not occur in Microsoft Azure
Operated by 21Vianet, Azure Germany, or Azure Government clouds yet.
This change impacts the following data types:
Azure resource logs (see list of resources here)
Azure resource metrics being exported by diagnostic settings
Azure Activity log data being exported by log profiles
This change does not impact:
Network flow logs
Azure service logs not made available through Azure Monitor yet (for example, Azure App Service
resource logs, storage analytics logs)
Routing of Azure resource logs and activity logs to other destinations (Event Hubs, Log Analytics)
How to see if you are impacted
You are only impacted by this change if you:
1. Are sending log data to an Azure storage account using a diagnostic setting, and
2. Have tooling that depends on the JSON structure of these logs in storage.
To identify if you have diagnostic settings that are sending data to an Azure storage account, you can navigate to
the Monitor section of the portal, click on Diagnostic Settings, and identify any resources that have Diagnostic
Status set to Enabled:
If Diagnostic Status is set to enabled, you have an active diagnostic setting on that resource. Click on the resource
to see if any diagnostic settings are sending data to a storage account:
If you do have resources sending data to a storage account using these resource diagnostic settings, the format of
the data in that storage account will be impacted by this change. Unless you have custom tooling that operates off
of these storage accounts, the format change will not impact you.
Details of the format change
The current format of the PT1H.json file in Azure blob storage uses a JSON array of records. Here is a sample of a
KeyVault log file now:
{
"records": [
{
"time": "2016-01-05T01:32:01.2691226Z",
"resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT",
"operationName": "VaultGet",
"operationVersion": "2015-06-01",
"category": "AuditEvent",
"resultType": "Success",
"resultSignature": "OK",
"resultDescription": "",
"durationMs": "78",
"callerIpAddress": "104.40.82.76",
"correlationId": "",
"identity": {
"claim": {
"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-bd64-XXXXXXXXXXXX",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn": "live.com#username@outlook.com",
"appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"
}
},
"properties": {
"clientInfo": "azure-resource-manager/2.0",
"requestUri": "https://control-prod-wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01",
"id": "https://contosokeyvault.vault.azure.net/",
"httpStatusCode": 200
}
},
{
"time": "2016-01-05T01:33:56.5264523Z",
"resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT",
"operationName": "VaultGet",
"operationVersion": "2015-06-01",
"category": "AuditEvent",
"resultType": "Success",
"resultSignature": "OK",
"resultDescription": "",
"durationMs": "83",
"callerIpAddress": "104.40.82.76",
"correlationId": "",
"identity": {
"claim": {
"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-bd64-XXXXXXXXXXXX",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn": "live.com#username@outlook.com",
"appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"
}
},
"properties": {
"clientInfo": "azure-resource-manager/2.0",
"requestUri": "https://control-prod-wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01",
"id": "https://contosokeyvault.vault.azure.net/",
"httpStatusCode": 200
}
}
]
}
The new format uses JSON lines, where each event is a line and the newline character indicates a new event. Here
is what the above sample will look like in the PT1H.json file after the change:
{"time": "2016-01-05T01:32:01.2691226Z","resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT","operationName":
"VaultGet","operationVersion": "2015-06-01","category": "AuditEvent","resultType":
"Success","resultSignature": "OK","resultDescription": "","durationMs": "78","callerIpAddress":
"104.40.82.76","correlationId": "","identity": {"claim":
{"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-bd64-
XXXXXXXXXXXX","http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn":
"live.com#username@outlook.com","appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"}},"properties": {"clientInfo":
"azure-resource-manager/2.0","requestUri": "https://control-prod-
wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01","id": "https://contosokeyvault.vault.azure.net/","httpStatusCode": 200}}
{"time": "2016-01-05T01:33:56.5264523Z","resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT","operationName":
"VaultGet","operationVersion": "2015-06-01","category": "AuditEvent","resultType":
"Success","resultSignature": "OK","resultDescription": "","durationMs": "83","callerIpAddress":
"104.40.82.76","correlationId": "","identity": {"claim":
{"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-bd64-
XXXXXXXXXXXX","http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn":
"live.com#username@outlook.com","appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"}},"properties": {"clientInfo":
"azure-resource-manager/2.0","requestUri": "https://control-prod-
wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01","id": "https://contosokeyvault.vault.azure.net/","httpStatusCode": 200}}
This new format enables Azure Monitor to push log files using append blobs, which are more efficient for
continuously appending new event data.
How to update
You only need to make updates if you have custom tooling that ingests these log files for further processing. If you
are making use of an external log analytics or SIEM tool, we recommend using event hubs to ingest this data
instead. Event hubs integration is easier in terms of processing logs from many services and bookmarking location
in a particular log.
Custom tools should be updated to handle both the current format and the JSON Lines format described above.
This will ensure that when data starts to appear in the new format, your tools do not break.
Next steps
Learn about archiving resource resource logs to a storage account
Learn about archiving activity log data to a storage account
Stream Azure monitoring data to an event hub
1/8/2020 • 5 minutes to read • Edit Online
Azure Monitor provides a complete full stack monitoring solution for applications and services in Azure, in other
clouds, and on-premises. In addition to using Azure Monitor for analyzing that data and leveraging it for different
monitoring scenarios, you may need to send it to other monitoring tools in your environment. The most effective
method to stream monitoring data to external tools in most cases is using Azure Event Hubs. This article provides
a brief description for how you can stream monitoring data from different sources to an event hub and links to
detailed guidance.
Azure tenant Azure Active Directory audit logs Configure a tenant diagnostic setting
on your AAD tenant. See Tutorial:
Stream Azure Active Directory logs to
an Azure event hub for details.
TIER DATA METHOD
Azure subscription Azure Activity Log Create a log profile to export Activity
Log events to Event Hubs. See Export
Azure Activity log to storage or Azure
Event Hubs for details.
Azure resources Platform metrics Both types of data are sent to an event
Resource logs hub using a resource diagnostic setting.
See Stream Azure resource logs to an
event hub for details.
Operating system (guest) Azure virtual machines Install the Azure Diagnostics Extension
on Windows and Linux virtual machines
in Azure. See Streaming Azure
Diagnostics data in the hot path by
using Event Hubs for details on
Windows VMs and Use Linux
Diagnostic Extension to monitor metrics
and logs for details on Linux VMs.
Next Steps
Archive the Activity log to a storage account
Read the overview of the Azure Activity log
Set up an alert based on an Activity log event
Overview of the Azure Monitor agents
11/17/2019 • 4 minutes to read • Edit Online
Compute resources such as virtual machines generate data to monitor their performance and availability just like
other cloud resources. Compute resources though also have a guest operating system and workloads that need
to be monitored. Collecting this monitoring data from inside the resource requires an agent. This article describes
the agents used by Azure Monitor and helps you determine which you need to meet the requirements for your
particular environment.
Summary of agents
NOTE
Azure Monitor currently has multiple agents because of recent consolidation of Azure Monitor and Log Analytics. Both
agents share some capabilities while other features are unique to a particular agent. Depending on your requirements, you
may need one of the agents or both.
Azure Monitor has three agents that each provide specific functionality. Depending on your requirements, you
may install a single agent or multiple on your virtual machines and other compute resources.
Azure Diagnostics extension
Log Analytics agent
Dependency agent
The following table provides a quick comparison of the different agents. See the rest of this article for details on
each.
AZURE DIAGNOSTIC
EX TENSION LOG ANALYTICS AGENT DEPENDENCY AGENT
Data sent to Azure Storage Azure Monitor Logs Azure Monitor Logs
Azure Monitor Metrics
Event Hub
Azure Diagnostic extension
The Azure Diagnostics extension collects monitoring data from the guest operating system and workloads of
Azure compute resources. It primarily collects data into Azure Storage. You can configure Azure Monitor to copy
the data from storage to a Log Analytics workspace. You can also collect guest performance data into Azure
Monitor Metrics.
Azure Diagnostic Extension is often referred to as the Windows Azure Diagnostic (WAD ) or Linux Azure
Diagnostic (LAD ) extension.
Scenarios supported
Scenarios supported by the Azure Diagnostics extension include the following:
Collect logs and performance data from Azure compute resources.
Archive logs and performance data from the guest operating system to Azure storage.
View monitoring data in storage using a tool such as Azure Storage Explorer.
Collect performance data in a metrics database to take advantage of features supported by Azure Monitor
Metrics such as near real-time metric alerts and autoscale.
Collect monitoring data from storage to a Log Analytics workspace to take advantage of features supported
by Azure Monitor Logs such as log queries.
Send monitoring data to third-party tools using Azure Event Hubs.
Investigate VM boot issues with Boot Diagnostics.
Copy data from applications running in your VM to Application Insights to integrate with other application
monitoring.
Next steps
See Overview of the Log Analytics agent to review requirements and supported methods to deploy the agent
to machines hosted in Azure, in your datacenter, or other cloud environment.
Azure Monitor Frequently Asked Questions
1/24/2020 • 39 minutes to read • Edit Online
This Microsoft FAQ is a list of commonly asked questions about Azure Monitor.
General
What is Azure Monitor?
Azure Monitor is a service in Azure that provides performance and availability monitoring for applications and
services in Azure, other cloud environments, or on-premises. Azure Monitor collects data from multiple sources
into a common data platform where it can be analyzed for trends and anomalies. Rich features in Azure Monitor
assist you in quickly identifying and responding to critical situations that may affect your application.
What's the difference between Azure Monitor, Log Analytics, and Application Insights?
In September 2018, Microsoft combined Azure Monitor, Log Analytics, and Application Insights into a single
service to provide powerful end-to-end monitoring of your applications and the components they rely on. Features
in Log Analytics and Application Insights have not changed, although some features have been rebranded to Azure
Monitor in order to better reflect their new scope. The log data engine and query language of Log Analytics is now
referred to as Azure Monitor Logs. See Azure Monitor terminology updates.
What does Azure Monitor cost?
Features of Azure Monitor that are automatically enabled such as collection of metrics and activity logs are
provided at no cost. There is a cost associated with other features such as log queries and alerting. See the Azure
Monitor pricing page for detailed pricing information.
How do I enable Azure Monitor?
Azure Monitor is enabled the moment that you create a new Azure subscription, and Activity log and platform
metrics are automatically collected. Create diagnostic settings to collect more detailed information about the
operation of your Azure resources, and add monitoring solutions and insights to provide additional analysis on
collected data for particular services.
How do I access Azure Monitor?
Access all Azure Monitor features and data from the Monitor menu in the Azure portal. The Monitoring section
of the menu for different Azure services provides access to the same tools with data filtered to a particular
resource. Azure Monitor data is also accessible for a variety of scenarios using CLI, PowerShell, and a REST API.
Is there an on-premises version of Azure Monitor?
No. Azure Monitor is a scalable cloud service that processes and stores large amounts of data, although Azure
Monitor can monitor resources that are on-premises and in other clouds.
Can Azure Monitor monitor on-premises resources?
Yes, in addition to collecting monitoring data from Azure resources, Azure Monitor can collect data from virtual
machines and applications in other clouds and on-premises. See Sources of monitoring data for Azure Monitor.
Does Azure Monitor integrate with System Center Operations Manager?
You can connect your existing System Center Operations Manager management group to Azure Monitor to collect
data from agents into Azure Monitor Logs. This allows you to use log queries and solution to analyze data collected
from agents. You can also configure existing System Center Operations Manager agents to send data directly to
Azure Monitor. See Connect Operations Manager to Azure Monitor.
What IP addresses does Azure Monitor use?
See IP addresses used by Application Insights and Log Analytics for a listing of the IP addresses and ports required
for agents and other external resources to access Azure Monitor.
Monitoring data
Where does Azure Monitor get its data?
Azure Monitor collects data from a variety of sources including logs and metrics from Azure platform and
resources, custom applications, and agents running on virtual machines. Other services such as Azure Security
Center and Network Watcher collect data into a Log Analytics workspace so it can be analyzed with Azure Monitor
data. You can also send custom data to Azure Monitor using the REST API for logs or metrics. See Sources of
monitoring data for Azure Monitor.
What data is collected by Azure Monitor?
Azure Monitor collects data from a variety of sources into logs or metrics. Each type of data has its own relative
advantages, and each supports a particular set of features in Azure Monitor. There is a single metrics database for
each Azure subscription, while you can create multiple Log Analytics workspaces to collect logs depending on your
requirements. See Azure Monitor data platform.
Is there a maximum amount of data that I can collect in Azure Monitor?
There is no limit to the amount of metric data you can collect, but this data is stored for a maximum of 93 days. See
Retention of Metrics. There is no limit on the amount of log data that you can collected, but it may be affected by
the pricing tier you choose for the Log Analytics workspace. See pricing details.
How do I access data collected by Azure Monitor?
Insights and solutions provide a custom experience for working with data stored in Azure Monitor. You can work
directly with log data using a log query written in Kusto Query Language (KQL ). In the Azure portal, you can write
and run queries and interactively analyze data using Log Analytics. Analyze metrics in the Azure portal with the
Metrics Explorer. See Analyze log data in Azure Monitor and Getting started with Azure Metrics Explorer.
Logs
What's the difference between Azure Monitor Logs and Azure Data Explorer?
Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. Azure Monitor
Logs is built on top of Azure Data Explorer and uses the same Kusto Query Language (KQL ) with some minor
differences. See Azure Monitor log query language differences.
How do I retrieve log data?
All data is retrieved from a Log Analytics workspace using a log query written using Kusto Query Language (KQL ).
You can write your own queries or use solutions and insights that include log queries for a particular application or
service. See Overview of log queries in Azure Monitor.
What is a Log Analytics workspace?
All log data collected by Azure Monitor is stored in a Log Analytics workspace. A workspace is essentially a
container where log data is collected from a variety of sources. You may have a single Log Analytics workspace for
all your monitoring data or may have requirements for multiple workspaces. See Designing your Azure Monitor
Logs deployment.
Can you move an existing Log Analytics workspace to another Azure subscription?
You can move a workspace between resource groups or subscriptions but not to a different region. See Move a Log
Analytics workspace to different subscription or resource group.
Why can't I see Query Explorer and Save buttons in Log Analytics?
Query Explorer, Save and New alert rule buttons are not available when the query scope is set to a specific
resource. To create alerts, save or load a query, Log Analytics must be scoped to a workspace. To open Log
Analytics in workspace context, select Logs from the Azure Monitor menu. The last used workspace is selected,
but you can select any other workspace. See Log query scope and time range in Azure Monitor Log Analytics
Why am I getting the error: "Register resource provider 'Microsoft.Insights' for this subscription to enable this
query" when opening Log Analytics from a VM?
Many resource providers are automatically registered, but you may need to manually register some resource
providers. The scope for registration is always the subscription. See Resource providers and types for more
information.
Why am I am getting no access error message when opening Log Analytics from a VM?
To view VM Logs, you need to be granted with read permission to the workspaces that stores the VM logs. In these
cases, your administrator must grant you with to permissions in Azure.
Alerts
What is an alert in Azure Monitor?
Alerts proactively notify you when important conditions are found in your monitoring data. They allow you to
identify and address issues before the users of your system notice them. There are multiple kinds of alerts:
Metric - Metric value exceeds a threshold.
Log query - Results of a log query match defined criteria.
Activity log - Activity log event matches defined criteria.
Web test - Results of availability test match defined criteria.
See Overview of alerts in Microsoft Azure.
What is an action group?
An action group is a collection of notifications and actions that can be triggered by an alert. Multiple alerts can use
a single action group allowing you to leverage common sets of notifications and actions. See Create and manage
action groups in the Azure portal.
What is an action rule?
An action rule allows you to modify the behavior of a set of alerts that match a certain criteria. This allows you to to
perform such requirements as disable alert actions during a maintenance window. You can also apply an action
group to a set of alerts rather than applying them directly to the alert rules. See Action rules.
Agents
Does Azure Monitor require an agent?
An agent is only required to collect data from the operating system and workloads in virtual machines. The virtual
machines can be located in Azure, another cloud environment, or on-premises. See Overview of the Azure Monitor
agents.
What's the difference between the Azure Monitor agents?
Azure Diagnostic extension is for Azure virtual machines and collects data to Azure Monitor Metrics, Azure
Storage, and Azure Event Hubs. The Log Analytics agent is for virtual machines in Azure, another cloud
environment, or on-premises and collects data to Azure Monitor Logs. The Dependency agent requires the Log
Analytics agent and collected process details and dependencies. See Overview of the Azure Monitor agents.
Does my agent traffic use my ExpressRoute connection?
Traffic to Azure Monitor uses the Microsoft peering ExpressRoute circuit. See ExpressRoute documentation for a
description of the different types of ExpressRoute traffic.
How can I confirm that the Log Analytics agent is able to communicate with Azure Monitor?
From Control Panel on the agent computer, select Security & Settings, Microsoft Monitoring Agent . Under
the Azure Log Analytics (OMS ) tab, a green check mark icon confirms that the agent is able to communicate with
Azure Monitor. A yellow warning icon means the agent is having issues. One common reason is the Microsoft
Monitoring Agent service has stopped. Use service control manager to restart the service.
How do I stop the Log Analytics agent from communicating with Azure Monitor?
For agents connected to Log Analytics directly, open the Control Panel and select Security & Settings, Microsoft
Monitoring Agent. Under the Azure Log Analytics (OMS ) tab, remove all workspaces listed. In System Center
Operations Manager, remove the computer from the Log Analytics managed computers list. Operations Manager
updates the configuration of the agent to no longer report to Log Analytics.
How much data is sent per agent?
The amount of data sent per agent depends on:
The solutions you have enabled
The number of logs and performance counters being collected
The volume of data in the logs
See Manage usage and costs with Azure Monitor Logs for details.
For computers that are able to run the WireData agent, use the following query to see how much data is being sent:
WireData
| where ProcessName == "C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe"
| where Direction == "Outbound"
| summarize sum(TotalBytes) by Computer
How much network bandwidth is used by the Microsoft Management Agent (MMA ) when sending data to Azure
Monitor?
Bandwidth is a function on the amount of data sent. Data is compressed as it is sent over the network.
How can I be notified when data collection from the Log Analytics agent stops?
Use the steps described in create a new log alert to be notified when data collection stops. Use the following
settings for the alert rule:
Define alert condition: Specify your Log Analytics workspace as the resource target.
Alert criteria
Signal Name: Custom log search
Search query:
Heartbeat | summarize LastCall = max(TimeGenerated) by Computer | where LastCall < ago(15m)
Alert logic: Based on number of results, Condition Greater than, Threshold value 0
Evaluated based on: Period (in minutes) 30, Frequency (in minutes) 10
Define alert details
Name: Data collection stopped
Severity: Warning
Specify an existing or new Action Group so that when the log alert matches criteria, you are notified if you have a
heartbeat missing for more than 15 minutes.
What are the firewall requirements for Azure Monitor agents?
See Network firewall requirementsfor details on firewall requirements.
Visualizations
Why can't I can’t see View Designer?
View Designer is only available for users assigned with Contributor permissions or higher in the Log Analytics
workspace.
Application Insights
Configuration problems
I'm having trouble setting up my:
.NET app
Monitoring an already-running app
Azure diagnostics
Java web app
I get no data from my server
Set firewall exceptions
Set up an ASP.NET server
Set up a Java server
Can I use Application Insights with ...?
Web apps on an IIS server in Azure VM or Azure virtual machine scale set
Web apps on an IIS server - on-premises or in a VM
Java web apps
Node.js apps
Web apps on Azure
Cloud Services on Azure
App servers running in Docker
Single-page web apps
SharePoint
Windows desktop app
Other platforms
Is it free?
Yes, for experimental use. In the basic pricing plan, your application can send a certain allowance of data each
month free of charge. The free allowance is large enough to cover development, and publishing an app for a small
number of users. You can set a cap to prevent more than a specified amount of data from being processed.
Larger volumes of telemetry are charged by the Gb. We provide some tips on how to limit your charges.
The Enterprise plan incurs a charge for each day that each web server node sends telemetry. It is suitable if you
want to use Continuous Export on a large scale.
Read the pricing plan.
How much does it cost?
Open the Usage and estimated costs page in an Application Insights resource. There's a chart of recent
usage. You can set a data volume cap, if you want.
Open the Azure Billing blade to see your bills across all resources.
What does Application Insights modify in my project?
The details depend on the type of project. For a web application:
Adds these files to your project:
ApplicationInsights.config
ai.js
Installs these NuGet packages:
Application Insights API - the core API
Application Insights API for Web Applications - used to send telemetry from the server
Application Insights API for JavaScript Applications - used to send telemetry from the client
The packages include these assemblies:
Microsoft.ApplicationInsights
Microsoft.ApplicationInsights.Platform
Inserts items into:
Web.config
packages.config
(New projects only - if you add Application Insights to an existing project, you have to do this manually.) Inserts
snippets into the client and server code to initialize them with the Application Insights resource ID. For example,
in an MVC app, code is inserted into the master page Views/Shared/_Layout.cshtml
How do I upgrade from older SDK versions?
See the release notes for the SDK appropriate to your type of application.
How can I change which Azure resource my project sends data to?
In Solution Explorer, right-click ApplicationInsights.config and choose Update Application Insights. You can
send the data to an existing or new resource in Azure. The update wizard changes the instrumentation key in
ApplicationInsights.config, which determines where the server SDK sends your data. Unless you deselect "Update
all," it will also change the key where it appears in your web pages.
What is Status Monitor?
A desktop app that you can use in your IIS web server to help configure Application Insights in web apps. It doesn't
collect telemetry: you can stop it when you are not configuring an app.
Learn more.
What telemetry is collected by Application Insights?
From server web apps:
HTTP requests
Dependencies. Calls to: SQL Databases; HTTP calls to external services; Azure Cosmos DB, table, blob storage,
and queue.
Exceptions and stack traces.
Performance Counters - If you use Status Monitor, Azure monitoring for App Services, Azure monitoring for
VM or virtual machine scale set, or the Application Insights collectd writer.
Custom events and metrics that you code.
Trace logs if you configure the appropriate collector.
From client web pages:
Page view counts
AJAX calls Requests made from a running script.
Page view load data
User and session counts
Authenticated user IDs
From other sources, if you configure them:
Azure diagnostics
Import to Analytics
Log Analytics
Logstash
Can I filter out or modify some telemetry?
Yes, in the server you can write:
Telemetry Processor to filter or add properties to selected telemetry items before they are sent from your app.
Telemetry Initializer to add properties to all items of telemetry.
Learn more for ASP.NET or Java.
How are city, country/region, and other geo location data calculated?
We look up the IP address (IPv4 or IPv6) of the web client using GeoLite2.
Browser telemetry: We collect the sender's IP address.
Server telemetry: The Application Insights module collects the client IP address. It is not collected if
X-Forwarded-For is set.
To learn more about how IP address and geolocation data is collected in Application Insights refer to this article.
You can configure the ClientIpHeaderTelemetryInitializer to take the IP address from a different header. In some
systems, for example, it is moved by a proxy, load balancer, or CDN to X-Originating-IP . Learn more.
You can use Power BI to display your request telemetry on a map.
How long is data retained in the portal? Is it secure?
Take a look at Data Retention and Privacy.
Could personal data be sent in the telemetry?
This is possible if your code sends such data. It can also happen if variables in stack traces include personal data.
Your development team should conduct risk assessments to ensure that personal data is properly handled. Learn
more about data retention and privacy.
All octets of the client web address are always set to 0 after the geo location attributes are looked up.
My Instrumentation Key is visible in my web page source.
This is common practice in monitoring solutions.
It can't be used to steal your data.
It could be used to skew your data or trigger alerts.
We have not heard that any customer has had such problems.
You could:
Use two separate Instrumentation Keys (separate Application Insights resources), for client and server data. Or
Write a proxy that runs in your server, and have the web client send data through that proxy.
How do I see POST data in Diagnostic search?
We don't log POST data automatically, but you can use a TrackTrace call: put the data in the message parameter.
This has a longer size limit than the limits on string properties, though you can't filter on it.
Should I use single or multiple Application Insights resources?
Use a single resource for all the components or roles in a single business system. Use separate resources for
development, test, and release versions, and for independent applications.
See the discussion here
Example - cloud service with worker and web roles
How do I dynamically change the instrumentation key?
Discussion here
Example - cloud service with worker and web roles
What are the User and Session counts?
The JavaScript SDK sets a user cookie on the web client, to identify returning users, and a session cookie to
group activities.
If there is no client-side script, you can set cookies at the server.
If one real user uses your site in different browsers, or using in-private/incognito browsing, or different
machines, then they will be counted more than once.
To identify a logged-in user across machines and browsers, add a call to setAuthenticatedUserContext().
Have I enabled everything in Application Insights?
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Server app perf: response times, ... Add Application Insights to your project Detect perf issues
or Install AI Status Monitor on server
(or write your own code to track
dependencies)
Dependency telemetry Install AI Status Monitor on server Diagnose issues with databases or other
external components
Get stack traces from exceptions Insert TrackException calls in your code Detect and diagnose exceptions
(but some are reported automatically)
Search log traces Add a logging adapter Diagnose exceptions, perf issues
Client usage basics: page views, JavaScript initializer in web pages Usage analytics
sessions, ...
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Client custom metrics Tracking calls in web pages Enhance user experience
Automation
Configuring Application Insights
You can write PowerShell scripts using Azure Resource Monitor to:
Create and update Application Insights resources.
Set the pricing plan.
Get the instrumentation key.
Add a metric alert.
Add an availability test.
You can't set up a Metric Explorer report or set up continuous export.
Querying the telemetry
Use the REST API to run Analytics queries.
How can I set an alert on an event?
Azure alerts are only on metrics. Create a custom metric that crosses a value threshold whenever your event
occurs. Then set an alert on the metric. You'll get a notification whenever the metric crosses the threshold in either
direction; you won't get a notification until the first crossing, no matter whether the initial value is high or low; there
is always a latency of a few minutes.
Are there data transfer charges between an Azure web app and Application Insights?
If your Azure web app is hosted in a data center where there is an Application Insights collection endpoint, there
is no charge.
If there is no collection endpoint in your host data center, then your app's telemetry will incur Azure outgoing
charges.
This doesn't depend on where your Application Insights resource is hosted. It just depends on the distribution of
our endpoints.
Can I send telemetry to the Application Insights portal?
We recommend you use our SDKs and use the SDK API. There are variants of the SDK for various platforms.
These SDKs handle buffering, compression, throttling, retries, and so on. However, the ingestion schema and
endpoint protocol are public.
Can I monitor an intranet web server?
Yes, but you will need to allow traffic to our services by either firewall exceptions or proxy redirects.
QuickPulse https://rt.services.visualstudio.com:443
ApplicationIdProvider https://dc.services.visualstudio.com:443
TelemetryChannel https://dc.services.visualstudio.com:443
<ApplicationInsights>
...
<TelemetryModules>
<Add
Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule,
Microsoft.AI.PerfCounterCollector">
<QuickPulseServiceEndpoint>https://rt.services.visualstudio.com/QuickPulseService.svc</QuickPulseServiceEndpoin
t>
</Add>
</TelemetryModules>
...
<TelemetryChannel>
<EndpointAddress>https://dc.services.visualstudio.com/v2/track</EndpointAddress>
</TelemetryChannel>
...
<ApplicationIdProvider
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplicationId
Provider, Microsoft.ApplicationInsights">
<ProfileQueryEndpoint>https://dc.services.visualstudio.com/api/profiles/{0}/appId</ProfileQueryEndpoint>
</ApplicationIdProvider>
...
</ApplicationInsights>
NOTE
ApplicationIdProvider is available starting in v2.6.0.
Proxy passthrough
Proxy passthrough can be achieved by configuring either a machine level or application level proxy. For more
information see dotnet's article on DefaultProxy.
Example Web.config:
<system.net>
<defaultProxy>
<proxy proxyaddress="http://xx.xx.xx.xx:yyyy" bypassonlocal="true"/>
</defaultProxy>
</system.net>
Option 2
Re-enable collection for these properties for every container log line.
If the first option is not convenient due to query changes involved, you can re-enable collecting these fields by
enabling the setting log_collection_settings.enrich_container_logs in the agent config map as described in the
data collection configuration settings.
NOTE
The second option is not recommended with large clusters that have more than 50 nodes because it generates API server
calls from every node in the cluster to perform this enrichment. This option also increases data size for every log line collected.
console.log(json.stringify({
"Hello": "This example has multiple lines:",
"Docker/Moby": "will not break this into multiple lines",
"and you will receive":"all of them in log analytics",
"as one": "log entry"
}));
This data will look like the following example in Azure Monitor for logs when you query for it:
LogEntry : ({“Hello": "This example has multiple lines:","Docker/Moby": "will not break this into multiple
lines", "and you will receive":"all of them in log analytics", "as one": "log entry"}
For a detailed look at the issue, review the following GitHub link.
How do I resolve Azure AD errors when I enable live logs?
You may see the following error: The reply url specified in the request does not match the reply urls
configured for the application: '<application ID>'. The solution to solve it can be found in the article How to
view container data in real time with Azure Monitor for containers.
Why can't I upgrade cluster after onboarding?
If after you enable Azure Monitor for containers for an AKS cluster, you delete the Log Analytics workspace the
cluster was sending its data to, when attempting to upgrade the cluster it will fail. To work around this, you will have
to disable monitoring and then re-enable it referencing a different valid workspace in your subscription. When you
try to perform the cluster upgrade again, it should process and complete successfully.
Which ports and domains do I need to open/whitelist for the agent?
See the Network firewall requirements for the proxy and firewall configuration information required for the
containerized agent with Azure, Azure US Government, and Azure China 21Vianet clouds.
NOTE
We configure performance counters for the workspace that affects all VMs that report to the workspace, whether or not you
have chosen to onboard them to Azure Monitor for VMs. For more details on how performance counters are configured for
the workspace, please refer to our documentation. For information about the counters configured for Azure Monitor for VMs,
please refer to our enable Azure Monitor for VMs article.
Next steps
If your question isn't answered here, you can refer to the following forums to additional questions and answers.
Log Analytics
Application Insights
For general feedback on Azure Monitor please visit the feedback forum.
What is Azure Diagnostics extension
12/23/2019 • 3 minutes to read • Edit Online
The Azure Diagnostics extension is an agent within Azure that enables the collection of diagnostic data on a
deployed application. You can use the diagnostics extension from a number of different sources. Currently
supported are Azure Cloud Service (classic) Web and Worker Roles, Virtual Machines, Virtual Machine scale sets,
and Service Fabric. Other Azure services have different diagnostics methods. See Overview of monitoring in
Azure.
Linux Agent
A Linux version of the extension is available for Virtual Machines running Linux. The statistics collected and
behavior vary from the Windows version.
Windows Event logs Information sent to the Windows event logging system
.NET EventSource logs Code writing events using the .NET EventSource class
Manifest based ETW logs Event Tracing for Windows events generated by any process.
(1)
Crash dumps (logs) Information about the state of the process if an application
crashes
(1) To get a list of ETW providers, run c:\Windows\System32\logman.exe query providers in a console window on
the machine you'd like to gather information from.
Data storage
The extension stores its data in an Azure Storage account that you specify.
You can also send it to Application Insights.
Another option is to stream it to Event Hub, which then allows you to send it to non-Azure monitoring services.
You also have the choice of sending your data to Azure Monitor metrics time-series database. At this time, this
sink is only applicable to Performance Counters. It enables you to send performance counters in as custom
metrics. This feature is in Preview. The Azure Monitor sink supports:
Retrieving all performance counters sent to Azure Monitor via the Azure Monitor metrics APIs.
Alerting on all performance counters sent to Azure Monitor via the metric alerts in Azure Monitor
Treating wildcard operator in performance counters as the "Instance" dimension on your metric. For example
if you collected the "LogicalDisk(*)/DiskWrites/sec" counter you would be able to filter and split on the
"Instance" dimension to plot or alert on the Disk Writes/sec for each Logical Disk on the VM (for example, C:)
To learn more on how to configure this sink, refer to the Azure diagnostics schema documentation.
Costs
Each of the options above may incur costs. Be sure to research them to avoid unexpected bills. Application
Insights, Event hub, and Azure Storage have separate costs associated with ingestion and the time stored. In
particular, Azure Storage will hold any data forever so you may want to purge older data after a certain time
period to keep your costs down.
Next steps
Choose which service you're trying to collect diagnostics on and use the following articles to get started. Use the
general Azure diagnostics links for reference for specific tasks.
Virtual Machines
If using Visual Studio, see Use Visual Studio to trace Azure Virtual Machines to get started. Otherwise, see
Set up Azure Diagnostics on an Azure Virtual Machine
For more advanced topics, see
Use PowerShell to set up diagnostics on Azure Virtual Machines
Create a Windows Virtual machine with monitoring and diagnostics using Azure Resource Manager Template
Service Fabric
Get started at Monitor a Service Fabric application. Many other Service Fabric diagnostics articles are available
in the navigation tree on the left once you get to this article.
General articles
Learn to use Performance Counters in Azure Diagnostics.
If you have trouble with diagnostics starting or finding your data in Azure storage tables, see
TroubleShooting Azure Diagnostics
Collect Azure resource logs from Azure Storage
12/13/2019 • 6 minutes to read • Edit Online
Azure Monitor can read the logs for the following services that write diagnostics to table storage or IIS logs
written to blob storage:
Service Fabric clusters (Preview )
Virtual Machines
Web/Worker Roles
Before Azure Monitor can collect data into a Log Analytics workspace for these resources, Azure diagnostics must
be enabled.
Once diagnostics are enabled, you can use the Azure portal or PowerShell configure the workspace to collect the
logs.
Azure Diagnostics is an Azure extension that enables you to collect diagnostic data from a worker role, web role, or
virtual machine running in Azure. The data is stored in an Azure storage account and can then be collected by
Azure Monitor.
For Azure Monitor to collect these Azure Diagnostics logs, the logs must be in the following locations:
NOTE
IIS logs from Azure Websites are not currently supported.
For virtual machines, you have the option of installing the Log Analytics agent into your virtual machine to enable
additional insights. In addition to being able to analyze IIS logs and Event Logs, you can perform additional
analysis including configuration change tracking, SQL assessment, and update assessment.
Enable Azure diagnostics in a virtual machine for event log and IIS log
collection
Use the following procedure to enable Azure diagnostics in a virtual machine for Event Log and IIS log collection
using the Microsoft Azure portal.
To enable Azure diagnostics in a virtual machine with the Azure portal
1. Install the VM Agent when you create a virtual machine. If the virtual machine already exists, verify that the
VM Agent is already installed.
In the Azure portal, navigate to the virtual machine, select Optional Configuration, then
Diagnostics and set Status to On.
Upon completion, the VM has the Azure Diagnostics extension installed and running. This extension
is responsible for collecting your diagnostics data.
2. Enable monitoring and configure event logging on an existing VM. You can enable diagnostics at the VM
level. To enable diagnostics and then configure event logging, perform the following steps:
a. Select the VM.
b. Click Monitoring.
c. Click Diagnostics.
d. Set the Status to ON.
e. Select each diagnostics log that you want to collect.
f. Click OK.
Enable Azure diagnostics in a Web role for IIS log and event collection
Refer to How To Enable Diagnostics in a Cloud Service for general steps on enabling Azure diagnostics. The
instructions below use this information and customize it for use with Log Analytics.
With Azure diagnostics enabled:
IIS logs are stored by default, with log data transferred at the scheduledTransferPeriod transfer interval.
Windows Event Logs are not transferred by default.
To enable diagnostics
To enable Windows Event Logs, or to change the scheduledTransferPeriod, configure Azure Diagnostics using the
XML configuration file (diagnostics.wadcfg), as shown in Step 4: Create your Diagnostics configuration file and
install the extension
The following example configuration file collects IIS Logs and all Events from the Application and System logs:
<?xml version="1.0" encoding="utf-8" ?>
<DiagnosticMonitorConfiguration
xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration"
configurationChangePollInterval="PT1M"
overallQuotaInMB="4096">
<Directories bufferQuotaInMB="0"
scheduledTransferPeriod="PT10M">
<!-- IISLogs are only relevant to Web roles -->
<IISLogs container="wad-iis" directoryQuotaInMB="0" />
</Directories>
<WindowsEventLog bufferQuotaInMB="0"
scheduledTransferLogLevelFilter="Verbose"
scheduledTransferPeriod="PT10M">
<DataSource name="Application!*" />
<DataSource name="System!*" />
</WindowsEventLog>
</DiagnosticMonitorConfiguration>
Ensure that your ConfigurationSettings specifies a storage account, as in the following example:
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
value="DefaultEndpointsProtocol=https;AccountName=<AccountName>;AccountKey=<AccountKey>"/>
</ConfigurationSettings>
The AccountName and AccountKey values are found in the Azure portal in the storage account dashboard,
under Manage Access Keys. The protocol for the connection string must be https.
Once the updated diagnostic configuration is applied to your cloud service and it is writing diagnostics to Azure
Storage, then you are ready to configure the Log Analytics workspace.
NOTE
The portal does not validate that the Source exists in the storage account or if new data is being written.
Enable Azure diagnostics in a virtual machine for event log and IIS log
collection using PowerShell
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Use the steps in Configuring Azure Monitor to index Azure diagnostics to use PowerShell to read from Azure
diagnostics that are written to table storage.
Using Azure PowerShell you can more precisely specify the events that are written to Azure Storage. For more
information, see Enabling Diagnostics in Azure Virtual Machines.
You can enable and update Azure diagnostics using the following PowerShell script. You can also use this script
with a custom logging configuration. Modify the script to set the storage account, service name, and virtual
machine name. The script uses cmdlets for classic virtual machines.
Review the following script sample, copy it, modify it as needed, save the sample as a PowerShell script file, and
then run the script.
#Connect to Azure
Add-AzureAccount
# settings to change:
$wad_storage_account_name = "myStorageAccount"
$service_name = "myService"
$vm_name = "myVM"
$wad_b64_config = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($wad_xml_config))
$wad_public_config = [string]::Format("{{""xmlCfg"":""{0}""}}",$wad_b64_config)
$wad_extension_name = "IaaSDiagnostics"
$wad_publisher = "Microsoft.Azure.Diagnostics"
$wad_version = (Get-AzureVMAvailableExtension -Publisher $wad_publisher -ExtensionName
$wad_extension_name).Version # Gets latest version of the extension
Next steps
Collect logs and metrics for Azure services for supported Azure services.
Enable Solutions to provide insight into the data.
Use search queries to analyze the data.
Azure Diagnostics extension configuration schema
versions and history
12/23/2019 • 7 minutes to read • Edit Online
This page indexes Azure Diagnostics extension schema versions shipped as part of the Microsoft Azure SDK.
NOTE
The Azure Diagnostics extension is the component used to collect performance counters and other statistics from:
Azure Virtual Machines
Virtual Machine Scale Sets
Service Fabric
Cloud Services
Network Security Groups
This page is only relevant if you are using one of these services.
The Azure Diagnostics extension is used with other Microsoft diagnostics products like Azure Monitor, which
includes Application Insights and Log Analytics. For more information, see Microsoft Monitoring Tools Overview.
Schemas index
Different versions of Azure diagnostics use different configuration schemas. Schema 1.0 and 1.2 have been
deprecated. For more information on version 1.3 and later, see Diagnostics 1.3 and later Configuration Schema
Version history
Diagnostics extension 1.11
Added support for the Azure Monitor sink. This sink is only applicable to performance counters. Enables sending
performance counters collected on your VM, VMSS, or cloud service to Azure Monitor as custom metrics. The
Azure Monitor sink supports:
Retrieving all performance counters sent to Azure Monitor via the Azure Monitor metrics APIs.
Alerting on all performance counters sent to Azure Monitor via the new unified alerts experience in Azure
Monitor
Treating wildcard operator in performance counters as the "Instance" dimension on your metric. For example if
you collected the "LogicalDisk(*)/DiskWrites/sec" counter you would be able to filter and split on the "Instance"
dimension to plot or alert on the Disk Writes/sec for each Logical Disk (C:, D:, etc.)
Define Azure Monitor as a new sink in your diagnostics extension configuration
"SinksConfig": {
"Sink": [
{
"name": "AzureMonitorSink",
"AzureMonitor": {}
},
]
}
<SinksConfig>
<Sink name="AzureMonitorSink">
<AzureMonitor/>
</Sink>
</SinksConfig>
NOTE
Configuring the Azure Monitor sink for Classic VMs and Classic CLoud Service requires more parameters to be defined in the
Diagnostics extension's private config.
For more details please reference the detailed diagnostics extension schema documentation.
Next, you can configure your performance counters to be routed to the Azure Monitor Sink.
"PerformanceCounters": {
"scheduledTransferPeriod": "PT1M",
"sinks": "AzureMonitorSink",
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Processor(_Total)\\% Processor Time",
"sampleRate": "PT1M",
"unit": "percent"
}
]
},
{
"storageAccountName": "diagstorageaccount",
"storageAccountEndPoint": "https://core.windows.net",
"storageAccountSasToken": "{sas token}",
"SecondaryStorageAccounts": {
"StorageAccount": [
{
"name": "secondarydiagstorageaccount",
"endpoint": "https://core.windows.net",
"sasToken": "{sas token}"
}
]
}
}
<PrivateConfig>
<StorageAccount name="diagstorageaccount" endpoint="https://core.windows.net" sasToken="{sas token}" />
<SecondaryStorageAccounts>
<StorageAccount name="secondarydiagstorageaccount" endpoint="https://core.windows.net" sasToken="{sas
token}" />
</SecondaryStorageAccounts>
</PrivateConfig>
{
"WadCfg": {
},
"StorageAccount": "diagstorageaccount",
"StorageType": "TableAndBlob"
}
<PublicConfig>
<WadCfg />
<StorageAccount>diagstorageaccount</StorageAccount>
<StorageType>TableAndBlob</StorageType>
</PublicConfig>
NOTE
The Azure Diagnostics extension is the component used to collect performance counters and other statistics from:
Azure Virtual Machines
Virtual Machine Scale Sets
Service Fabric
Cloud Services
Network Security Groups
This page is only relevant if you are using one of these services.
This page is valid for versions 1.3 and newer (Azure SDK 2.4 and newer). Newer configuration sections are commented to
show in what version they were added. Version 1.0 and 1.2 of the schema have been archived and no longer available.
The configuration file described here is used to set diagnostic configuration settings when the diagnostics monitor starts.
The extension is used in conjunction with other Microsoft diagnostics products like Azure Monitor, which includes Application
Insights and Log Analytics.
Download the public configuration file schema definition by executing the following PowerShell command:
For more information about using Azure Diagnostics, see Azure Diagnostics Extension.
<Directories scheduledTransferPeriod="PT5M">
<IISLogs containerName="iislogs" />
<FailedRequestLogs containerName="iisfailed" />
<DataSources>
<DirectoryConfiguration containerName="mynewprocess">
<Absolute path="C:\MyNewProcess" expandEnvironment="false" />
</DirectoryConfiguration>
<DirectoryConfiguration containerName="badapp">
<Absolute path="%SYSTEMDRIVE%\BadApp" expandEnvironment="true" />
</DirectoryConfiguration>
<DirectoryConfiguration containerName="goodapp">
<LocalResource name="Skippy" relativePath="..\PeanutButter"/>
</DirectoryConfiguration>
</DataSources>
</Directories>
<EtwProviders>
<EtwEventSourceProviderConfiguration
provider="MyProviderClass"
scheduledTransferPeriod="PT5M">
<Event id="0"/>
<Event id="1" eventDestination="errorTable"/>
<DefaultEvents />
</EtwEventSourceProviderConfiguration>
<EtwManifestProviderConfiguration provider="5974b00b-84c2-44bc-9e58-3a2451b4e3ad"
scheduledTransferLogLevelFilter="Information" scheduledTransferPeriod="PT2M">
<Event id="0"/>
<DefaultEvents eventDestination="defaultTable"/>
</EtwManifestProviderConfiguration>
</EtwProviders>
<WindowsEventLog scheduledTransferPeriod="PT5M">
<DataSource name="System!*[System[Provider[@Name='Microsoft Antimalware']]]"/>
<DataSource name="System!*[System[Provider[@Name='NTFS'] and (EventID=55)]]" />
<DataSource name="System!*[System[Provider[@Name='disk'] and (EventID=7 or EventID=52 or EventID=55)]]" />
</WindowsEventLog>
<Logs bufferQuotaInMB="1024"
scheduledTransferPeriod="PT1M"
scheduledTransferLogLevelFilter="Verbose"
sinks="ApplicationInsights.AppLogs"/> <!-- sinks attribute added in 1.5 -->
</DiagnosticMonitorConfiguration>
</WadCfg>
<StorageAccount>diagstorageaccount</StorageAccount>
<StorageType>TableAndBlob</StorageType> <!-- Added in 1.8 -->
</PublicConfig>
<AzureMonitorAccount>
<ServicePrincipalMeta> <!-- Added in 1.11; only needed for classic VMs and Classic cloud services -->
<PrincipalId>{Insert service principal clientId}</PrincipalId>
<Secret>{Insert service principal client secret}</Secret>
</ServicePrincipalMeta>
</AzureMonitorAccount>
<SecondaryStorageAccounts>
<StorageAccount name="secondarydiagstorageaccount" key="{base64 encoded key}" endpoint="https://core.windows.net"
sasToken="{sas token}" />
</SecondaryStorageAccounts>
<SecondaryEventHubs>
<EventHub Url="https://myeventhub-ns.servicebus.windows.net/secondarydiageventhub" SharedAccessKeyName="SendRule"
SharedAccessKey="{base64 encoded key}" />
</SecondaryEventHubs>
</PrivateConfig>
<IsEnabled>true</IsEnabled>
</DiagnosticsConfiguration>
NOTE
The public config Azure Monitor sink definition has two properties, resourceId and region. These are only required for Classic VMs and
Classic Cloud services. These properties should not be used for Resource Manager Virtual Machines or Virtual Machine Scale sets. There is
also an additional Private Config element for the Azure Monitor sink, that passes in a Principal Id and Secret. This is only required for
Classic VMs and Classic Cloud Services. For Resource Manager VMs and VMSS the Azure Monitor definition in the private config element
can be excluded.
"PublicConfig" {
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": 10000,
"DiagnosticInfrastructureLogs": {
"scheduledTransferLogLevelFilter": "Error"
},
"PerformanceCounters": {
"scheduledTransferPeriod": "PT1M",
"sinks": "AzureMonitorSink",
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Processor(_Total)\\% Processor Time",
"sampleRate": "PT1M",
"unit": "percent"
}
]
},
"Directories": {
"scheduledTransferPeriod": "PT5M",
"IISLogs": {
"containerName": "iislogs"
},
"FailedRequestLogs": {
"containerName": "iisfailed"
},
"DataSources": [
{
"containerName": "mynewprocess",
"Absolute": {
"path": "C:\\MyNewProcess",
"expandEnvironment": false
}
},
},
{
"containerName": "badapp",
"Absolute": {
"path": "%SYSTEMDRIVE%\\BadApp",
"expandEnvironment": true
}
},
{
"containerName": "goodapp",
"LocalResource": {
"relativePath": "..\\PeanutButter",
"name": "Skippy"
}
}
]
},
"EtwProviders": {
"sinks": "",
"EtwEventSourceProviderConfiguration": [
{
"scheduledTransferPeriod": "PT5M",
"provider": "MyProviderClass",
"Event": [
{
"id": 0
},
{
"id": 1,
"eventDestination": "errorTable"
}
],
"DefaultEvents": {
}
}
],
"EtwManifestProviderConfiguration": [
{
"scheduledTransferPeriod": "PT2M",
"scheduledTransferLogLevelFilter": "Information",
"provider": "5974b00b-84c2-44bc-9e58-3a2451b4e3ad",
"Event": [
{
"id": 0
}
],
"DefaultEvents": {
}
}
]
},
"WindowsEventLog": {
"scheduledTransferPeriod": "PT5M",
"DataSource": [
{
"name": "System!*[System[Provider[@Name='Microsoft Antimalware']]]"
},
{
"name": "System!*[System[Provider[@Name='NTFS'] and (EventID=55)]]"
},
{
"name": "System!*[System[Provider[@Name='disk'] and (EventID=7 or EventID=52 or EventID=55)]]"
}
]
},
"Logs": {
"scheduledTransferPeriod": "PT1M",
"scheduledTransferLogLevelFilter": "Verbose",
"sinks": "ApplicationInsights.AppLogs"
},
"CrashDumps": {
"directoryQuotaPercentage": 30,
"dumpType": "Mini",
"containerName": "wad-crashdumps",
"CrashDumpConfiguration": [
{
"processName": "mynewprocess.exe"
"processName": "mynewprocess.exe"
},
{
"processName": "badapp.exe"
}
]
}
},
"SinksConfig": {
"Sink": [
{
"name": "AzureMonitorSink",
"AzureMonitor":
{
"ResourceId": "{insert resourceId if a classic VM or cloud service, else property not needed}",
"Region": "{insert Azure region of resource if a classic VM or cloud service, else property not
needed}"
}
},
{
"name": "ApplicationInsights",
"ApplicationInsights": "{Insert InstrumentationKey}",
"Channels": {
"Channel": [
{
"logLevel": "Error",
"name": "Errors"
},
{
"logLevel": "Verbose",
"name": "AppLogs"
}
]
}
},
{
"name": "EventHub",
"EventHub": {
"Url": "https://myeventhub-ns.servicebus.windows.net/diageventhub",
"SharedAccessKeyName": "SendRule",
"usePublisherId": false
}
},
{
"name": "secondaryEventHub",
"EventHub": {
"Url": "https://myeventhub-ns.servicebus.windows.net/secondarydiageventhub",
"SharedAccessKeyName": "SendRule",
"usePublisherId": false
}
},
{
"name": "secondaryStorageAccount",
"StorageAccount": {
"name": "secondarydiagstorageaccount",
"endpoint": "https://core.windows.net"
}
}
]
}
},
"StorageAccount": "diagstorageaccount",
"StorageType": "TableAndBlob"
}
NOTE
The public config Azure Monitor sink definition has two properties, resourceId and region. These are only required for Classic VMs and
Classic Cloud services. These properties should not be used for Resource Manager Virtual Machines or Virtual Machine Scale sets.
"PrivateConfig" {
"storageAccountName": "diagstorageaccount",
"storageAccountKey": "{base64 encoded key}",
"storageAccountEndPoint": "https://core.windows.net",
"storageAccountSasToken": "{sas token}",
"EventHub": {
"Url": "https://myeventhub-ns.servicebus.windows.net/diageventhub",
"SharedAccessKeyName": "SendRule",
"SharedAccessKey": "{base64 encoded key}"
},
"AzureMonitorAccount": {
"ServicePrincipalMeta": {
"PrincipalId": "{Insert service principal client Id}",
"Secret": "{Insert service principal client secret}"
}
},
"SecondaryStorageAccounts": {
"StorageAccount": [
{
"name": "secondarydiagstorageaccount",
"key": "{base64 encoded key}",
"endpoint": "https://core.windows.net",
"sasToken": "{sas token}"
}
]
},
"SecondaryEventHubs": {
"EventHub": [
{
"Url": "https://myeventhub-ns.servicebus.windows.net/secondarydiageventhub",
"SharedAccessKeyName": "SendRule",
"SharedAccessKey": "{base64 encoded key}"
}
]
}
}
NOTE
There is an additional Private Config element for the Azure Monitor sink, that passes in a Principal Id and Secret. This is only required for
Classic VMs and Classic Cloud Services. For Resource Manager VMs and VMSS the Azure Monitor definition in the private config element
can be excluded.
DiagnosticsConfiguration Element
Tree: Root - DiagnosticsConfiguration
Added in version 1.3.
The top-level element of the diagnostics configuration file.
Attribute xmlns - The XML namespace for the diagnostics configuration file is:
http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration
CHILD ELEMENTS DESCRIPTION
PublicConfig Element
Tree: Root - DiagnosticsConfiguration - PublicConfig
Describes the public diagnostics configuration.
StorageAccount The name of the Azure Storage account to store the data in. May
also be specified as a parameter when executing the Set-
AzureServiceDiagnosticsExtension cmdlet.
LocalResourceDirectory The directory on the virtual machine where the Monitoring Agent
stores event data. If not, set, the default directory is used:
WadCFG Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG
Identifies and configures the telemetry data to be collected.
DiagnosticMonitorConfiguration Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration
Required
ATTRIBUTES DESCRIPTION
overallQuotaInMB The maximum amount of local disk space that may be consumed by
the various types of diagnostic data collected by Azure Diagnostics.
The default setting is 4096 MB.
ATTRIBUTES DESCRIPTION
useProxyServer Configure Azure Diagnostics to use the proxy server settings as set
in IE settings.
CrashDumps Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - CrashDumps
Enable the collection of crash dumps.
ATTRIBUTES DESCRIPTION
containerName Optional. The name of the blob container in your Azure Storage
account to be used to store crash dumps.
Directories Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - Directories
Enables the collection of the contents of a directory, IIS failed access request logs and/or IIS logs.
Optional scheduledTransferPeriod attribute. See explanation earlier.
DataSources Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - Directories -
DataSources
A list of directories to monitor.
DirectoryConfiguration Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - Directories -
DataSources - DirectoryConfiguration
May include either the Absolute or LocalResource element but not both.
EtwProviders Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - EtwProviders
Configures collection of ETW events from EventSource and/or ETW Manifest based providers.
EtwEventSourceProviderConfiguration Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - EtwProviders-
EtwEventSourceProviderConfiguration
Configures collection of events generated from EventSource Class.
Optional attribute:
Optional attribute:
Metrics Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - Metrics
Enables you to generate a performance counter table that is optimized for fast queries. Each performance counter that is
defined in the PerformanceCounters element is stored in the Metrics table in addition to the Performance Counter table.
The resourceId attribute is required. The resource ID of the Virtual Machine or Virtual Machine Scale Set you are deploying
Azure Diagnostics to. Get the resourceID from the Azure portal. Select Browse -> Resource Groups -> <Name>. Click the
Properties tile and copy the value from the ID field.
PerformanceCounters Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - PerformanceCounters
Enables the collection of performance counters.
Optional attribute:
Optional scheduledTransferPeriod attribute. See explanation earlier.
Optional attribute:
Logs Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - Logs
Present in version 1.0 and 1.1. Missing in 1.2. Added back in 1.3.
Defines the buffer configuration for basic Azure logs.
The default is 0.
DockerSources
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - DiagnosticMonitorConfiguration - DockerSources
Added in 1.9.
Sink Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - SinksConfig - Sink
Added in version 1.5.
Defines locations to send diagnostic data to. For example, the Application Insights service.
Channels Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - SinksConfig - Sink - Channels
Added in version 1.5.
Defines filters for streams of log data passing through a sink.
Channel Element
Tree: Root - DiagnosticsConfiguration - PublicConfig - WadCFG - SinksConfig - Sink - Channels - Channel
Added in version 1.5.
Defines locations to send diagnostic data to. For example, the Application Insights service.
StorageAccount The storage account to use. The following attributes are required
IsEnabled Element
Tree: Root - DiagnosticsConfiguration - IsEnabled
Boolean. Use true to enable the diagnostics or false to disable the diagnostics.
Store and view diagnostic data in Azure Storage
12/23/2019 • 2 minutes to read • Edit Online
Diagnostic data is not permanently stored unless you transfer it to the Microsoft Azure storage emulator or to
Azure storage. Once in storage, it can be viewed with one of several available tools.
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
value="UseDevelopmentStorage=true" />
</ConfigurationSettings>
You can change this connection string to provide account information for an Azure storage account.
Depending on the type of diagnostic data that is being collected, Azure Diagnostics uses either the Blob service or
the Table service. The following table shows the data sources that are persisted and their format.
Next Steps
Trace the flow in a Cloud Services application with Azure Diagnostics
Send Cloud Service, Virtual Machine, or Service
Fabric diagnostic data to Application Insights
12/23/2019 • 4 minutes to read • Edit Online
Cloud services, Virtual Machines, Virtual Machine Scale Sets and Service Fabric all use the Azure Diagnostics
extension to collect data. Azure diagnostics sends data to Azure Storage tables. However, you can also pipe all
or a subset of the data to other locations using Azure Diagnostics extension 1.5 or later.
This article describes how to send data from the Azure Diagnostics extension to Application Insights.
<SinksConfig>
<Sink name="ApplicationInsights">
<ApplicationInsights>{Insert InstrumentationKey}</ApplicationInsights>
<Channels>
<Channel logLevel="Error" name="MyTopDiagData" />
<Channel logLevel="Verbose" name="MyLogData" />
</Channels>
</Sink>
</SinksConfig>
"SinksConfig": {
"Sink": [
{
"name": "ApplicationInsights",
"ApplicationInsights": "{Insert InstrumentationKey}",
"Channels": {
"Channel": [
{
"logLevel": "Error",
"name": "MyTopDiagData"
},
{
"logLevel": "Error",
"name": "MyLogData"
}
]
}
}
]
}
The Sink name attribute is a string value that uniquely identifies the sink.
The ApplicationInsights element specifies instrumentation key of the Application insights resource
where the Azure diagnostics data is sent.
If you don't have an existing Application Insights resource, see Create a new Application Insights
resource for more information on creating a resource and getting the instrumentation key.
If you are developing a Cloud Service with Azure SDK 2.8 and later, this instrumentation key is
automatically populated. The value is based on the APPINSIGHTS_INSTRUMENTATIONKEY
service configuration setting when packaging the Cloud Service project. See Use Application Insights
with Cloud Services.
The Channels element contains one or more Channel elements.
The name attribute uniquely refers to that channel.
The loglevel attribute lets you specify the log level that the channel allows. The available log levels in
order of most to least information are:
Verbose
Information
Warning
Error
Critical
A channel acts like a filter and allows you to select specific log levels to send to the target sink. For example, you
could collect verbose logs and send them to storage, but send only Errors to the sink.
The following graphic shows this relationship.
The following graphic summarizes the configuration values and how they work. You can include multiple sinks
in the configuration at different levels in the hierarchy. The sink at the top level acts as a global setting and the
one specified at the individual element acts like an override to that global setting.
<SinksConfig>
<Sink name="ApplicationInsights">
<ApplicationInsights>{Insert InstrumentationKey}</ApplicationInsights>
<Channels>
<Channel logLevel="Error" name="MyTopDiagData" />
<Channel logLevel="Verbose" name="MyLogData" />
</Channels>
</Sink>
</SinksConfig>
</WadCfg>
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": 4096,
"sinks": "ApplicationInsights.MyTopDiagData", "_comment": "All info below sent to this channel",
"DiagnosticInfrastructureLogs": {
},
"PerformanceCounters": {
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Processor(_Total)\\% Processor Time",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\Memory\\Available MBytes",
"sampleRate": "PT3M"
}
]
},
"WindowsEventLog": {
"scheduledTransferPeriod": "PT1M",
"DataSource": [
{
"name": "Application!*"
}
]
},
"Logs": {
"scheduledTransferPeriod": "PT1M",
"scheduledTransferLogLevelFilter": "Verbose",
"sinks": "ApplicationInsights.MyLogData", "_comment": "This specific info sent to this channel"
}
},
"SinksConfig": {
"Sink": [
{
"name": "ApplicationInsights",
"ApplicationInsights": "{Insert InstrumentationKey}",
"Channels": {
"Channel": [
{
"logLevel": "Error",
"name": "MyTopDiagData"
},
{
"logLevel": "Verbose",
"name": "MyLogData"
}
]
}
}
]
}
}
In the previous configuration, the following lines have the following meanings:
Send all the data that is being collected by Azure diagnostics
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": 4096,
"sinks": "ApplicationInsights.MyTopDiagData",
}
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": 4096,
"sinks": "ApplicationInsights.MyLogData",
}
Limitations
Channels only log type and not performance counters. If you specify a channel with a performance
counter element, it is ignored.
The log level for a channel cannot exceed the log level for what is being collected by Azure
diagnostics. For example, you cannot collect Application Log errors in the Logs element and try to send
Verbose logs to the Application Insight sink. The scheduledTransferLogLevelFilter attribute must always
collect equal or more logs than the logs you are trying to send to a sink.
You cannot send blob data collected by Azure diagnostics extension to Application Insights. For
example, anything specified under the Directories node. For Crash Dumps the actual crash dump is sent to
blob storage and only a notification that the crash dump was generated is sent to Application Insights.
Next Steps
Learn how to view your Azure diagnostics information in Application Insights.
Use PowerShell to enable the Azure diagnostics extension for your application.
Use Visual Studio to enable the Azure diagnostics extension for your application
Streaming Azure Diagnostics data in the hot path by
using Event Hubs
1/8/2020 • 11 minutes to read • Edit Online
Azure Diagnostics provides flexible ways to collect metrics and logs from cloud services virtual machines (VMs)
and transfer results to Azure Storage. Starting in the March 2016 (SDK 2.9) time frame, you can send Diagnostics
to custom data sources and transfer hot path data in seconds by using Azure Event Hubs.
Supported data types include:
Event Tracing for Windows (ETW ) events
Performance counters
Windows event logs, including application logs in the Windows event log
Azure Diagnostics infrastructure logs
This article shows you how to configure Azure Diagnostics with Event Hubs from end to end. Guidance is also
provided for the following common scenarios:
How to customize the logs and metrics that get sent to Event Hubs
How to change configurations in each environment
How to view Event Hubs stream data
How to troubleshoot the connection
Prerequisites
Event Hubs receiving data from Azure Diagnostics is supported in Cloud Services, VMs, Virtual Machine Scale
Sets, and Service Fabric starting in the Azure SDK 2.9 and the corresponding Azure Tools for Visual Studio.
Azure Diagnostics extension 1.6 (Azure SDK for .NET 2.9 or later targets this by default)
Visual Studio 2013 or later
Existing configurations of Azure Diagnostics in an application by using a .wadcfgx file and one of the following
methods:
Visual Studio: Configuring Diagnostics for Azure Cloud Services and Virtual Machines
Windows PowerShell: Enable diagnostics in Azure Cloud Services using PowerShell
Event Hubs namespace provisioned per the article, Get started with Event Hubs
<SinksConfig>
<Sink name="HotPath">
<EventHub Url="https://diags-mycompany-ns.servicebus.windows.net/diageventhub"
SharedAccessKeyName="SendRule" />
</Sink>
</SinksConfig>
"SinksConfig": {
"Sink": [
{
"name": "HotPath",
"EventHub": {
"Url": "https://diags-mycompany-ns.servicebus.windows.net/diageventhub",
"SharedAccessKeyName": "SendRule"
}
}
]
}
In this example, the event hub URL is set to the fully qualified namespace of the event hub: Event Hubs
namespace + "/" + event hub name.
The event hub URL is displayed in the Azure portal on the Event Hubs dashboard.
The Sink name can be set to any valid string as long as the same value is used consistently throughout the config
file.
NOTE
There may be additional sinks, such as applicationInsights configured in this section. Azure Diagnostics allows one or more
sinks to be defined if each sink is also declared in the PrivateConfig section.
The Event Hubs sink must also be declared and defined in the PrivateConfig section of the .wadcfgx config file.
<PrivateConfig xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">
<StorageAccount name="{account name}" key="{account key}" endpoint="{optional storage endpoint}" />
<EventHub Url="https://diags-mycompany-ns.servicebus.windows.net/diageventhub"
SharedAccessKeyName="SendRule" SharedAccessKey="{base64 encoded key}" />
</PrivateConfig>
{
"storageAccountName": "{account name}",
"storageAccountKey": "{account key}",
"storageAccountEndPoint": "{optional storage endpoint}",
"EventHub": {
"Url": "https://diags-mycompany-ns.servicebus.windows.net/diageventhub",
"SharedAccessKeyName": "SendRule",
"SharedAccessKey": "{base64 encoded key}"
}
}
The SharedAccessKeyName value must match a Shared Access Signature (SAS ) key and policy that has been
defined in the Event Hubs namespace. Browse to the Event Hubs dashboard in the Azure portal, click the
Configure tab, and set up a named policy (for example, "SendRule") that has Send permissions. The
StorageAccount is also declared in PrivateConfig. There is no need to change values here if they are working.
In this example, we leave the values empty, which is a sign that a downstream asset will set the values. For
example, the ServiceConfiguration.Cloud.cscfg environment configuration file sets the environment-appropriate
names and keys.
WARNING
The Event Hubs SAS key is stored in plain text in the .wadcfgx file. Often, this key is checked in to source code control or is
available as an asset in your build server, so you should protect it as appropriate. We recommend that you use a SAS key
here with Send only permissions so that a malicious user can write to the event hub, but not listen to it or manage it.
"PerformanceCounters": {
"scheduledTransferPeriod": "PT1M",
"sinks": "HotPath",
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Processor(_Total)\\% Processor Time",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\Memory\\Available MBytes",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\Web Service(_Total)\\ISAPI Extension Requests/sec",
"sampleRate": "PT3M"
}
]
}
In the above example, the sink is applied to the parent PerformanceCounters node in the hierarchy, which means
all child PerformanceCounters will be sent to Event Hubs.
<PerformanceCounters scheduledTransferPeriod="PT1M">
<PerformanceCounterConfiguration counterSpecifier="\Memory\Available MBytes" sampleRate="PT3M" />
<PerformanceCounterConfiguration counterSpecifier="\Web Service(_Total)\ISAPI Extension Requests/sec"
sampleRate="PT3M" />
<PerformanceCounterConfiguration counterSpecifier="\ASP.NET\Requests Queued" sampleRate="PT3M"
sinks="HotPath" />
<PerformanceCounterConfiguration counterSpecifier="\ASP.NET\Requests Rejected" sampleRate="PT3M"
sinks="HotPath"/>
<PerformanceCounterConfiguration counterSpecifier="\Processor(_Total)\% Processor Time" sampleRate="PT3M"
sinks="HotPath"/>
</PerformanceCounters>
"PerformanceCounters": {
"scheduledTransferPeriod": "PT1M",
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Processor(_Total)\\% Processor Time",
"sampleRate": "PT3M",
"sinks": "HotPath"
},
{
"counterSpecifier": "\\Memory\\Available MBytes",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\Web Service(_Total)\\ISAPI Extension Requests/sec",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\ASP.NET\\Requests Rejected",
"sampleRate": "PT3M",
"sinks": "HotPath"
},
{
"counterSpecifier": "\\ASP.NET\\Requests Queued",
"sampleRate": "PT3M",
"sinks": "HotPath"
}
]
}
In the previous example, the sink is applied to only three counters: Requests Queued, Requests Rejected, and %
Processor time.
The following example shows how a developer can limit the amount of sent data to be the critical metrics that are
used for this service’s health.
"Logs": {
"scheduledTransferPeriod": "PT1M",
"scheduledTransferLogLevelFilter": "Error",
"sinks": "HotPath"
}
In this example, the sink is applied to logs and is filtered only to error level trace.
NOTE
When you make updates to the Azure Diagnostics config file (.wadcfgx), it's recommended that you push the updates to the
entire application as well as the configuration by using either Visual Studio publishing, or a Windows PowerShell script.
namespace EventHubListener
{
class SimpleEventProcessor : IEventProcessor
{
Stopwatch checkpointStopWatch;
//Call checkpoint every 5 minutes, so that worker can resume processing from 5 minutes back if it
restarts.
if (this.checkpointStopWatch.Elapsed > TimeSpan.FromMinutes(5))
{
await context.CheckpointAsync();
this.checkpointStopWatch.Restart();
}
}
}
class Program
{
static void Main(string[] args)
{
string eventHubConnectionString = "Endpoint= <your connection string>";
string eventHubName = "<Event hub name>";
string storageAccountName = "<Storage account name>";
string storageAccountKey = "<Storage account key>";
string storageConnectionString = string.Format("DefaultEndpointsProtocol=https;AccountName=
{0};AccountKey={1}", storageAccountName, storageAccountKey);
Next steps
• Learn more about Event Hubs
The complementary ServiceConfiguration.Cloud.cscfg for this example looks like the following.
<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="MyFixItCloudService"
xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="3" osVersion="*"
schemaVersion="2015-04.2.6">
<Role name="MyFixIt.WorkerRole">
<Instances count="1" />
<ConfigurationSettings>
<Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString"
value="YOUR_CONNECTION_STRING" />
</ConfigurationSettings>
</Role>
</ServiceConfiguration>
{
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": 4096,
"sinks": "applicationInsights.errors",
"DiagnosticInfrastructureLogs": {
"scheduledTransferLogLevelFilter": "Error"
},
"Directories": {
"scheduledTransferPeriod": "PT1M",
"IISLogs": {
"containerName": "wad-iis-logfiles"
},
"FailedRequestLogs": {
"containerName": "wad-failedrequestlogs"
}
},
"PerformanceCounters": {
"scheduledTransferPeriod": "PT1M",
"sinks": "HotPath",
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Memory\\Available MBytes",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\Web Service(_Total)\\ISAPI Extension Requests/sec",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\Web Service(_Total)\\Bytes Total/Sec",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\ASP.NET Applications(__Total__)\\Requests/Sec",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\ASP.NET Applications(__Total__)\\Errors Total/Sec",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\ASP.NET\\Requests Queued",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\ASP.NET\\Requests Rejected",
"sampleRate": "PT3M"
},
{
"counterSpecifier": "\\Processor(_Total)\\% Processor Time",
"sampleRate": "PT3M"
}
]
},
"WindowsEventLog": {
"scheduledTransferPeriod": "PT1M",
"DataSource": [
{
"name": "Application!*"
}
]
},
"Logs": {
"scheduledTransferPeriod": "PT1M",
"scheduledTransferLogLevelFilter": "Error",
"sinks": "HotPath"
}
},
"SinksConfig": {
"Sink": [
{
"name": "HotPath",
"EventHub": {
"Url": "https://diageventhub-py-ns.servicebus.windows.net/diageventhub-py",
"SharedAccessKeyName": "SendRule"
}
},
{
"name": "applicationInsights",
"ApplicationInsights": "",
"Channels": {
"Channel": [
{
"logLevel": "Error",
"name": "errors"
}
]
}
}
]
}
},
"StorageAccount": "{account name}"
}
Protected Settings:
{
"storageAccountName": "{account name}",
"storageAccountKey": "{account key}",
"storageAccountEndPoint": "{storage endpoint}",
"EventHub": {
"Url": "https://diageventhub-py-ns.servicebus.windows.net/diageventhub-py",
"SharedAccessKeyName": "SendRule",
"SharedAccessKey": "YOUR_KEY_HERE"
}
}
Next steps
You can learn more about Event Hubs by visiting the following links:
Event Hubs overview
Create an event hub
Event Hubs FAQ
Send Guest OS metrics to the Azure Monitor metric
store using a Resource Manager template for a
Windows virtual machine
12/23/2019 • 5 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
By using the Azure Monitor Diagnostics extension, you can collect metrics and logs from the guest operating
system (Guest OS ) that's running as part of a virtual machine, cloud service, or Service Fabric cluster. The
extension can send telemetry to many different locations.
This article describes the process for sending Guest OS performance metrics for a Windows virtual machine to the
Azure Monitor data store. Starting with Diagnostics version 1.11, you can write metrics directly to the Azure
Monitor metrics store, where standard platform metrics are already collected.
Storing them in this location allows you to access the same actions for platform metrics. Actions include near-real
time alerting, charting, routing, and access from a REST API and more. In the past, the Diagnostics extension
wrote to Azure Storage, but not to the Azure Monitor data store.
If you're new to Resource Manager templates, learn about template deployments and their structure and syntax.
Prerequisites
Your subscription must be registered with Microsoft.Insights.
You need to have either Azure PowerShell or Azure Cloud Shell installed.
Your VM resource must be in a region that supports custom metrics.
Add this Managed Service Identity (MSI) extension to the template at the top of the resources section. The
extension ensures that Azure Monitor accepts the metrics that are being emitted.
Add the identity configuration to the VM resource to ensure that Azure assigns a system identity to the MSI
extension. This step ensures that the VM can emit guest metrics about itself to Azure Monitor.
// Find this section
"subnet": {
"id": "[variables('subnetRef')]"
}
}
}
]
}
},
{
"apiVersion": "2017-03-30",
"type": "Microsoft.Compute/virtualMachines",
"name": "[variables('vmName')]",
"location": "[resourceGroup().location]",
// add these 3 lines below
"identity": {
"type": "SystemAssigned"
},
//end of added lines
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts/', variables('storageAccountName'))]",
"[resourceId('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
],
"properties": {
"hardwareProfile": {
...
Add the following configuration to enable the Diagnostics extension on a Windows virtual machine. For a simple
Resource Manager-based virtual machine, we can add the extension configuration to theresourcesarray for the
virtual machine. The line "sinks"— "AzMonSink" and the corresponding "SinksConfig" later in the section—enable
the extension to emit metrics directly to Azure Monitor. Feel free to add or remove performance counters as
needed.
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "[reference(resourceId('Microsoft.Storage/storageAccounts/',
variables('storageAccountName'))).primaryEndpoints.blob]"
}
}
},
//Start of section to add
"resources": [
{
"type": "Microsoft.Compute/virtualMachines/extensions",
"name": "[concat(variables('vmName'), '/', 'Microsoft.Insights.VMDiagnosticsSettings')]",
"apiVersion": "2017-12-01",
"location": "[resourceGroup().location]",
"dependsOn": [
"[concat('Microsoft.Compute/virtualMachines/', variables('vmName'))]"
],
"properties": {
"publisher": "Microsoft.Azure.Diagnostics",
"type": "IaaSDiagnostics",
"typeHandlerVersion": "1.12",
"autoUpgradeMinorVersion": true,
"settings": {
"WadCfg": {
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": 4096,
"DiagnosticInfrastructureLogs": {
"scheduledTransferLogLevelFilter": "Error"
},
"Directories": {
"scheduledTransferPeriod": "PT1M",
"IISLogs": {
"containerName": "wad-iis-logfiles"
},
"FailedRequestLogs": {
"containerName": "wad-failedrequestlogs"
}
},
"PerformanceCounters": {
"scheduledTransferPeriod": "PT1M",
"sinks": "AzMonSink",
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Memory\\Available Bytes",
"sampleRate": "PT15S"
},
{
"counterSpecifier": "\\Memory\\% Committed Bytes In Use",
"sampleRate": "PT15S"
},
{
"counterSpecifier": "\\Memory\\Committed Bytes",
"sampleRate": "PT15S"
}
]
},
"WindowsEventLog": {
"scheduledTransferPeriod": "PT1M",
"DataSource": [
{
"name": "Application!*"
}
]
},
"Logs": {
"scheduledTransferPeriod": "PT1M",
"scheduledTransferLogLevelFilter": "Error"
}
},
"SinksConfig": {
"Sink": [
{
"name" : "AzMonSink",
"AzureMonitor" : {}
}
]
}
},
"StorageAccount": "[variables('storageAccountName')]"
},
"protectedSettings": {
"storageAccountName": "[variables('storageAccountName')]",
"storageAccountKey": "[listKeys(variables('accountid'),'2015-06-15').key1]",
"storageAccountEndPoint": "https://core.windows.net/"
}
}
}
]
//End of section to add
5. To create a new resource group for the VM that's being deployed, run the following command:
NOTE
Remember to use an Azure region that is enabled for custom metrics.
6. Run the following commands to deploy the VM using the Resource Manager template.
NOTE
If you wish to update an existing VM, simply add -Mode Incremental to the end of the following command.
7. After your deployment succeeds, the VM should be in the Azure portal, emitting metrics to Azure Monitor.
NOTE
You might run into errors around the selected vmSkuSize. If this happens, go back to your azuredeploy.json file, and
update the default value of the vmSkuSize parameter. In this case, we recommend trying "Standard_DS1_v2").
Next steps
Learn more about custom metrics.
Send guest OS metrics to the Azure Monitor metric
store by using an Azure Resource Manager template
for a Windows virtual machine scale set
12/23/2019 • 6 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
By using the Azure Monitor Windows Azure Diagnostics (WAD ) extension, you can collect metrics and logs from
the guest operating system (guest OS ) that runs as part of a virtual machine, cloud service, or Azure Service Fabric
cluster. The extension can send telemetry to many different locations listed in the previously linked article.
This article describes the process to send guest OS performance metrics for a Windows virtual machine scale set
to the Azure Monitor data store. Starting with Windows Azure Diagnostics version 1.11, you can write metrics
directly to the Azure Monitor metrics store, where standard platform metrics are already collected. By storing them
in this location, you can access the same actions that are available for platform metrics. Actions include near real-
time alerting, charting, routing, access from the REST API, and more. In the past, the Windows Azure Diagnostics
extension wrote to Azure Storage but not the Azure Monitor data store.
If you're new to Resource Manager templates, learn about template deployments and their structure and syntax.
Prerequisites
Your subscription must be registered with Microsoft.Insights.
You need to have Azure PowerShell installed, or you can use Azure Cloud Shell.
Your VM resource must be in a region that supports custom metrics.
"variables": {
//add this line
"storageAccountName": "[concat('storage', uniqueString(resourceGroup().id))]",
Find the virtual machine scale set definition in the resources section and add the identity section to the
configuration. This addition ensures that Azure assigns it a system identity. This step also ensures that the VMs in
the scale set can emit guest metrics about themselves to Azure Monitor:
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[variables('namingInfix')]",
"location": "[resourceGroup().location]",
"apiVersion": "2017-03-30",
//add these lines below
"identity": {
"type": "systemAssigned"
},
//end of lines to add
In the virtual machine scale set resource, find the virtualMachineProfile section. Add a new profile called
extensionsProfile to manage extensions.
In the extensionProfile, add a new extension to the template as shown in the VMSS -WAD -extension section.
This section is the managed identities for Azure resources extension that ensures the metrics being emitted are
accepted by Azure Monitor. The name field can contain any name.
The following code from the MSI extension also adds the diagnostics extension and configuration as an extension
resource to the virtual machine scale set resource. Feel free to add or remove performance counters as needed:
"extensionProfile": {
"extensions": [
// BEGINNING of added code
// Managed identities for Azure resources
{
{
"name": "VMSS-WAD-extension",
"properties": {
"publisher": "Microsoft.ManagedIdentity",
"type": "ManagedIdentityExtensionForWindows",
"typeHandlerVersion": "1.0",
"autoUpgradeMinorVersion": true,
"settings": {
"port": 50342
},
"protectedSettings": {}
}
},
// add diagnostic extension. (Remove this comment after pasting.)
{
"name": "[concat('VMDiagnosticsVmExt','_vmNodeType0Name')]",
"properties": {
"type": "IaaSDiagnostics",
"autoUpgradeMinorVersion": true,
"protectedSettings": {
"storageAccountName": "[variables('storageAccountName')]",
"storageAccountKey": "[listKeys(resourceId('Microsoft.Storage/storageAccounts',
variables('storageAccountName')),'2015-05-01-preview').key1]",
"storageAccountEndPoint": "https://core.windows.net/"
},
"publisher": "Microsoft.Azure.Diagnostics",
"settings": {
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": "50000",
"PerformanceCounters": {
"scheduledTransferPeriod": "PT1M",
"sinks": "AzMonSink",
"PerformanceCounterConfiguration": [
{
"counterSpecifier": "\\Memory\\% Committed Bytes In Use",
"sampleRate": "PT15S"
},
{
"counterSpecifier": "\\Memory\\Available Bytes",
"sampleRate": "PT15S"
},
{
"counterSpecifier": "\\Memory\\Committed Bytes",
"sampleRate": "PT15S"
}
]
},
"EtwProviders": {
"EtwEventSourceProviderConfiguration": [
{
"provider": "Microsoft-ServiceFabric-Actors",
"scheduledTransferKeywordFilter": "1",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableActorEventTable"
}
},
{
"provider": "Microsoft-ServiceFabric-Services",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableServiceEventTable"
}
}
],
"EtwManifestProviderConfiguration": [
{
"provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
"provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
"scheduledTransferLogLevelFilter": "Information",
"scheduledTransferKeywordFilter": "4611686018427387904",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricSystemEventTable"
}
}
]
}
},
"SinksConfig": {
"Sink": [
{
"name": "AzMonSink",
"AzureMonitor": {}
}
]
}
},
"StorageAccount": "[variables('storageAccountName')]"
},
"typeHandlerVersion": "1.11"
}
}
]
}
}
}
},
//end of added code plus a few brackets. Be sure that the number and type of brackets match properly when
done.
{
"type": "Microsoft.Insights/autoscaleSettings",
...
Add a dependsOn for the storage account to ensure it's created in the correct order:
"dependsOn": [
"[concat('Microsoft.Network/loadBalancers/', variables('loadBalancerName'))]",
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
//add this line below
"[concat('Microsoft.Storage/storageAccounts/', variables('storageAccountName'))]"
"resources": [
// add this code
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[variables('storageAccountName')]",
"apiVersion": "2015-05-01-preview",
"location": "[resourceGroup().location]",
"properties": {
"accountType": "Standard_LRS"
}
},
// end added code
{
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
5. Create a new resource group for the VM being deployed. Run the following command:
NOTE
Remember to use an Azure region that's enabled for custom metrics. Remember to use an Azure region that's
enabled for custom metrics.
NOTE
If you want to update an existing scale set, add -Mode Incremental to the end of the command.
7. After your deployment succeeds, you should find the virtual machine scale set in the Azure portal. It should
emit metrics to Azure Monitor.
NOTE
You might run into errors around the selected vmSkuSize. In that case, go back to your azuredeploy.json file and
update the default value of the vmSkuSize parameter. We recommend that you try Standard_DS1_v2.
Next steps
Learn more about custom metrics.
Send Guest OS metrics to the Azure Monitor metrics
database for a Windows virtual machine (classic)
1/14/2020 • 4 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
The Azure Monitor Diagnostics extension (known as "WAD" or "Diagnostics") allows you to collect metrics and
logs from the guest operating system (Guest OS ) running as part of a virtual machine, cloud service, or Service
Fabric cluster. The extension can send telemetry to many different locations.
This article describes the process for sending Guest OS performance metrics for a Windows virtual machine
(classic) to the Azure Monitor metric database. Starting with Diagnostics version 1.11, you can write metrics
directly to the Azure Monitor metrics store, where standard platform metrics are already collected.
Storing them in this location allows you to access the same actions as you do for platform metrics. Actions include
near-real time alerting, charting, routing, access from a REST API, and more. In the past, the Diagnostics extension
wrote to Azure Storage, but not to the Azure Monitor data store.
The process that's outlined in this article only works on classic virtual machines that are running the Windows
operating system.
Prerequisites
You must be a service administrator or co-administrator on your Azure subscription.
Your subscription must be registered with Microsoft.Insights.
You need to have either Azure PowerShell or Azure Cloud Shell installed.
Your VM resource must be in a region that supports custom metrics.
NOTE
The Diagnostics extension uses the service principal to authenticate against Azure Monitor and emit metrics for your classic
VM.
2. In the “SinksConfig” section of your diagnostics file, define a new Azure Monitor sink, as follows:
<SinksConfig>
<Sink name="AzMonSink">
<AzureMonitor>
<ResourceId>Provide the resource ID of your classic VM </ResourceId>
<Region>The region your VM is deployed in</Region>
</AzureMonitor>
</Sink>
</SinksConfig>
3. In the section of your configuration file where the list of performance counters to be collected is listed, route
the performance counters to the Azure Monitor sink "AzMonSink".
<PerformanceCounters scheduledTransferPeriod="PT1M" sinks="AzMonSink">
<PerformanceCounterConfiguration counterSpecifier="\Processor(_Total)\% Processor Time"
sampleRate="PT15S" />
...
</PerformanceCounters>
4. In the private configuration, define the Azure Monitor account. Then add the service principal information to
use to emit metrics.
<PrivateConfig xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">
<StorageAccount name="" endpoint="" />
<AzureMonitorAccount>
<ServicePrincipalMeta>
<PrincipalId>clientId for your service principal</PrincipalId>
<Secret>client secret of your service principal</Secret>
</ServicePrincipalMeta>
</AzureMonitorAccount>
</PrivateConfig>
Login-AzAccount
3. Set the context of the classic storage account that was created with the VM.
4. Set the Diagnostics file path to a variable by using the following command:
$diagconfig = “<path of the diagnostics configuration file with the Azure Monitor sink configured>”
5. Prepare the update for your classic VM with the diagnostics file that has the Azure Monitor sink configured.
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
With the Azure Monitor Diagnostics extension, you can collect metrics and logs from the guest operating system
(Guest OS ) running as part of a virtual machine, cloud service, or Service Fabric cluster. The extension can send
telemetry to many different locations.
This article describes the process for sending Guest OS performance metrics for Azure classic Cloud Services to
the Azure Monitor metric store. Starting with Diagnostics version 1.11, you can write metrics directly to the Azure
Monitor metrics store, where standard platform metrics are already collected.
Storing them in this location allows you to access the same actions that you can for platform metrics. Actions
include near-real time alerting, charting, routing, access from a REST API, and more. In the past, the Diagnostics
extension wrote to Azure Storage, but not to the Azure Monitor data store.
The process that's outlined in this article works only for performance counters in Azure Cloud Services. It doesn't
work for other custom metrics.
Prerequisites
You must be a service administrator or co-administrator on your Azure subscription.
Your subscription must be registered with Microsoft.Insights.
You need to have either Azure PowerShell or Azure Cloud Shell installed.
Your Cloud Service must be in a region that supports custom metrics.
NOTE
The Diagnostics extension uses the service principal to authenticate against Azure Monitor and emit metrics for your cloud
service.
In the "SinksConfig" section of your diagnostics file, define a new Azure Monitor sink:
<SinksConfig>
<Sink name="AzMonSink">
<AzureMonitor>
<ResourceId>-Provide ClassicCloudService’s Resource ID-</ResourceId>
<Region>-Azure Region your Cloud Service is deployed in-</Region>
</AzureMonitor>
</Sink>
</SinksConfig>
In the section of your configuration file where you list the performance counters to collect, add the Azure Monitor
sink. This entry ensures that all the performance counters that you specified are routed to Azure Monitor as
metrics. You can add or remove performance counters according to your needs.
Finally, in the private configuration, add an Azure Monitor Account section. Enter the service principal client ID and
secret that you created earlier.
<PrivateConfig xmlns="http://schemas.microsoft.com/ServiceHosting/2010/10/DiagnosticsConfiguration">
<StorageAccount name="" endpoint="" />
<AzureMonitorAccount>
<ServicePrincipalMeta>
<PrincipalId>clientId from step 3</PrincipalId>
<Secret>client secret from step 3</Secret>
</ServicePrincipalMeta>
</AzureMonitorAccount>
</PrivateConfig>
Login-AzAccount
Use the following commands to store the details of the storage account that you created earlier.
Similarly, set the diagnostics file path to a variable by using the following command:
$diagconfig = “<path of the Diagnostics configuration file with the Azure Monitor sink configured>”
Deploy the Diagnostics extension to your cloud service with the diagnostics file with the Azure Monitor sink
configured using the following command:
NOTE
It's still mandatory to provide a storage account as part of the installation of the Diagnostics extension. Any logs or
performance counters that are specified in the diagnostics config file are written to the specified storage account.
Next steps
Learn more about custom metrics.
Prepare for format change to Azure Monitor resource
logs archived to a storage account
12/6/2019 • 4 minutes to read • Edit Online
WARNING
If you are sending Azure resource resource logs or metrics to a storage account using resource diagnostic settings or activity
logs to a storage account using log profiles, the format of the data in the storage account will change to JSON Lines on Nov.
1, 2018. The instructions below describe the impact and how to update your tooling to handle the new format.
What is changing
Azure Monitor offers a capability that enables you to send resource diagnostic data and activity log data into an
Azure storage account, Event Hubs namespace, or into a Log Analytics workspace in Azure Monitor. In order to
address a system performance issue, on November 1, 2018 at 12:00 midnight UTC the format of log data send
to blob storage will change. If you have tooling that is reading data out of blob storage, you need to update your
tooling to understand the new data format.
On Thursday, November 1, 2018 at 12:00 midnight UTC, the blob format will change to be JSON Lines. This
means each record will be delimited by a newline, with no outer records array and no commas between JSON
records.
The blob format changes for all diagnostic settings across all subscriptions at once. The first PT1H.json file
emitted for November 1 will use this new format. The blob and container names remain the same.
Setting a diagnostic setting between now and November 1 continues to emit data in the current format until
November 1.
This change will occur at once across all public cloud regions. The change will not occur in Microsoft Azure
Operated by 21Vianet, Azure Germany, or Azure Government clouds yet.
This change impacts the following data types:
Azure resource resource logs (see list of resources here)
Azure resource metrics being exported by diagnostic settings
Azure Activity log data being exported by log profiles
This change does not impact:
Network flow logs
Azure service logs not made available through Azure Monitor yet (for example, Azure App Service
resource logs, storage analytics logs)
Routing of Azure resource logs and activity logs to other destinations (Event Hubs, Log Analytics)
How to see if you are impacted
You are only impacted by this change if you:
1. Are sending log data to an Azure storage account using a resource diagnostic setting, and
2. Have tooling that depends on the JSON structure of these logs in storage.
To identify if you have resource diagnostic settings that are sending data to an Azure storage account, you can
navigate to the Monitor section of the portal, click on Diagnostic Settings, and identify any resources that have
Diagnostic Status set to Enabled:
If Diagnostic Status is set to enabled, you have an active diagnostic setting on that resource. Click on the resource
to see if any diagnostic settings are sending data to a storage account:
If you do have resources sending data to a storage account using these resource diagnostic settings, the format of
the data in that storage account will be impacted by this change. Unless you have custom tooling that operates off
of these storage accounts, the format change will not impact you.
Details of the format change
The current format of the PT1H.json file in Azure blob storage uses a JSON array of records. Here is a sample of a
KeyVault log file now:
{
"records": [
{
"time": "2016-01-05T01:32:01.2691226Z",
"resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT",
"operationName": "VaultGet",
"operationVersion": "2015-06-01",
"category": "AuditEvent",
"resultType": "Success",
"resultSignature": "OK",
"resultDescription": "",
"durationMs": "78",
"callerIpAddress": "104.40.82.76",
"correlationId": "",
"identity": {
"claim": {
"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-bd64-XXXXXXXXXXXX",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn": "live.com#username@outlook.com",
"appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"
}
},
"properties": {
"clientInfo": "azure-resource-manager/2.0",
"requestUri": "https://control-prod-wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01",
"id": "https://contosokeyvault.vault.azure.net/",
"httpStatusCode": 200
}
},
{
"time": "2016-01-05T01:33:56.5264523Z",
"resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT",
"operationName": "VaultGet",
"operationVersion": "2015-06-01",
"category": "AuditEvent",
"resultType": "Success",
"resultSignature": "OK",
"resultDescription": "",
"durationMs": "83",
"callerIpAddress": "104.40.82.76",
"correlationId": "",
"identity": {
"claim": {
"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-bd64-XXXXXXXXXXXX",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn": "live.com#username@outlook.com",
"appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"
}
},
"properties": {
"clientInfo": "azure-resource-manager/2.0",
"requestUri": "https://control-prod-wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01",
"id": "https://contosokeyvault.vault.azure.net/",
"httpStatusCode": 200
}
}
]
}
The new format uses JSON lines, where each event is a line and the newline character indicates a new event. Here
is what the above sample will look like in the PT1H.json file after the change:
{"time": "2016-01-05T01:32:01.2691226Z","resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT","operationName":
"VaultGet","operationVersion": "2015-06-01","category": "AuditEvent","resultType": "Success","resultSignature":
"OK","resultDescription": "","durationMs": "78","callerIpAddress": "104.40.82.76","correlationId":
"","identity": {"claim": {"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-
bd64-XXXXXXXXXXXX","http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn":
"live.com#username@outlook.com","appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"}},"properties": {"clientInfo":
"azure-resource-manager/2.0","requestUri": "https://control-prod-
wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01","id": "https://contosokeyvault.vault.azure.net/","httpStatusCode": 200}}
{"time": "2016-01-05T01:33:56.5264523Z","resourceId": "/SUBSCRIPTIONS/361DA5D4-A47A-4C79-AFDD-
XXXXXXXXXXXX/RESOURCEGROUPS/CONTOSOGROUP/PROVIDERS/MICROSOFT.KEYVAULT/VAULTS/CONTOSOKEYVAULT","operationName":
"VaultGet","operationVersion": "2015-06-01","category": "AuditEvent","resultType": "Success","resultSignature":
"OK","resultDescription": "","durationMs": "83","callerIpAddress": "104.40.82.76","correlationId":
"","identity": {"claim": {"http://schemas.microsoft.com/identity/claims/objectidentifier": "d9da5048-2737-4770-
bd64-XXXXXXXXXXXX","http://schemas.xmlsoap.org/ws/2005/05/identity/claims/upn":
"live.com#username@outlook.com","appid": "1950a258-227b-4e31-a9cf-XXXXXXXXXXXX"}},"properties": {"clientInfo":
"azure-resource-manager/2.0","requestUri": "https://control-prod-
wus.vaultcore.azure.net/subscriptions/361da5d4-a47a-4c79-afdd-
XXXXXXXXXXXX/resourcegroups/contosoresourcegroup/providers/Microsoft.KeyVault/vaults/contosokeyvault?api-
version=2015-06-01","id": "https://contosokeyvault.vault.azure.net/","httpStatusCode": 200}}
This new format enables Azure Monitor to push log files using append blobs, which are more efficient for
continuously appending new event data.
How to update
You only need to make updates if you have custom tooling that ingests these log files for further processing. If you
are making use of an external log analytics or SIEM tool, we recommend using event hubs to ingest this data
instead. Event hubs integration is easier in terms of processing logs from many services and bookmarking location
in a particular log.
Custom tools should be updated to handle both the current format and the JSON Lines format described above.
This will ensure that when data starts to appear in the new format, your tools do not break.
Next steps
Learn about archiving resource resource logs to a storage account
Learn about archiving activity log data to a storage account
Azure Diagnostics troubleshooting
12/23/2019 • 12 minutes to read • Edit Online
This article describes troubleshooting information that's relevant to using Azure Diagnostics. For more information
about Azure diagnostics, see Azure Diagnostics overview.
Logical components
Diagnostics Plugin Launcher (DiagnosticsPluginLauncher.exe): Launches the Azure Diagnostics extension.
Serves as the entry point process.
Diagnostics Plugin (DiagnosticsPlugin.exe): Configures, launches, and manages the lifetime of the monitoring
agent. It is the main process that is launched by the launcher.
Monitoring Agent (MonAgent*.exe processes): Monitors, collects, and transfers the diagnostics data.
Log/artifact paths
Following are the paths to some important logs and artifacts. We refer to this information throughout the rest of
the document.
Azure Cloud Services
ARTIFACT PATH
Virtual machines
ARTIFACT PATH
ARTIFACT PATH
If you find a negative exit code, refer to the exit code table in the References section.
Verify that the provided storage account is correct. Make sure you don't have network restrictions that
prevent the components from reaching public storage endpoints. One way to do that is to remote access
into the machine, and then try to write something to the same storage account yourself.
Finally, you can look at what failures are being reported by the monitoring Agent. The monitoring agent
writes its logs in maeventtable.tsf , which is located in the local store for diagnostics data. Follow the
instructions in the Local log extraction section for opening this file. Then try to determine if there are
errors that indicate failures reading to local files writing to storage.
if (String.IsNullOrEmpty(eventDestination)) {
if (e == "DefaultEvents")
tableName = "WADDefault" + MD5(provider);
else
tableName = "WADEvent" + MD5(provider) + eventId;
}
else
tableName = "WAD" + eventDestination;
Here is an example:
<EtwEventSourceProviderConfiguration provider="prov1">
<Event id="1" />
<Event id="2" eventDestination="dest1" />
<DefaultEvents />
</EtwEventSourceProviderConfiguration>
<EtwEventSourceProviderConfiguration provider="prov2">
<DefaultEvents eventDestination="dest2" />
</EtwEventSourceProviderConfiguration>
"EtwEventSourceProviderConfiguration": [
{
"provider": "prov1",
"Event": [
{
"id": 1
},
{
"id": 2,
"eventDestination": "dest1"
}
],
"DefaultEvents": {
"eventDestination": "DefaultEventDestination",
"sinks": ""
}
},
{
"provider": "prov2",
"DefaultEvents": {
"eventDestination": "dest2"
}
}
]
References
How to check Diagnostics extension configuration
The easiest way to check your extension configuration is to go to Azure Resource Explorer, and then go to the
virtual machine or cloud service where the Azure Diagnostics extension (IaaSDiagnostics / PaaDiagnostics) is.
Alternatively, remote desktop into the machine and look at the Azure Diagnostics Configuration file that's
described in the Log artifacts path section.
In either case, search for Microsoft.Azure.Diagnostics, and then for the xmlCfg or WadCfg field.
If you're searching on a virtual machine and the WadCfg field is present, it means the config is in JSON format. If
the xmlCfg field is present, it means the config is in XML and is base64-encoded. You need to decode it to see the
XML that was loaded by Diagnostics.
For the cloud service role, if you pick the configuration from disk, the data is base64-encoded, so you need to
decode it to see the XML that was loaded by Diagnostics.
Azure Diagnostics plugin exit codes
The plugin returns the following exit codes:
0 Success.
-1 Generic error.
-11 The guest agent was unable to create the process responsible
for launching and monitoring the monitoring agent.
Solution: Verify that sufficient system resources are
available to launch new processes.
-104 Unable to load the rcf file provided by the guest agent.
This internal error should only happen if the guest agent
plugin launcher is manually invoked incorrectly on the
VM.
A new file called <relevantLogFile>.csv is created in the same path as the corresponding .tsf file.
NOTE
You only need to run this utility against the main .tsf file (for example, PerformanceCountersTable.tsf). The accompanying files
(for example, PerformanceCountersTables_**001.tsf, PerformanceCountersTables_**002.tsf, and so on) are automatically
processed.
NOTE
The following information applies mostly to Azure Cloud Services unless you have configured the
DiagnosticsMonitorTraceListener on an application that's running on your IaaS VM.
This document describes version 3.0 and newer of the Linux Diagnostic Extension.
IMPORTANT
For information about version 2.3 and older, see this document.
Introduction
The Linux Diagnostic Extension helps a user monitor the health of a Linux VM running on Microsoft Azure. It has
the following capabilities:
Collects system performance metrics from the VM and stores them in a specific table in a designated storage
account.
Retrieves log events from syslog and stores them in a specific table in the designated storage account.
Enables users to customize the data metrics that are collected and uploaded.
Enables users to customize the syslog facilities and severity levels of events that are collected and uploaded.
Enables users to upload specified log files to a designated storage table.
Supports sending metrics and log events to arbitrary EventHub endpoints and JSON -formatted blobs in the
designated storage account.
This extension works with both Azure deployment models.
# Download the sample Public settings. (You could also use curl or any web browser)
wget https://raw.githubusercontent.com/Azure/azure-linux-
extensions/master/Diagnostic/tests/lad_2_3_compatible_portal_pub_settings.json -O portal_public_settings.json
# Build the VM resource ID. Replace storage account name and resource ID in the public settings.
my_vm_resource_id=$(az vm show -g $my_resource_group -n $my_linux_vm --query "id" -o tsv)
sed -i "s#__DIAGNOSTIC_STORAGE_ACCOUNT__#$my_diagnostic_storage_account#g" portal_public_settings.json
sed -i "s#__VM_RESOURCE_ID__#$my_vm_resource_id#g" portal_public_settings.json
The URL for the sample configuration, and its contents, are subject to change. Download a copy of the portal
settings JSON file and customize it for your needs. Any templates or automation you construct should use your
own copy, rather than downloading that URL each time.
PowerShell sample
// Set your Azure VM diagnostics variables correctly below - don't forget to replace the VMResourceID
$SASKey = '<SASKeyForDiagStorageAccount>'
$ladCfg = "{
'diagnosticMonitorConfiguration': {
'performanceCounters': {
'sinks': 'WADMetricEventHub,WADMetricJsonBlob',
'performanceCounterConfiguration': [
{
'unit': 'Percent',
'type': 'builtin',
'counter': 'PercentProcessorTime',
'counterSpecifier': '/builtin/Processor/PercentProcessorTime',
'annotation': [
{
'locale': 'en-us',
'displayName': 'Aggregate CPU %utilization'
}
],
'condition': 'IsAggregate=TRUE',
'class': 'Processor'
},
},
{
'unit': 'Bytes',
'type': 'builtin',
'counter': 'UsedSpace',
'counterSpecifier': '/builtin/FileSystem/UsedSpace',
'annotation': [
{
'locale': 'en-us',
'displayName': 'Used disk space on /'
}
],
'condition': 'Name='/'',
'class': 'Filesystem'
}
]
},
'metrics': {
'metricAggregation': [
{
'scheduledTransferPeriod': 'PT1H'
},
{
'scheduledTransferPeriod': 'PT1M'
}
],
'resourceId': '<VMResourceID>'
},
'eventVolume': 'Large',
'syslogEvents': {
'sinks': 'SyslogJsonBlob,LoggingEventHub',
'syslogEventConfiguration': {
'LOG_USER': 'LOG_INFO'
}
}
}
}"
$ladCfg = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($ladCfg))
$perfCfg = "[
{
'query': 'SELECT PercentProcessorTime, PercentIdleTime FROM SCX_ProcessorStatisticalInformation WHERE
Name='_TOTAL'',
'table': 'LinuxCpu',
'frequency': 60,
'sinks': 'LinuxCpuJsonBlob'
}
]"
IMPORTANT
This extension introduces breaking changes to the configuration of the extension. One such change was made to improve
the security of the extension; as a result, backwards compatibility with 2.x could not be maintained. Also, the Extension
Publisher for this extension is different than the publisher for the 2.x versions.
To migrate from 2.x to this new version of the extension, you must uninstall the old extension (under the old publisher
name), then install version 3 of the extension.
Recommendations:
Install the extension with automatic minor version upgrade enabled.
On classic deployment model VMs, specify '3.*' as the version if you are installing the extension through
Azure XPLAT CLI or Powershell.
On Azure Resource Manager deployment model VMs, include '"autoUpgradeMinorVersion": true' in the
VM deployment template.
Use a new/different storage account for LAD 3.0. There are several small incompatibilities between LAD 2.3
and LAD 3.0 that make sharing an account troublesome:
LAD 3.0 stores syslog events in a table with a different name.
The counterSpecifier strings for builtin metrics differ in LAD 3.0.
Protected settings
This set of configuration information contains sensitive information that should be protected from public view, for
example, storage credentials. These settings are transmitted to and stored by the extension in encrypted form.
{
"storageAccountName" : "the storage account to receive data",
"storageAccountEndPoint": "the hostname suffix for the cloud for this account",
"storageAccountSasToken": "SAS access token",
"mdsdHttpProxy": "HTTP proxy settings",
"sinksConfig": { ... }
}
NAME VALUE
storageAccountSasToken An Account SAS token for Blob and Table services ( ss='bt' ),
applicable to containers and objects ( srt='co' ), which
grants add, create, list, update, and write permissions (
sp='acluw' ). Do not include the leading question-mark (?).
To get a SAS token within a Resource Manager template, use the listAccountSas function. For an example
template, see List function example.
You can easily construct the required SAS token through the Azure portal.
1. Select the general-purpose storage account to which you want the extension to write
2. Select "Shared access signature" from the Settings part of the left menu
3. Make the appropriate sections as previously described
4. Click the "Generate SAS" button.
Copy the generated SAS into the storageAccountSasToken field; remove the leading question-mark ("?").
sinksConfig
"sinksConfig": {
"sink": [
{
"name": "sinkname",
"type": "sinktype",
...
},
...
]
},
This optional section defines additional destinations to which the extension sends the information it collects. The
"sink" array contains an object for each additional data sink. The "type" attribute determines the other attributes in
the object.
ELEMENT VALUE
type The type of sink being defined. Determines the other values (if
any) in instances of this type.
Version 3.0 of the Linux Diagnostic Extension supports two sink types: EventHub, and JsonBlob.
The EventHub sink
"sink": [
{
"name": "sinkname",
"type": "EventHub",
"sasURL": "https SAS URL"
},
...
]
The "sasURL" entry contains the full URL, including SAS token, for the Event Hub to which data should be
published. LAD requires a SAS naming a policy that enables the Send claim. An example:
Create an Event Hubs namespace called contosohub
Create an Event Hub in the namespace called syslogmsgs
Create a Shared access policy on the Event Hub named writer that enables the Send claim
If you created a SAS good until midnight UTC on January 1, 2018, the sasURL value might be:
https://contosohub.servicebus.windows.net/syslogmsgs?
sr=contosohub.servicebus.windows.net%2fsyslogmsgs&sig=xxxxxxxxxxxxxxxxxxxxxxxxx&se=1514764800&skn=writer
For more information about generating and retrieving information on SAS tokens for Event Hubs, see this web
page.
The JsonBlob sink
"sink": [
{
"name": "sinkname",
"type": "JsonBlob"
},
...
]
Data directed to a JsonBlob sink is stored in blobs in Azure storage. Each instance of LAD creates a blob every
hour for each sink name. Each blob always contains a syntactically valid JSON array of object. New entries are
atomically added to the array. Blobs are stored in a container with the same name as the sink. The Azure storage
rules for blob container names apply to the names of JsonBlob sinks: between 3 and 63 lower-case alphanumeric
ASCII characters or dashes.
Public settings
This structure contains various blocks of settings that control the information collected by the extension. Each
setting is optional. If you specify ladCfg , you must also specify StorageAccount .
{
"ladCfg": { ... },
"perfCfg": { ... },
"fileLogs": { ... },
"StorageAccount": "the storage account to receive data",
"mdsdHttpProxy" : ""
}
ELEMENT VALUE
"ladCfg": {
"diagnosticMonitorConfiguration": {
"eventVolume": "Medium",
"metrics": { ... },
"performanceCounters": { ... },
"syslogEvents": { ... }
},
"sampleRateInSeconds": 15
}
This optional structure controls the gathering of metrics and logs for delivery to the Azure Metrics service and to
other data sinks. You must specify either performanceCounters or syslogEvents or both. You must specify the
metrics structure.
ELEMENT VALUE
metrics
"metrics": {
"resourceId": "/subscriptions/...",
"metricAggregation" : [
{ "scheduledTransferPeriod" : "PT1H" },
{ "scheduledTransferPeriod" : "PT5M" }
]
}
ELEMENT VALUE
Samples of the metrics specified in the performanceCounters section are collected every 15 seconds or at the
sample rate explicitly defined for the counter. If multiple scheduledTransferPeriod frequencies appear (as in the
example), each aggregation is computed independently.
performanceCounters
"performanceCounters": {
"sinks": "",
"performanceCounterConfiguration": [
{
"type": "builtin",
"class": "Processor",
"counter": "PercentIdleTime",
"counterSpecifier": "/builtin/Processor/PercentIdleTime",
"condition": "IsAggregate=TRUE",
"sampleRate": "PT15S",
"unit": "Percent",
"annotation": [
{
"displayName" : "Aggregate CPU %idle time",
"locale" : "en-us"
}
]
}
]
}
This optional section controls the collection of metrics. Raw samples are aggregated for each
scheduledTransferPeriod to produce these values:
mean
minimum
maximum
last-collected value
count of raw samples used to compute the aggregate
ELEMENT VALUE
counter Together with "class", identifies the specific metric within the
provider's namespace.
sampleRate IS 8601 interval that sets the rate at which raw samples for
this metric are collected. If not set, the collection interval is set
by the value of sampleRateInSeconds. The shortest supported
sample rate is 15 seconds (PT15S).
displayName The label (in the language specified by the associated locale
setting) to be attached to this data in Azure Metrics. LAD
ignores this field.
The counterSpecifier is an arbitrary identifier. Consumers of metrics, like the Azure portal charting and alerting
feature, use counterSpecifier as the "key" that identifies a metric or an instance of a metric. For builtin metrics,
we recommend you use counterSpecifier values that begin with /builtin/ . If you are collecting a specific instance
of a metric, we recommend you attach the identifier of the instance to the counterSpecifier value. Some examples:
/builtin/Processor/PercentIdleTime - Idle time averaged across all vCPUs
/builtin/Disk/FreeSpace(/mnt) - Free space for the /mnt filesystem
/builtin/Disk/FreeSpace - Free space averaged across all mounted filesystems
Neither LAD nor the Azure portal expects the counterSpecifier value to match any pattern. Be consistent in how
you construct counterSpecifier values.
When you specify performanceCounters , LAD always writes data to a table in Azure storage. You can have the
same data written to JSON blobs and/or Event Hubs, but you cannot disable storing data to a table. All instances
of the diagnostic extension configured to use the same storage account name and endpoint add their metrics and
logs to the same table. If too many VMs are writing to the same table partition, Azure can throttle writes to that
partition. The eventVolume setting causes entries to be spread across 1 (Small), 10 (Medium), or 100 (Large)
different partitions. Usually, "Medium" is sufficient to ensure traffic is not throttled. The Azure Metrics feature of
the Azure portal uses the data in this table to produce graphs or to trigger alerts. The table name is the
concatenation of these strings:
WADMetrics
The "scheduledTransferPeriod" for the aggregated values stored in the table
P10DV2S
A date, in the form "YYYYMMDD", which changes every 10 days
Examples include WADMetricsPT1HP10DV2S20170410 and WADMetricsPT1MP10DV2S20170609 .
syslogEvents
"syslogEvents": {
"sinks": "",
"syslogEventConfiguration": {
"facilityName1": "minSeverity",
"facilityName2": "minSeverity",
...
}
}
This optional section controls the collection of log events from syslog. If the section is omitted, syslog events are
not captured at all.
The syslogEventConfiguration collection has one entry for each syslog facility of interest. If minSeverity is
"NONE" for a particular facility, or if that facility does not appear in the element at all, no events from that facility
are captured.
ELEMENT VALUE
When you specify syslogEvents , LAD always writes data to a table in Azure storage. You can have the same data
written to JSON blobs and/or Event Hubs, but you cannot disable storing data to a table. The partitioning
behavior for this table is the same as described for performanceCounters . The table name is the concatenation of
these strings:
LinuxSyslog
A date, in the form "YYYYMMDD", which changes every 10 days
Examples include LinuxSyslog20170410 and LinuxSyslog20170609 .
perfCfg
This optional section controls execution of arbitrary OMI queries.
"perfCfg": [
{
"namespace": "root/scx",
"query": "SELECT PercentAvailableMemory, PercentUsedSwap FROM SCX_MemoryStatisticalInformation",
"table": "LinuxOldMemory",
"frequency": 300,
"sinks": ""
}
]
ELEMENT VALUE
namespace (optional) The OMI namespace within which the query should
be executed. If unspecified, the default value is "root/scx",
implemented by the System Center Cross-platform Providers.
"fileLogs": [
{
"file": "/var/log/mydaemonlog",
"table": "MyDaemonEvents",
"sinks": ""
}
]
ELEMENT VALUE
ELEMENT VALUE
file The full pathname of the log file to be watched and captured.
The pathname must name a single file; it cannot name a
directory or contain wildcards.
COUNTER MEANING
The first four counters should sum to 100%. The last three counters also sum to 100%; they subdivide the sum of
PercentProcessorTime, PercentIOWaitTime, and PercentInterruptTime.
To obtain a single metric aggregated across all processors, set "condition": "IsAggregate=TRUE" . To obtain a metric
for a specific processor, such as the second logical processor of a four-vCPU VM, set "condition": "Name=\\"1\\"" .
Logical processor numbers are in the range [0..n-1] .
builtin metrics for the Memory class
The Memory class of metrics provides information about memory utilization, paging, and swapping.
COUNTER MEANING
PagesReadPerSec Pages read from backing store (swap file, program file,
mapped file, etc.)
PagesWrittenPerSec Pages written to backing store (swap file, mapped file, etc.)
This class of metrics has only a single instance. The "condition" attribute has no useful settings and should be
omitted.
builtin metrics for the Network class
The Network class of metrics provides information about network activity on an individual network interfaces
since boot. LAD does not expose bandwidth metrics, which can be retrieved from host metrics.
COUNTER MEANING
Although this class is instanced, LAD does not support capturing Network metrics aggregated across all network
devices. To obtain the metrics for a specific interface, such as eth0, set "condition": "InstanceID=\\"eth0\\"" .
builtin metrics for the Filesystem class
The Filesystem class of metrics provides information about filesystem usage. Absolute and percentage values are
reported as they'd be displayed to an ordinary user (not root).
COUNTER MEANING
Aggregated values across all file systems can be obtained by setting "condition": "IsAggregate=True" . Values for a
specific mounted file system, such as "/mnt", can be obtained by setting "condition": 'Name="/mnt"' .
NOTE: If using the Azure Portal instead of JSON, the correct condition field form is Name='/mnt'
builtin metrics for the Disk class
The Disk class of metrics provides information about disk device usage. These statistics apply to the entire drive. If
there are multiple file systems on a device, the counters for that device are, effectively, aggregated across all of
them.
COUNTER MEANING
Aggregated values across all disks can be obtained by setting "condition": "IsAggregate=True" . To get information
for a specific device (for example, /dev/sdf1), set "condition": "Name=\\"/dev/sdf1\\"" .
The command assumes you are using the Azure Resource Management mode (arm) of the Azure CLI. To
configure LAD for classic deployment model (ASM ) VMs, switch to "asm" mode ( azure config mode asm ) and omit
the resource group name in the command. For more information, see the cross-platform CLI documentation.
PublicConfig.json
These public settings cause LAD to:
Upload percent-processor-time and used-disk-space metrics to the WADMetrics* table
Upload messages from syslog facility "user" and severity "info" to the LinuxSyslog* table
Upload raw OMI query results (PercentProcessorTime and PercentIdleTime) to the named LinuxCPU table
Upload appended lines in file /var/log/myladtestlog to the MyLadTestLog table
In each case, data is also uploaded to:
Azure Blob storage (container name is as defined in the JsonBlob sink)
EventHubs endpoint (as specified in the EventHubs sink)
{
"StorageAccount": "yourdiagstgacct",
"ladCfg": {
"ladCfg": {
"sampleRateInSeconds": 15,
"diagnosticMonitorConfiguration": {
"performanceCounters": {
"sinks": "MyMetricEventHub,MyJsonMetricsBlob",
"performanceCounterConfiguration": [
{
"unit": "Percent",
"type": "builtin",
"counter": "PercentProcessorTime",
"counterSpecifier": "/builtin/Processor/PercentProcessorTime",
"annotation": [
{
"locale": "en-us",
"displayName": "Aggregate CPU %utilization"
}
],
"condition": "IsAggregate=TRUE",
"class": "Processor"
},
{
"unit": "Bytes",
"type": "builtin",
"counter": "UsedSpace",
"counterSpecifier": "/builtin/FileSystem/UsedSpace",
"annotation": [
{
"locale": "en-us",
"displayName": "Used disk space on /"
}
],
"condition": "Name=\"/\"",
"class": "Filesystem"
}
]
},
"metrics": {
"metricAggregation": [
{
"scheduledTransferPeriod": "PT1H"
},
{
"scheduledTransferPeriod": "PT1M"
}
],
"resourceId":
"/subscriptions/your_azure_subscription_id/resourceGroups/your_resource_group_name/providers/Microsoft.Compute
/virtualMachines/your_vm_name"
},
"eventVolume": "Large",
"syslogEvents": {
"sinks": "SyslogJsonBlob,LoggingEventHub",
"syslogEventConfiguration": {
"LOG_USER": "LOG_INFO"
}
}
}
},
"perfCfg": [
{
"query": "SELECT PercentProcessorTime, PercentIdleTime FROM SCX_ProcessorStatisticalInformation WHERE
Name='_TOTAL'",
"table": "LinuxCpu",
"frequency": 60,
"sinks": "LinuxCpuJsonBlob,LinuxCpuEventHub"
}
],
"fileLogs": [
{
"file": "/var/log/myladtestlog",
"file": "/var/log/myladtestlog",
"table": "MyLadTestLog",
"sinks": "FilelogJsonBlob,LoggingEventHub"
}
]
}
The resourceId in the configuration must match that of the VM or the virtual machine scale set.
Azure platform metrics charting and alerting knows the resourceId of the VM you're working on. It expects to
find the data for your VM using the resourceId the lookup key.
If you use Azure autoscale, the resourceId in the autoscale configuration must match the resourceId used by
LAD.
The resourceId is built into the names of JsonBlobs written by LAD.
The performanceCounters data are always stored in an Azure Storage table. Azure Storage APIs are available for
many languages and platforms.
Data sent to JsonBlob sinks is stored in blobs in the storage account named in the Protected settings. You can
consume the blob data using any Azure Blob Storage APIs.
In addition, you can use these UI tools to access the data in Azure Storage:
Visual Studio Server Explorer.
Microsoft Azure Storage Explorer.
This snapshot of a Microsoft Azure Storage Explorer session shows the generated Azure Storage tables and
containers from a correctly configured LAD 3.0 extension on a test VM. The image doesn't match exactly with the
sample LAD 3.0 configuration.
See the relevant EventHubs documentation to learn how to consume messages published to an EventHubs
endpoint.
Next steps
Create metric alerts in Azure Monitor for the metrics you collect.
Create monitoring charts for your metrics.
Learn how to create a virtual machine scale set using your metrics to control autoscaling.
Collect custom metrics for a Linux VM with the
InfluxData Telegraf agent
12/6/2019 • 4 minutes to read • Edit Online
By using Azure Monitor, you can collect custom metrics via your application telemetry, an agent running on your
Azure resources, or even outside-in monitoring systems. Then you can submit them directly to Azure Monitor.
This article provides instructions on how to deploy the InfluxData Telegraf agent on a Linux VM in Azure and
configure the agent to publish metrics to Azure Monitor.
8. Select a size for the VM. You can filter by Compute type or Disk type, for example.
9. On the Settings page in Network > Network Security Group > Select public inbound ports, select
HTTP and SSH (22). Leave the rest of the defaults and select OK.
10. On the summary page, select Create to start the VM deployment.
11. The VM is pinned to the Azure portal dashboard. After the deployment finishes, the VM summary
automatically opens.
12. In the VM pane, navigate to the Identity tab. Ensure that your VM has a system-assigned identity set to
On.
Connect to the VM
Create an SSH connection with the VM. Select the Connect button on the overview page for your VM.
In the Connect to virtual machine page, keep the default options to connect by DNS name over port 22. In
Login using VM local account, a connection command is shown. Select the button to copy the command. The
following example shows what the SSH connection command looks like:
ssh azureuser@XXXX.XX.XXX
Paste the SSH connection command into a shell, such as Azure Cloud Shell or Bash on Ubuntu on Windows, or
use an SSH client of your choice to create the connection.
Telegraf’s configuration file defines Telegraf’s operations. By default, an example configuration file is installed at
the path /etc/telegraf/telegraf.conf. The example configuration file lists all possible input and output plug-ins.
However, we'll create a custom configuration file and have the agent use it by running the following commands:
NOTE
The preceding code enables only two input plug-ins: cpu and mem. You can add more input plug-ins, depending on the
workload that runs on your machine. Examples are Docker, MySQL, and NGINX. For a full list of input plug-ins, see the
Additional configuration section.
Finally, to have the agent start using the new configuration, we force the agent to stop and start by running the
following commands:
Now the agent will collect metrics from each of the input plug-ins specified and emit them to Azure Monitor.
4. Select the Telegraf/CPU namespace, and select the usage_system metric. You can choose to filter by the
dimensions on this metric or split on them.
Additional configuration
The preceding walkthrough provides information on how to configure the Telegraf agent to collect metrics from a
few basic input plug-ins. The Telegraf agent has support for over 150 input plug-ins, with some supporting
additional configuration options. InfluxData has published a list of supported plugins and instructions on how to
configure them.
Additionally, in this walkthrough, you used the Telegraf agent to emit metrics about the VM the agent is deployed
on. The Telegraf agent can also be used as a collector and forwarder of metrics for other resources. To learn how
to configure the agent to emit metrics for other Azure resources, see Azure Monitor Custom Metric Output for
Telegraf.
Clean up resources
When they're no longer needed, you can delete the resource group, virtual machine, and all related resources. To
do so, select the resource group for the virtual machine and select Delete. Then confirm the name of the resource
group to delete.
Next steps
Learn more about custom metrics.
Collect log data with the Log Analytics agent
1/22/2020 • 7 minutes to read • Edit Online
The Azure Log Analytics agent, previously referred to as the Microsoft Monitoring Agent (MMA) or OMS Linux
agent, was developed for comprehensive management across on-premises machines, computers monitored by
System Center Operations Manager, and virtual machines in any cloud. The Windows and Linux agents attach to
Azure Monitor and store collected log data from different sources in your Log Analytics workspace, as well as
any unique logs or metrics as defined in a monitoring solution.
This article provides a detailed overview of the agent, system and network requirements, and the different
deployment methods.
Overview
Before analyzing and acting on collected data, you first need to install and connect agents for all of the machines
that you want to send data to the Azure Monitor service. You can install agents on your Azure VMs using the
Azure Log Analytics VM extension for Windows and Linux, and for machines in a hybrid environment using
setup, command line, or with Desired State Configuration (DSC ) in Azure Automation.
The agent for Linux and Windows communicates outbound to the Azure Monitor service over TCP port 443, and
if the machine connects through a firewall or proxy server to communicate over the Internet, review
requirements below to understand the network configuration required. If your IT security policies do not allow
computers on the network to connect to the Internet, you can set up a Log Analytics gateway and then configure
the agent to connect through the gateway to Azure Monitor logs. The agent can then receive configuration
information and send data collected depending on what data collection rules and monitoring solutions you have
enabled in your workspace.
When using the Log Analytics agents to collect data, you need to understand the following in order to plan your
agent deployment:
To collect data from Windows agents, you can configure each agent to report to one or more workspaces,
even while it is reporting to a System Center Operations Manager management group. The Windows agent
can report up to four workspaces.
The Linux agent does not support multi-homing and can only report to a single workspace.
The Windows agent supports the FIPS 140 standard, while the Linux agent does not support it.
If you are using System Center Operations Manager 2012 R2 or later:
Each Operations Manager management group can be connected to only one workspace.
Linux computers reporting to a management group must be configured to report directly to a Log Analytics
workspace. If your Linux computers are already reporting directly to a workspace and you want to monitor
them with Operations Manager, follow these steps to report to an Operations Manager management group.
You can install the Log Analytics Windows agent on the Windows computer and have it report to both
Operations Manager integrated with a workspace, and a different workspace.
The agent for Linux and Windows isn't only for connecting to Azure Monitor, it also supports Azure Automation
to host the Hybrid Runbook worker role and other services such as Change Tracking, Update Management, and
Azure Security Center. For more information about the Hybrid Runbook Worker role, see Azure Automation
Hybrid Runbook Worker.
NOTE
While the Log Analytics agent for Windows was designed to support server monitoring scenarios, we realize you may run
Windows client to support workloads configured and optimized for the server operating system. The agent does support
Windows client, however our monitoring solutions don't focus on client monitoring scenarios unless explicitly stated.
NOTE
If you are using a distro or version that is not currently supported and doesn't align to our support model, we recommend
that you fork this repo, acknowledging that Microsoft support will not provide assistance with forked agent versions.
NOTE
OpenSSL 1.1.0 is only supported on x86_x64 platforms (64-bit) and OpenSSL earlier than 1.x is not supported on any
platform.
Agent prerequisites
The following table highlights the packages required for supported Linux distros that the agent will be installed
on.
Python-ctypes
NOTE
Either rsyslog or syslog-ng are required to collect syslog messages. The default syslog daemon on version 5 of Red Hat
Enterprise Linux, CentOS, and Oracle Linux version (sysklog) is not supported for syslog event collection. To collect syslog
data from this version of these distributions, the rsyslog daemon should be installed and configured to replace sysklog.
For firewall information required for Azure Government, see Azure Government management.
If you plan to use the Azure Automation Hybrid Runbook Worker to connect to and register with the
Automation service to use runbooks or management solutions in your environment, it must have access to the
port number and the URLs described in Configure your network for the Hybrid Runbook Worker.
The Windows and Linux agent supports communicating either through a proxy server or Log Analytics gateway
to Azure Monitor using the HTTPS protocol. Both anonymous and basic authentication (username/password)
are supported. For the Windows agent connected directly to the service, the proxy configuration is specified
during installation or after deployment from Control Panel or with PowerShell.
For the Linux agent, the proxy server is specified during installation or after installation by modifying the
proxy.conf configuration file. The Linux agent proxy configuration value has the following syntax:
[protocol://][user:password@]proxyhost[:port]
NOTE
If your proxy server does not require you to authenticate, the Linux agent still requires providing a pseudo user/password.
This can be any username or password.
PROPERTY DESCRIPTION
Protocol https
NOTE
If you use special characters such as “@” in your password, you receive a proxy connection error because value is parsed
incorrectly. To work around this issue, encode the password in the URL using a tool such as URLDecode.
Azure VM - Log Analytics VM extension for - The extension installs the Log
Windows or Linux using Azure CLI or Analytics agent on Azure virtual
with an Azure Resource Manager machines and enrolls them into an
template existing Azure Monitor workspace.
- Manually from the Azure portal - Azure Security Center can provision
- Azure Security Center Automatic the Log Analytics agent on all
provisioning supported Azure VMs and any new
ones that are created if you enable it to
monitor for security vulnerabilities and
threats. If enabled, any new or existing
VM without an installed agent will be
provisioned.
Hybrid Windows computer - Manual install Install the Microsoft Monitoring agent
- Azure Automation DSC from the command line or using an
- Resource Manager template with automated method such as Azure
Azure Stack Automation DSC, Configuration
Manager, or with an Azure Resource
Manager template if you have
deployed Microsoft Azure Stack in your
datacenter.
Hybrid Linux computer Manual install Install the agent for Linux calling a
wrapper-script hosted on GitHub.
System Center Operations Manager Integrate Operations Manager with Configure integration between
Log Analytics Operations Manager and Azure
Monitor logs to forward collected data
from Windows computers reporting to
a management group.
Next steps
Review data sources to understand the data sources available to collect data from your Windows or Linux
system.
Learn about log queries to analyze the data collected from data sources and solutions.
Learn about monitoring solutions that add functionality to Azure Monitor and also collect data into the
Log Analytics workspace.
Connect Windows computers to Azure Monitor
1/6/2020 • 9 minutes to read • Edit Online
In order to monitor and manage virtual machines or physical computers in your local datacenter or other cloud
environment with Azure Monitor, you need to deploy the Log Analytics agent (also referred to as the Microsoft
Monitoring Agent (MMA)) and configure it to report to one or more Log Analytics workspaces. The agent also
supports the Hybrid Runbook Worker role for Azure Automation.
On a monitored Windows computer, the agent is listed as the Microsoft Monitoring Agent service. The Microsoft
Monitoring Agent service collects events from log files and Windows event log, performance data, and other
telemetry. Even when the agent is unable to communicate with Azure Monitor it reports to, the agent continues
to run and queues the collected data on the disk of the monitored computer. When the connection is restored,
the Microsoft Monitoring Agent service sends collected data to the service.
The agent may be installed by using one of the following methods. Most installations use a combination of these
methods to install different sets of computers, as appropriate. Details on using each method are provided later in
the article.
Manual installation. Setup is manually run on the computer using the setup wizard, from the command line,
or deployed using an existing software distribution tool.
Azure Automation Desired State Configuration (DSC ). Using DSC in Azure Automation with a script for
Windows computers already deployed in your environment.
PowerShell script.
Resource Manager template for virtual machines running Windows on-premises in Azure Stack.
NOTE
Azure Security Center (ASC) depends on the Microsoft Monitoring Agent (also referred to as the Log Analytics Windows
agent) and will install and configure it to report to a Log Analytics workspace as part of its deployment. ASC includes an
automatic provisioning option which enables automatic installation of the Log Analytics Windows agent on all VMs in your
subscription and configures it to report to a specific workspace. For more information about this option, see Enable
automatic provisioning of Log Analytics agent.
If you need to configure the agent to report to more than one workspace, this cannot be performed during initial
setup, only afterwards by updating the settings from Control Panel or PowerShell as described in Adding or
removing a workspace.
To understand the supported configuration, review supported Windows operating systems and network firewall
configuration.
NOTE
If you are configuring a VM running Windows Server 2008 SP2 x64 to use TLS 1.2, you first need to install the following
SHA-2 code signing support update before performing the steps below.
NOTE
If you want to upgrade an agent, you need to use the Log Analytics scripting API. See the topic Managing and maintaining
the Log Analytics agent for Windows and Linux for further information.
The following table highlights the specific parameters supported by setup for the agent, including when
deployed using Automation DSC.
1. To extract the agent installation files, from an elevated command prompt run MMASetup-<platform>.exe /c
and it will prompt you for the path to extract files to. Alternatively, you can specify the path by passing the
arguments MMASetup-<platform>.exe /c /t:<Full Path> .
2. To silently install the agent and configure it to report to a workspace in Azure commercial cloud, from the
folder you extracted the setup files to type:
NOTE
The string values for the parameters OPINSIGHTS_WORKSPACE_ID and OPINSIGHTS_WORKSPACE_KEY need to be
encapsulated in double-quotes to instruct Windows Installer to interprit as valid options for the package.
NOTE
This procedure and script example does not support upgrading the agent already deployed to a Windows computer.
The 32-bit and 64-bit versions of the agent package have different product codes and new versions released also
have a unique value. The product code is a GUID that is the principal identification of an application or product
and is represented by the Windows Installer ProductCode property. The ProductId value in the
MMAgent.ps1 script has to match the product code from the 32-bit or 64-bit agent installer package.
To retrieve the product code from the agent install package directly, you can use Orca.exe from the Windows
SDK Components for Windows Installer Developers that is a component of the Windows Software
Development Kit or using PowerShell following an example script written by a Microsoft Valuable Professional
(MVP ). For either approach, you first need to extract the MOMagent.msi file from the MMASetup installation
package. This is shown earlier in the first step under the section Install the agent using the command line.
1. Import the xPSDesiredStateConfiguration DSC Module from
https://www.powershellgallery.com/packages/xPSDesiredStateConfiguration into Azure Automation.
2. Create Azure Automation variable assets for OPSINSIGHTS_WS_ID and OPSINSIGHTS_WS_KEY. Set
OPSINSIGHTS_WS_ID to your Log Analytics workspace ID and set OPSINSIGHTS_WS_KEY to the
primary key of your workspace.
3. Copy the script and save it as MMAgent.ps1.
Configuration MMAgent
{
$OIPackageLocalPath = "C:\Deploy\MMASetup-AMD64.exe"
$OPSINSIGHTS_WS_ID = Get-AutomationVariable -Name "OPSINSIGHTS_WS_ID"
$OPSINSIGHTS_WS_KEY = Get-AutomationVariable -Name "OPSINSIGHTS_WS_KEY"
Node OMSnode {
Service OIService
{
Name = "HealthService"
State = "Running"
DependsOn = "[Package]OI"
}
xRemoteFile OIPackage {
Uri = "https://go.microsoft.com/fwlink/?LinkId=828603"
DestinationPath = $OIPackageLocalPath
}
Package OI {
Ensure = "Present"
Path = $OIPackageLocalPath
Name = "Microsoft Monitoring Agent"
ProductId = "8A7F2C51-4C7D-4BFD-9014-91D11F24AAE2"
Arguments = '/C:"setup.exe /qn NOAPM=1 ADD_OPINSIGHTS_WORKSPACE=1 OPINSIGHTS_WORKSPACE_ID=' +
$OPSINSIGHTS_WS_ID + ' OPINSIGHTS_WORKSPACE_KEY=' + $OPSINSIGHTS_WS_KEY + '
AcceptEndUserLicenseAgreement=1"'
DependsOn = "[xRemoteFile]OIPackage"
}
}
}
4. Update the ProductId value in the script with the product code extracted from the latest version of the
agent install package using the methods recommended earlier.
5. Import the MMAgent.ps1 configuration script into your Automation account.
6. Assign a Windows computer or node to the configuration. Within 15 minutes, the node checks its
configuration and the agent is pushed to the node.
You can also perform a simple log query in the Azure portal.
1. In the Azure portal, search for and select Monitor.
2. Select Logs in the menu.
3. On the Logs pane, in the query field type:
Heartbeat
| where Category == "Direct Agent"
| where TimeGenerated > ago(30m)
In the search results returned, you should see heartbeat records for the computer indicating it is connected and
reporting to the service.
Next steps
Review Managing and maintaining the Log Analytics agent for Windows and Linux to learn about how to
reconfigure, upgrade, or remove the agent from the virtual machine.
Review Troubleshooting the Windows agent if you encounter issues while installing or managing the
agent.
Connect Linux computers to Azure Monitor
1/21/2020 • 7 minutes to read • Edit Online
In order to monitor and manage virtual machines or physical computers in your local datacenter or other cloud
environment with Azure Monitor, you need to deploy the Log Analytics agent and configure it to report to a Log
Analytics workspace. The agent also supports the Hybrid Runbook Worker role for Azure Automation.
The Log Analytics agent for Linux can be installed by using one of the following methods. Details on using each
method are provided later in the article.
Manually download and install the agent. This is required when the Linux computer does not have access to the
Internet and will be communicating with Azure Monitor or Azure Automation through the Log Analytics
gateway.
Install the agent for Linux using a wrapper-script hosted on GitHub. This is the recommended method to install
and upgrade the agent when the computer has connectivity with the Internet, directly or through a proxy server.
To understand the supported configuration, review supported Linux operating systems and network firewall
configuration.
NOTE
The Log Analytics agent for Linux cannot be configured to report to more than one Log Analytics workspace. It can only be
configured to report to both a System Center Operations Manager management group and Log Analytics workspace
concurrently, or to either individually.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
1. In the upper-left corner of the Azure portal, select All services. In the search box, enter Log Analytics. As
you type, the list filters based on your input. Select Log Analytics workspaces.
2. In your list of Log Analytics workspaces, select the workspace you created earlier. (You might have named it
DefaultLAWorkspace.)
3. Select Advanced settings:
4. Select Connected Sources, and then select Linux Servers.
5. The value to the right of Workspace ID and Primary Key. Copy and paste both into your favorite editor.
NOTE
For Azure VMs, we recommend you install the agent on them using the Azure Log Analytics VM extension for Linux.
1. Download and transfer the appropriate bundle (x64 or x86) to your Linux VM or physical computer, using
scp/sftp.
2. Install the bundle by using the --install argument. To onboard to a Log Analytics workspace during
installation, provide the -w <WorkspaceID> and -s <workspaceKey> parameters copied earlier.
NOTE
You need to use the --upgrade argument if any dependent packages such as omi, scx, omsconfig or their older
versions are installed, as would be the case if the system Center Operations Manager agent for Linux is already
installed.
3. To configure the Linux agent to install and connect to a Log Analytics workspace through a Log Analytics
gateway, run the following command providing the proxy, workspace ID, and workspace key parameters.
This configuration can be specified on the command line by including
-p [protocol://][user:password@]proxyhost[:port] . The proxyhost property accepts a fully qualified domain
name or IP address of the Log Analytics gateway server.
If authentication is required, you need to specify the username and password. For example:
4. To configure the Linux computer to connect to a Log Analytics workspace in Azure Government cloud, run
the following command providing the workspace ID and primary key copied earlier.
If you want to install the agent packages and configure it to report to a specific Log Analytics workspace at a later
time, run the following command:
If you want to extract the agent packages from the bundle without installing the agent, run the following command:
If authentication is required in either case, you need to specify the username and password. For example:
https://user01:password@proxy01.contoso.com:30443
1. To configure the Linux computer to connect to a Log Analytics workspace, run the following command
providing the workspace ID and primary key. The following command downloads the agent, validates its
checksum, and installs it.
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -w <YOUR WORKSPACE ID> -s <YOUR
WORKSPACE PRIMARY KEY>
The following command includes the -p proxy parameter and example syntax when authentication is
required by your proxy server:
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -p [protocol://]<proxy user>:
<proxy password>@<proxyhost>[:port] -w <YOUR WORKSPACE ID> -s <YOUR WORKSPACE PRIMARY KEY>
2. To configure the Linux computer to connect to Log Analytics workspace in Azure Government cloud, run the
following command providing the workspace ID and primary key copied earlier. The following command
downloads the agent, validates its checksum, and installs it.
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -w <YOUR WORKSPACE ID> -s <YOUR
WORKSPACE PRIMARY KEY> -d opinsights.azure.us
The following command includes the -p proxy parameter and example syntax when authentication is
required by your proxy server:
wget https://raw.githubusercontent.com/Microsoft/OMS-Agent-for-
Linux/master/installer/scripts/onboard_agent.sh && sh onboard_agent.sh -p [protocol://]<proxy user>:
<proxy password>@<proxyhost>[:port] -w <YOUR WORKSPACE ID> -s <YOUR WORKSPACE PRIMARY KEY> -d
opinsights.azure.us
Next steps
Review Managing and maintaining the Log Analytics agent for Windows and Linux to learn about how to
reconfigure, upgrade, or remove the agent from the virtual machine.
Review Troubleshooting the Linux agent if you encounter issues while installing or managing the agent.
2 minutes to read
2 minutes to read
Connect computers without internet access by using
the Log Analytics gateway in Azure Monitor
1/23/2020 • 19 minutes to read • Edit Online
NOTE
As Microsoft Operations Management Suite (OMS) transitions to Microsoft Azure Monitor, terminology is changing. This
article refers to OMS Gateway as the Azure Log Analytics gateway.
This article describes how to configure communication with Azure Automation and Azure Monitor by using the
Log Analytics gateway when computers that are directly connected or that are monitored by Operations Manager
have no internet access.
The Log Analytics gateway is an HTTP forward proxy that supports HTTP tunneling using the HTTP CONNECT
command. This gateway sends data to Azure Automation and a Log Analytics workspace in Azure Monitor on
behalf of the computers that cannot directly connect to the internet.
The Log Analytics gateway supports:
Reporting up to the same Log Analytics workspaces configured on each agent behind it and that are
configured with Azure Automation Hybrid Runbook Workers.
Windows computers on which the Microsoft Monitoring Agent is directly connected to a Log Analytics
workspace in Azure Monitor.
Linux computers on which a Log Analytics agent for Linux is directly connected to a Log Analytics workspace
in Azure Monitor.
System Center Operations Manager 2012 SP1 with UR7, Operations Manager 2012 R2 with UR3, or a
management group in Operations Manager 2016 or later that is integrated with Log Analytics.
Some IT security policies don't allow internet connection for network computers. These unconnected computers
could be point of sale (POS ) devices or servers supporting IT services, for example. To connect these devices to
Azure Automation or a Log Analytics workspace so you can manage and monitor them, configure them to
communicate directly with the Log Analytics gateway. The Log Analytics gateway can receive configuration
information and forward data on their behalf. If the computers are configured with the Log Analytics agent to
directly connect to a Log Analytics workspace, the computers instead communicate with the Log Analytics
gateway.
The Log Analytics gateway transfers data from the agents to the service directly. It doesn't analyze any of the data
in transit and the gateway does not cache data when it loses connectivity with the service. When the gateway is
unable to communicate with service, the agent continues to run and queues the collected data on the disk of the
monitored computer. When the connection is restored, the agent sends the cached data collected to Azure
Monitor.
When an Operations Manager management group is integrated with Log Analytics, the management servers can
be configured to connect to the Log Analytics gateway to receive configuration information and send collected
data, depending on the solution you have enabled. Operations Manager agents send some data to the
management server. For example, agents might send Operations Manager alerts, configuration assessment data,
instance space data, and capacity data. Other high-volume data, such as Internet Information Services (IIS ) logs,
performance data, and security events, is sent directly to the Log Analytics gateway.
If one or more Operations Manager Gateway servers are deployed to monitor untrusted systems in a perimeter
network or an isolated network, those servers can't communicate with a Log Analytics gateway. Operations
Manager Gateway servers can report only to a management server. When an Operations Manager management
group is configured to communicate with the Log Analytics gateway, the proxy configuration information is
automatically distributed to every agent-managed computer that is configured to collect log data for Azure
Monitor, even if the setting is empty.
To provide high availability for directly connected or Operations Management groups that communicate with a
Log Analytics workspace through the gateway, use network load balancing (NLB ) to redirect and distribute traffic
across multiple gateway servers. That way, if one gateway server goes down, the traffic is redirected to another
available node.
The computer that runs the Log Analytics gateway requires the Log Analytics Windows agent to identify the
service endpoints that the gateway needs to communicate with. The agent also needs to direct the gateway to
report to the same workspaces that the agents or Operations Manager management group behind the gateway
are configured with. This configuration allows the gateway and the agent to communicate with their assigned
workspace.
A gateway can be multihomed to up to four workspaces. This is the total number of workspaces a Windows agent
supports.
Each agent must have network connectivity to the gateway so that agents can automatically transfer data to and
from the gateway. Avoid installing the gateway on a domain controller. Linux computers that are behind a gateway
server cannot use the wrapper script installation method to install the Log Analytics agent for Linux. The agent
must be downloaded manually, copied to the computer, and installed manually because the gateway only
supports communicating with the Azure services mentioned earlier.
The following diagram shows data flowing from direct agents, through the gateway, to Azure Automation and
Log Analytics. The agent proxy configuration must match the port that the Log Analytics gateway is configured
with.
The following diagram shows data flow from an Operations Manager management group to Log Analytics.
Set up your system
Computers designated to run the Log Analytics gateway must have the following configuration:
Windows 10, Windows 8.1, or Windows 7
Windows Server 2019, Windows Server 2016, Windows Server 2012 R2, Windows Server 2012, Windows
Server 2008 R2, or Windows Server 2008
Microsoft .NET Framework 4.5
At least a 4-core processor and 8 GB of memory
A Log Analytics agent for Windows that is configured to report to the same workspace as the agents that
communicate through the gateway
Language availability
The Log Analytics gateway is available in these languages:
Chinese (Simplified)
Chinese (Traditional)
Czech
Dutch
English
French
German
Hungarian
Italian
Japanese
Korean
Polish
Portuguese (Brazil)
Portuguese (Portugal)
Russian
Spanish (International)
Supported encryption protocols
The Log Analytics gateway supports only Transport Layer Security (TLS ) 1.0, 1.1, and 1.2. It doesn't support
Secure Sockets Layer (SSL ). To ensure the security of data in transit to Log Analytics, configure the gateway to
use at least TLS 1.2. Older versions of TLS or SSL are vulnerable. Although they currently allow backward
compatibility, avoid using them.
For additional information, review Sending data securely using TLS 1.2.
Supported number of agent connections
The following table shows approximately how many agents can communicate with a gateway server. Support is
based on agents that upload about 200 KB of data every 6 seconds. For each agent tested, data volume is about
2.7 GB per day.
3. On the License Agreement page, select I accept the terms in the License Agreement to agree to the
Microsoft Software License Terms, and then select Next.
4. On the Port and proxy address page:
a. Enter the TCP port number to be used for the gateway. Setup uses this port number to configure an
inbound rule on Windows Firewall. The default value is 8080. The valid range of the port number is 1
through 65535. If the input does not fall into this range, an error message appears.
b. If the server where the gateway is installed needs to communicate through a proxy, enter the proxy
address where the gateway needs to connect. For example, enter http://myorgname.corp.contoso.com:80 . If
you leave this field blank, the gateway will try to connect to the internet directly. If your proxy server
requires authentication, enter a username and password.
c. Select Next.
5. If you do not have Microsoft Update enabled, the Microsoft Update page appears, and you can choose to
enable it. Make a selection and then select Next. Otherwise, continue to the next step.
6. On the Destination Folder page, either leave the default folder C:\Program Files\OMS Gateway or enter
the location where you want to install the gateway. Then select Next.
7. On the Ready to install page, select Install. If User Account Control requests permission to install, select
Yes.
8. After Setup finishes, select Finish. To verify that the service is running, open the services.msc snap-in and
verify that OMS Gateway appears in the list of services and that its status is Running.
PARAMETERS NOTES
To silently install the gateway and configure it with a specific proxy address, port number, type the following:
Using the /qn command-line option hides setup, /qb shows setup during silent install.
If you need to provide credentials to authenticate with the proxy, type the following:
After installation, you can confirm the settings are accepted (excluding the username and password) using the
following PowerShell cmdlets:
Get-OMSGatewayConfig – Returns the TCP Port the gateway is configured to listen on.
Get-OMSGatewayRelayProxy – Returns the IP address of the proxy server you configured it to
communicate with.
NOTE
Configuring the Azure Load Balancer using the Basic SKU, requires that Azure virtual machines belong to an Availability Set.
To learn more about availability sets, see Manage the availability of Windows virtual machines in Azure. To add existing
virtual machines to an availability set, refer to Set Azure Resource Manager VM Availability Set.
After the load balancer is created, a backend pool needs to be created, which distributes traffic to one or more
gateway servers. Follow the steps described in the quickstart article section Create resources for the load
balancer.
NOTE
When configuring the health probe it should be configured to use the TCP port of the gateway server. The health probe
dynamically adds or removes the gateway servers from the load balancer rotation based on their response to health checks.
NOTE
To install the Log Analytics agent on the gateway and Windows computers that directly connect to Log Analytics, see
Connect Windows computers to the Log Analytics service in Azure. To connect Linux computers, see Connect Linux
computers to Azure Monitor.
After you install the agent on the gateway server, configure it to report to the workspace or workspace agents that
communicate with the gateway. If the Log Analytics Windows agent is not installed on the gateway, event 300 is
written to the OMS Gateway event log, indicating that the agent needs to be installed. If the agent is installed but
not configured to report to the same workspace as the agents that communicate through it, event 105 is written
to the same log, indicating that the agent on the gateway needs to be configured to report to the same workspace
as the agents that communicate with the gateway.
After you complete configuration, restart the OMS Gateway service to apply the changes. Otherwise, the
gateway will reject agents that attempt to communicate with Log Analytics and will report event 105 in the OMS
Gateway event log. This will also happen when you add or remove a workspace from the agent configuration on
the gateway server.
For information related to the Automation Hybrid Runbook Worker, see Automate resources in your datacenter
or cloud by using Hybrid Runbook Worker.
Configure Operations Manager, where all agents use the same proxy server
The Operations Manager proxy configuration is automatically applied to all agents that report to Operations
Manager, even if the setting is empty.
To use OMS Gateway to support Operations Manager, you must have:
Microsoft Monitoring Agent (version 8.0.10900.0 or later) installed on the OMS Gateway server and
configured with the same Log Analytics workspaces that your management group is configured to report to.
Internet connectivity. Alternatively, OMS Gateway must be connected to a proxy server that is connected to the
internet.
NOTE
If you specify no value for the gateway, blank values are pushed to all agents.
If your Operations Manager management group is registering with a Log Analytics workspace for the first time,
you won't see the option to specify the proxy configuration for the management group in the Operations console.
This option is available only if the management group has been registered with the service.
To configure integration, update the system proxy configuration by using Netsh on the system where you're
running the Operations console and on all management servers in the management group. Follow these steps:
1. Open an elevated command prompt:
a. Select Start and enter cmd.
b. Right-click Command Prompt and select Run as administrator.
2. Enter the following command:
netsh winhttp set proxy <proxy>:<port>
After completing the integration with Log Analytics, remove the change by running netsh winhttp reset proxy .
Then, in the Operations console, use the Configure proxy server option to specify the Log Analytics gateway
server.
1. On the Operations Manager console, under Operations Management Suite, select Connection, and
then select Configure Proxy Server.
2. Select Use a proxy server to access the Operations Management Suite and then enter the IP address
of the Log Analytics gateway server or virtual IP address of the load balancer. Be careful to start with the
prefix http:// .
3. Select Finish. Your Operations Manager management group is now configured to communicate through
the gateway server to the Log Analytics service.
Configure Operations Manager, where specific agents use a proxy server
For large or complex environments, you might want only specific servers (or groups) to use the Log Analytics
gateway server. For these servers, you can't update the Operations Manager agent directly because this value is
overwritten by the global value for the management group. Instead, override the rule used to push these values.
NOTE
Use this configuration technique if you want to allow for multiple Log Analytics gateway servers in your environment. For
example, you can require specific Log Analytics gateway servers to be specified on a regional basis.
To configure specific servers or groups to use the Log Analytics gateway server:
1. Open the Operations Manager console and select the Authoring workspace.
2. In the Authoring workspace, select Rules.
3. On the Operations Manager toolbar, select the Scope button. If this button is not available, make sure you
have selected an object, not a folder, in the Monitoring pane. The Scope Management Pack Objects
dialog box displays a list of common targeted classes, groups, or objects.
4. In the Look for field, enter Health Service and select it from the list. Select OK.
5. Search for Advisor Proxy Setting Rule.
6. On the Operations Manager toolbar, select Overrides and then point to Override the Rule\For a specific
object of class: Health Service and select an object from the list. Or create a custom group that contains
the health service object of the servers you want to apply this override to. Then apply the override to your
custom group.
7. In the Override Properties dialog box, add a check mark in the Override column next to the
WebProxyAddress parameter. In the Override Value field, enter the URL of the Log Analytics gateway
server. Be careful to start with the prefix http:// .
NOTE
You don't need to enable the rule. It's already managed automatically with an override in the Microsoft System
Center Advisor Secure Reference Override management pack that targets the Microsoft System Center Advisor
Monitoring Server Group.
8. Select a management pack from the Select destination management pack list, or create a new unsealed
management pack by selecting New.
9. When you finish, select OK.
Configure for Automation Hybrid Runbook Workers
If you have Automation Hybrid Runbook Workers in your environment, follow these steps to configure the
gateway to support the workers.
Refer to the Configure your network section of the Automation documentation to find the URL for each region.
If your computer is registered as a Hybrid Runbook Worker automatically, for example if the Update Management
solution is enabled for one or more VMs, follow these steps:
1. Add the Job Runtime Data service URLs to the Allowed Host list on the Log Analytics gateway. For example:
Add-OMSGatewayAllowedHost we-jobruntimedata-prod-su1.azure-automation.net
2. Restart the Log Analytics gateway service by using the following PowerShell cmdlet:
Restart-Service OMSGatewayService
If your computer is joined to Azure Automation by using the Hybrid Runbook Worker registration cmdlet, follow
these steps:
1. Add the agent service registration URL to the Allowed Host list on the Log Analytics gateway. For example:
Add-OMSGatewayAllowedHost ncus-agentservice-prod-1.azure-automation.net
2. Add the Job Runtime Data service URLs to the Allowed Host list on the Log Analytics gateway. For example:
Add-OMSGatewayAllowedHost we-jobruntimedata-prod-su1.azure-automation.net
3. Restart the Log Analytics gateway service. Restart-Service OMSGatewayService
Set- Address Sets the address (and 1. Set a relay proxy and
OMSGatewayRelayProxy Username credential) of relay credential:
Password (upstream) proxy Set-
OMSGatewayRelayProxy
-Address
http://www.myproxy.com:8080
-Username user1 -
Password 123
Troubleshooting
To collect events logged by the gateway, you should have the Log Analytics agent installed.
ID DESCRIPTION
The Log Analytics gateway supports only TLS 1.0, TLS 1.1,
and 1.2. It does not support SSL.
NAME DESCRIPTION
Log Analytics Gateway/Active Client Connection Number of active client network (TCP) connections
Log Analytics Gateway/Rejection Count Number of rejections due to any TLS validation error
Assistance
When you're signed in to the Azure portal, you can get help with the Log Analytics gateway or any other Azure
service or feature. To get help, select the question mark icon in the upper-right corner of the portal and select
New support request. Then complete the new support request form.
Next steps
Add data sources to collect data from connected sources, and store the data in your Log Analytics workspace.
Managing and maintaining the Log Analytics agent
for Windows and Linux
12/13/2019 • 9 minutes to read • Edit Online
After initial deployment of the Log Analytics Windows or Linux agent in Azure Monitor, you may need to
reconfigure the agent, upgrade it, or remove it from the computer if it has reached the retirement stage in its
lifecycle. You can easily manage these routine maintenance tasks manually or through automation, which reduces
both operational error and expenses.
Upgrading agent
The Log Analytics agent for Windows and Linux can be upgraded to the latest release manually or automatically
depending on the deployment scenario and environment the VM is running in. The following methods can be
used to upgrade the agent.
Custom Azure VM images Manual install of Log Analytics agent Updating VMs to the newest version of
for Windows/Linux the agent needs to be performed from
the command line running the
Windows installer package or Linux self-
extracting and installable shell script
bundle.
Non-Azure VMs Manual install of Log Analytics agent Updating VMs to the newest version of
for Windows/Linux the agent needs to be performed from
the command line running the
Windows installer package or Linux self-
extracting and installable shell script
bundle.
NOTE
During the upgrade of the Log Analytics agent for Windows, it does not support configuring or reconfiguring a workspace
to report to. To configure the agent, you need to follow one of the supported methods listed under Adding or removing a
workspace.
NOTE
If you've used the command line or script previously to install or configure the agent, EnableAzureOperationalInsights
was replaced by AddCloudWorkspace and RemoveCloudWorkspace .
Linux agent
The following steps demonstrate how to reconfigure the Linux agent if you decide to register it with a different
workspace or to remove a workspace from its configuration.
1. To verify it is registered to a workspace, run the following command:
/opt/microsoft/omsagent/bin/omsadmin.sh -l
It is important that the status also shows the agent is running, otherwise the following steps to reconfigure
the agent will not complete successfully.
2. If it is already registered with a workspace, remove the registered workspace by running the following
command. Otherwise if it is not registered, proceed to the next step.
/opt/microsoft/omsagent/bin/omsadmin.sh -X
The agent service does not need to be restarted in order for the changes to take effect.
param($ProxyDomainName="https://proxy.contoso.com:30443", $cred=(Get-Credential))
if (!$proxyMethod)
{
Write-Output 'Health Service proxy API not present, will not update settings.'
return
}
$ProxyUserName = $cred.username
Linux agent
Perform the following steps if your Linux computers need to communicate through a proxy server or Log
Analytics gateway. The proxy configuration value has the following syntax
[protocol://][user:password@]proxyhost[:port] . The proxyhost property accepts a fully qualified domain name or
IP address of the proxy server.
1. Edit the file /etc/opt/microsoft/omsagent/proxy.conf by running the following commands and change the
values to your specific settings.
proxyconf="https://proxyuser:proxypassword@proxyserver01:30443"
sudo echo $proxyconf >>/etc/opt/microsoft/omsagent/proxy.conf
sudo chown omsagent:omiusers /etc/opt/microsoft/omsagent/proxy.conf
Uninstall agent
Use one of the following procedures to uninstall the Windows or Linux agent using the command line or setup
wizard.
Windows agent
Uninstall from Control Panel
1. Sign on to the computer with an account that has administrative rights.
2. In Control Panel, click Programs and Features.
3. In Programs and Features, click Microsoft Monitoring Agent, click Uninstall, and then click Yes.
NOTE
The Agent Setup Wizard can also be run by double-clicking MMASetup-<platform>.exe, which is available for download
from a workspace in the Azure portal.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
2. Ensure that the line beginning with httpsport= defines the port 1270. Such as: httpsport=1270
Next steps
Review Troubleshooting the Linux agent if you encounter issues while installing or managing the Linux
agent.
Review Troubleshooting the Windows agent if you encounter issues while installing or managing the
Windows agent.
Agent Health solution in Azure Monitor
12/13/2019 • 5 minutes to read • Edit Online
The Agent Health solution in Azure helps you understand, for all of the agents reporting directly to the Log
Analytics workspace in Azure Monitor or a System Center Operations Manager management group connected to
Azure Monitor, which are unresponsive and submitting operational data. You can also keep track of how many
agents are deployed, where they are distributed geographically, and perform other queries to maintain awareness
of the distribution of agents deployed in Azure, other cloud environments, or on-premises.
Prerequisites
Before you deploy this solution, confirm you have currently supported Windows agents reporting to the Log
Analytics workspace or reporting to an Operations Manager management group integrated with your workspace.
Solution components
This solution consists of the following resources that are added to your workspace and directly connected agents
or Operations Manager connected management group.
Management packs
If your System Center Operations Manager management group is connected to a Log Analytics workspace, the
following management packs are installed in Operations Manager. These management packs are also installed on
directly connected Windows computers after adding this solution. There is nothing to configure or manage with
these management packs.
Microsoft System Center Advisor HealthAssessment Direct Channel Intelligence Pack
(Microsoft.IntelligencePacks.HealthAssessmentDirect)
Microsoft System Center Advisor HealthAssessment Server Channel Intelligence Pack
(Microsoft.IntelligencePacks.HealthAssessmentViaServer).
For more information on how solution management packs are updated, see Connect Operations Manager to Log
Analytics.
Configuration
Add the Agent Health solution to your Log Analytics workspace using the process described in Add solutions.
There is no further configuration required.
Data collection
Supported agents
The following table describes the connected sources that are supported by this solution.
System Center Operations Manager Yes Heartbeat events are collected from
management group agents reporting to the management
group every 60 seconds and then
forwarded to Azure Monitor. A direct
connection from Operations Manager
agents to Azure Monitor is not
required. Heartbeat event data is
forwarded from the management
group to the Log Analytics workspace.
Click on the Agent Health tile to open the Agent Health dashboard. The dashboard includes the columns in the
following table. Each column lists the top ten events by count that match that column’s criteria for the specified
time range. You can run a log search that provides the entire list by selecting See all at the right bottom of each
column, or by clicking the column header.
COLUMN DESCRIPTION
Agent count over time A trend of your agent count over a period of seven days for
both Linux and Windows agents.
Count of unresponsive agents A list of agents that haven’t sent a heartbeat in the past 24
hours.
Distribution by OS Type A partition of how many Windows and Linux agents you have
in your environment.
Distribution by Agent Version A partition of the different agent versions installed in your
environment and a count of each one.
Distribution by Agent Category A partition of the different categories of agents that are
sending up heartbeat events: direct agents, OpsMgr agents,
or the OpsMgr Management Server.
Count of Gateways Installed The number of servers that have the Log Analytics gateway
installed, and a list of these servers.
PROPERTY DESCRIPTION
Type Heartbeat
Each agent reporting to an Operations Manager management server will send two heartbeats, and
SCAgentChannel property's value will include both Direct and SCManagementServer depending on what data
sources and monitoring solutions you have enabled in your subscription. If you recall, data from solutions are
either sent directly from an Operations Manager management server to Azure Monitor, or because of the volume
of data collected on the agent, are sent directly from the agent to Azure Monitor. For heartbeat events which have
the value SCManagementServer, the ComputerIP value is the IP address of the management server since the
data is actually uploaded by it. For heartbeats where SCAgentChannel is set to Direct, it is the public IP address
of the agent.
QUERY DESCRIPTION
Heartbeat | summarize LastCall = max(TimeGenerated) by Count of unresponsive agents in the last 24 hours
Computer | where LastCall < ago(24h)
Heartbeat | summarize LastCall = max(TimeGenerated) by Count of unresponsive agents in the last 15 minutes
Computer | where LastCall < ago(15m)
Heartbeat | where TimeGenerated > ago(24h) and Computer Computers online (in the last 24 hours)
in ((Heartbeat | where TimeGenerated > ago(24h) | distinct
Computer)) | summarize LastCall = max(TimeGenerated) by
Computer
Heartbeat | where TimeGenerated > ago(24h) and Computer Total Agents Offline in Last 30 minutes (for the last 24 hours)
!in ((Heartbeat | where TimeGenerated > ago(30m) | distinct
Computer)) | summarize LastCall = max(TimeGenerated) by
Computer
Heartbeat | summarize AggregatedValue = dcount(Computer) Get a trend of number of agents over time by OSType
by OSType
Next steps
Learn about Alerts in Azure Monitor for details on generating alerts from log queries.
Troubleshooting the Log Analytics VM extension in
Azure Monitor
12/13/2019 • 2 minutes to read • Edit Online
This article provides help troubleshooting errors you might experience with the Log Analytics VM extension for
Windows and Linux virtual machines running on Microsoft Azure, and suggests possible solutions to resolve them.
To verify the status of the extension, perform the following steps from the Azure portal.
1. Sign into the Azure portal.
2. In the Azure portal, click All services. In the list of resources, type virtual machines. As you begin typing,
the list filters based on your input. Select Virtual machines.
3. In your list of virtual machines, find and select it.
4. On the virtual machine, click Extensions.
5. From the list, check to see if the Log Analytics extension is enabled or not. For Linux, the agent is listed as
OMSAgentforLinux and for Windows, the agent is listed as MicrosoftMonitoringAgent.
If the Log Analytics agent for Linux VM extension is not installing or reporting, you can perform the following steps
to troubleshoot the issue.
1. If the extension status is Unknown check if the Azure VM agent is installed and working correctly by reviewing
the VM agent log file /var/log/waagent.log
If the log does not exist, the VM agent is not installed.
Install the Azure VM Agent on Linux VMs
2. For other unhealthy statuses, review the Log Analytics agent for Linux VM extension logs files in
/var/log/azure/Microsoft.EnterpriseCloud.Monitoring.OmsAgentForLinux/*/extension.log and
/var/log/azure/Microsoft.EnterpriseCloud.Monitoring.OmsAgentForLinux/*/CommandExecution.log
3. If the extension status is healthy, but data is not being uploaded review the Log Analytics agent for Linux log
files in /var/opt/microsoft/omsagent/log/omsagent.log
For more information, see troubleshooting Linux extensions.
Next steps
For additional troubleshooting guidance related to the Log Analytics agent for Linux hosted on computers outside
of Azure, see Troubleshoot Azure Log Analytics Linux Agent.
How to troubleshoot issues with the Log Analytics
agent for Windows
12/13/2019 • 8 minutes to read • Edit Online
This article provides help troubleshooting errors you might experience with the Log Analytics agent for Windows
in Azure Monitor and suggests possible solutions to resolve them.
If none of these steps work for you, the following support channels are also available:
Customers with Premier support benefits can open a support request with Premier.
Customers with Azure support agreements can open a support request in the Azure portal.
Visit the Log Analytics Feedback page to review submitted ideas and bugs https://aka.ms/opinsightsfeedback
or file a new one.
Connectivity issues
If the agent is communicating through a proxy server or firewall, there may be restrictions in place preventing
communication from the source computer and the Azure Monitor service. In case communication is blocked,
because of misconfiguration, registration with a workspace might fail while attempting to install the agent or
configure the agent post-setup to report to an additional workspace. Agent communication may fail after
successful registration. This section describes the methods to troubleshoot this type of issue with the Windows
agent.
Double check that the firewall or proxy is configured to allow the following ports and URLs described in the
following table. Also confirm HTTP inspection is not enabled for web traffic, as it can prevent a secure TLS channel
between the agent and Azure Monitor.
For firewall information required for Azure Government, see Azure Government management. If you plan to use
the Azure Automation Hybrid Runbook Worker to connect to and register with the Automation service to use
runbooks or management solutions in your environment, it must have access to the port number and the URLs
described in Configure your network for the Hybrid Runbook Worker.
There are several ways you can verify if the agent is successfully communicating with Azure Monitor.
Enable the Azure Log Analytics Agent Health assessment in the workspace. From the Agent Health
dashboard, view the Count of unresponsive agents column to quickly see if the agent is listed.
Run the following query to confirm the agent is sending a heartbeat to the workspace it is configured to
report to. Replace <ComputerName> with the actual name of the machine.
Heartbeat
| where Computer like "<ComputerName>"
| summarize arg_max(TimeGenerated, * ) by Computer
If the computer is successfully communicating with the service, the query should return a result. If the
query did not return a result, first verify the agent is configured to report to the correct workspace. If it is
configured correctly, proceed to step 3 and search the Windows Event Log to identify if the agent is logging
what issue might be preventing it from communicating with Azure Monitor.
Another method to identify a connectivity issue is by running the TestCloudConnectivity tool. The tool is
installed by default with the agent in the folder %SystemRoot%\Program Files\Microsoft Monitoring
Agent\Agent. From an elevated command prompt, navigate to the folder and run the tool. The tool returns
the results and highlights where the test failed (for example, if it was related to a particular port/URL that
was blocked).
Filter the Operations Manager event log by Event sources - Health Service Modules, HealthService, and
Service Connector and filter by Event Level Warning and Error to confirm if it has written events from the
following table. If they are, review the resolution steps included for each possible event.
2133 & 2129 Health Service Connection to the service This error can occur when
from the agent failed the agent cannot
communicate directly or
through a firewall/proxy
server to the Azure
Monitor service. Verify
agent proxy settings or
that the network
firewall/proxy allows TCP
traffic from the computer
to the service.
EVENT ID SOURCE DESCRIPTION RESOLUTION
2138 Health Service Modules Proxy requires Configure the agent proxy
authentication settings and specify the
username/password
required to authenticate
with the proxy server.
4000 Service Connector DNS name resolution The machine could not
failed resolve the Internet
address used when
sending data to the
service. This might be DNS
resolver settings on your
machine, incorrect proxy
settings, or maybe a
temporary DNS issue with
your provider. If it happens
periodically, it could be
caused by a transient
network-related issue.
EVENT ID SOURCE DESCRIPTION RESOLUTION
4001 Service Connector Connection to the service This error can occur when
failed. the agent cannot
communicate directly or
through a firewall/proxy
server to the Azure
Monitor service. Verify
agent proxy settings or
that the network
firewall/proxy allows TCP
traffic from the computer
to the service.
4002 Service Connector The service returned HTTP This error is written during
status code 403 in the agent’s initial
response to a query. registration phase and
Check with the service you’ll see a URL similar to
administrator for the the following:
health of the service. The https://<workspaceID>.o
query will be retried later. ms.opinsights.azure.com/
AgentService.svc/AgentTo
pologyRequest. An error
code 403 means forbidden
and can be caused by a
mistyped Workspace ID or
key, or the data and time
is incorrect on the
computer. If the time is +/-
15 minutes from current
time, then onboarding
fails. To correct this,
update the date and/or
timezone of your Windows
computer.
Heartbeat
| where Computer like "<ComputerName>"
| summarize arg_max(TimeGenerated, * ) by Computer
If the query returns results, then you need to determine if a particular data type is not collected and forwarded to
the service. This could be caused by the agent not receiving updated configuration from the service, or some other
symptom preventing the agent from operating normally. Perform the following steps to further troubleshoot.
1. Open an elevated command prompt on the computer and restart the agent service by typing
net stop healthservice && net start healthservice .
2. Open the Operations Manager event log and search for event IDs 7023, 7024, 7025, 7028 and 1210 from
Event source HealthService. These events indicate the agent is successfully receiving configuration from
Azure Monitor and they are actively monitoring the computer. The event description for event ID 1210 will
also specify on the last line all of the solutions and Insights that are included in the scope of monitoring on
the agent.
3. If after several minutes you do not see the expected data in the query results or visualization, depending on
if you are viewing the data from a solution or Insight, from the Operations Manager event log, search for
Event sources HealthService and Health Service Modules and filter by Event Level Warning and Error to
confirm if it has written events from the following table.
8000 HealthService This event will specify if a Event ID 2136 from source
workflow related to HealthService is written
performance, event, or together with this event
other data type collected is and can indicate the agent
unable to forward to the is unable to communicate
service for ingestion to the with the service, possibly
workspace. due to misconfiguration of
the proxy and
authentication settings,
network outage, or the
network firewall/proxy
does not allow TCP traffic
from the computer to the
service.
10102 and 10103 Health Service Modules Workflow could not This can occur if the
resolve data source. specified performance
counter or instance does
not exist on the computer
or is incorrectly defined in
the workspace data
settings. If this is a user-
specified performance
counter, verify the
information specified is
following the correct
format and exists on the
target computers.
EVENT ID SOURCE DESCRIPTION RESOLUTION
26002 Health Service Modules Workflow could not This can occur if the
resolve data source. specified Windows event
log does not exist on the
computer. This error can
be safely ignored if the
computer is not expected
to have this event log
registered, otherwise if this
is a user-specified event
log, verify the information
specified is correct.
How to troubleshoot issues with the Log Analytics
agent for Linux
12/13/2019 • 16 minutes to read • Edit Online
This article provides help troubleshooting errors you might experience with the Log Analytics agent for Linux in
Azure Monitor and suggests possible solutions to resolve them.
If none of these steps work for you, the following support channels are also available:
Customers with Premier support benefits can open a support request with Premier.
Customers with Azure support agreements can open a support request in the Azure portal.
Diagnose OMI Problems with the OMI troubleshooting guide.
File a GitHub Issue.
Visit the Log Analytics Feedback page to review submitted ideas and bugs https://aka.ms/opinsightsfeedback
or file a new one.
We recommend you to use our log collector tool to retrieve important logs for troubleshooting or before
submitting a GitHub issue. You can read more about the tool and how to run it here.
NOTE
Editing configuration files for performance counters and Syslog is overwritten if the collection is configured from the data
menu Log Analytics Advanced Settings in the Azure portal for your workspace. To disable configuration for all agents, disable
collection from Log Analytics Advanced Settings or for a single agent run the following:
sudo su omsagent -c 'python /opt/microsoft/omsconfig/Scripts/OMS_MetaConfigHelper.py --disable'
Installation error codes
ERROR CODE MEANING
5 403 HTTP error received from Azure Monitor. See the full
output of the omsadmin script for details.
30 Internal script error. File a GitHub Issue with details from the
output.
In the OMS output plugin, before the end of the configuration file, change the log_level property from info to
debug :
Debug logging allows you to see batched uploads to Azure Monitor separated by type, number of data items, and
time taken to send:
Example debug enabled log:
Success sending oms.nagios x 1 in 0.14s
Success sending oms.omi x 4 in 0.52s
Success sending oms.syslog.authpriv.info x 1 in 0.91s
Verbose output
Instead of using the OMS output plugin you can also output data items directly to stdout , which is visible in the
Log Analytics agent for Linux log file.
In the Log Analytics general agent configuration file at
/etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.conf , comment out the OMS output plugin by adding a
# in front of each line:
Below the output plugin, uncomment the following section by removing the # in front of each line:
<match **>
type stdout
</match>
Issue: You see a 500 and 404 error in the log file right after onboarding
This is a known issue that occurs on first upload of Linux data into a Log Analytics workspace. This does not affect
data being sent or service experience.
3. Callstack will be dumped in omiagent_trace file, If you notice many Curl and NSS function calls, follow
resolution steps below.
Resolution (step by step)
1. Upgrade the nss-pem package to v1.0.3-5.el7_6.1.
sudo yum upgrade nss-pem
2. If nss-pem is not available for upgrade (mostly happens on Centos), then downgrade curl to 7.29.0-46. If by
mistake you run "yum update", then curl will be upgraded to 7.29.0-51 and the issue will happen again.
sudo yum downgrade curl libcurl
3. Restart OMI:
sudo scxadmin -restart
Issue: You are not seeing any data in the Azure portal
Probable causes
Onboarding to Azure Monitor failed
Connection to Azure Monitor is blocked
Log Analytics agent for Linux data is backed up
Resolution
1. Check if onboarding Azure Monitor was successful by checking if the following file exists:
/etc/opt/microsoft/omsagent/<workspace id>/conf/omsadmin.conf
NOTE
This issue is fixed in agent version 1.1.0-28 and later.
Issue: You are receiving Errno address already in use in omsagent log
file
If you see
[error]: unexpected error error_class=Errno::EADDRINUSE error=#<Errno::EADDRINUSE: Address already in use -
bind(2) for "127.0.0.1" port 25224>
in omsagent.log.
Probable causes
This error indicates that the Linux Diagnostic extension (LAD ) is installed side by side with the Log Analytics Linux
VM extension, and it is using same port for syslog data collection as omsagent.
Resolution
1. As root, execute the following commands (note that 25224 is an example and it is possible that in your
environment you see a different port number used by LAD ):
You then need to edit the correct rsyslogd or syslog_ng config file and change the LAD -related
configuration to write to port 25229.
2. If the VM is running rsyslogd , the file to be modified is: /etc/rsyslog.d/95-omsagent.conf (if it exists, else
/etc/rsyslog ). If the VM is running syslog_ng , the file to be modified is: /etc/syslog-ng/syslog-ng.conf .
<source>
type tail
path /var/log/nagios/nagios.log
format none
tag oms.nagios
</source>
<filter oms.nagios>
type filter_nagios_log
</filter>
Issue: You are not seeing any Linux data
Probable causes
Onboarding to Azure Monitor failed
Connection to Azure Monitor is blocked
Virtual machine was rebooted
OMI package was manually upgraded to a newer version compared to what was installed by Log Analytics
agent for Linux package
DSC resource logs class not found error in omsconfig.log log file
Log Analytics agent for data is backed up
DSC logs Current configuration does not exist. Execute Start-DscConfiguration command with -Path
parameter to specify a configuration file and create a current configuration first. in omsconfig.log log file, but
no log message exists about PerformRequiredConfigurationChecks operations.
Resolution
1. Install all dependencies like auditd package.
2. Check if onboarding to Azure Monitor was successful by checking if the following file exists:
/etc/opt/microsoft/omsagent/<workspace id>/conf/omsadmin.conf . If it was not, reonboard using the
omsadmin.sh command line instructions.
3. If using a proxy, check proxy troubleshooting steps above.
4. In some Azure distribution systems, omid OMI server daemon does not start after the virtual machine is
rebooted. This will result in not seeing Audit, ChangeTracking, or UpdateManagement solution-related
data. The workaround is to manually start omi server by running
sudo /opt/omi/bin/service_control restart .
5. After OMI package is manually upgraded to a newer version, it has to be manually restarted for Log
Analytics agent to continue functioning. This step is required for some distros where OMI server does not
automatically start after it is upgraded. Run sudo /opt/omi/bin/service_control restart to restart OMI.
6. If you see DSC resource class not found error in omsconfig.log, run
sudo /opt/omi/bin/service_control restart .
7. In some cases, when the Log Analytics agent for Linux cannot talk to Azure Monitor, data on the agent is
backed up to the full buffer size: 50 MB. The agent should be restarted by running the following command
/opt/microsoft/omsagent/bin/service_control restart .
NOTE
This issue is fixed in Agent version 1.1.0-28 or later
If omsconfig.log log file does not indicate that PerformRequiredConfigurationChecks operations are running
periodically on the system, there might be a problem with the cron job/service. Make sure cron job exists
under /etc/cron.d/OMSConsistencyInvoker . If needed run the following commands to create the cron job:
mkdir -p /etc/cron.d/
echo "*/15 * * * * omsagent /opt/omi/bin/OMSConsistencyInvoker >/dev/null 2>&1" | sudo tee
/etc/cron.d/OMSConsistencyInvoker
Also, make sure the cron service is running. You can use service cron status with Debian, Ubuntu, SUSE,
or service crond status with RHEL, CentOS, Oracle Linux to check the status of this service. If the service
does not exist, you can install the binaries and start the service using the following:
Ubuntu/Debian
SUSE
RHEL/CeonOS
Oracle Linux
Issue: When configuring collection from the portal for Syslog or Linux
performance counters, the settings are not applied
Probable causes
The Log Analytics agent for Linux has not picked up the latest configuration
The changed settings in the portal were not applied
Resolution
Background: omsconfig is the Log Analytics agent for Linux configuration agent that looks for new portal-side
configuration every five minutes. This configuration is then applied to the Log Analytics agent for Linux
configuration files located at /etc/opt/microsoft/omsagent/conf/omsagent.conf.
In some cases, the Log Analytics agent for Linux configuration agent might not be able to communicate with
the portal configuration service resulting in latest configuration not being applied.
1. Check that the omsconfig agent is installed by running dpkg --list omsconfig or rpm -qi omsconfig
. If it is not installed, reinstall the latest version of the Log Analytics agent for Linux.
2. Check that the omsconfig agent can communicate with Azure Monitor by running the following
command sudo su omsagent -c 'python /opt/microsoft/omsconfig/Scripts/GetDscConfiguration.py' .
This command returns the configuration that agent receives from the service, including Syslog
settings, Linux performance counters, and custom logs. If this command fails, run the following
command
sudo su omsagent -c 'python
/opt/microsoft/omsconfig/Scripts/PerformRequiredConfigurationChecks.py'
. This command forces the omsconfig agent to talk to Azure Monitor and retrieve the latest
configuration.
Or
sudo sh ./onboard_agent.sh --purge
Wait several minutes and the provisioning state changes to Provisioning succeeded.
wget https://github.com/Microsoft/OMS-Agent-for-Linux/releases/download/OMSAgent_GA_v1.4.2-
124/omsagent-1.4.2-124.universal.x64.sh
The data that Azure Monitor collects from agents is defined by the data sources that you configure. The data from
agents is stored as log data with a set of records. Each data source creates records of a particular type with each
type having its own set of properties.
OPERATIONS
MANAGER
AGENT DATA
LOG OPERATION OPERATION SENT VIA
DATA ANALYTICS S MANAGER AZURE S MANAGER MANAGEME COLLECTION
SOURCE PLATFORM AGENT AGENT STORAGE REQUIRED? NT GROUP FREQUENCY
Performanc Windows • • as
e counters scheduled,
minimum of
10 seconds
Performanc Linux • as
e counters scheduled,
minimum of
10 seconds
OPERATIONS
MANAGER
AGENT DATA
LOG OPERATION OPERATION SENT VIA
DATA ANALYTICS S MANAGER AZURE S MANAGER MANAGEME COLLECTION
SOURCE PLATFORM AGENT AGENT STORAGE REQUIRED? NT GROUP FREQUENCY
1. In the Azure portal, select Log Analytics workspaces > your workspace > Advanced Settings.
2. Select Data.
3. Click on the data source you want to configure.
4. Follow the link to the documentation for each data source in the above table for details on their configuration.
Data collection
Data source configurations are delivered to agents that are directly connected to Azure Monitor within a few
minutes. The specified data is collected from the agent and delivered directly to Azure Monitor at intervals
specific to each data source. See the documentation for each data source for these specifics.
For System Center Operations Manager agents in a connected management group, data source configurations
are translated into management packs and delivered to the management group every 5 minutes by default. The
agent downloads the management pack like any other and collects the specified data. Depending on the data
source, the data will be either sent to a management server which forwards the data to the Azure Monitor, or the
agent will send the data to Azure Monitor without going through the management server. See Data collection
details for monitoring solutions in Azure for details. You can read about details of connecting Operations
Manager and Azure Monitor and modifying the frequency that configuration is delivered at Configure
Integration with System Center Operations Manager.
If the agent is unable to connect to Azure Monitor or Operations Manager, it will continue to collect data that it
will deliver when it establishes a connection. Data can be lost if the amount of data reaches the maximum cache
size for the client, or if the agent is not able to establish a connection within 24 hours.
Log records
All log data collected by Azure Monitor is stored in the workspace as records. Records collected by different data
sources will have their own set of properties and be identified by their Type property. See the documentation for
each data source and solution for details on each record type.
Next steps
Learn about monitoring solutions that add functionality to Azure Monitor and also collect data into the
workspace.
Learn about log queries to analyze the data collected from data sources and monitoring solutions.
Configure alerts to proactively notify you of critical data collected from data sources and monitoring solutions.
Windows event log data sources in Azure Monitor
10/25/2019 • 3 minutes to read • Edit Online
Windows Event logs are one of the most common data sources for collecting data using Windows agents since
many applications write to the Windows event log. You can collect events from standard logs such as System and
Application in addition to specifying any custom logs created by applications you need to monitor.
NOTE
Critical events from the Windows event log will have a severity of "Error" in Azure Monitor Logs.
Data collection
Azure Monitor collects each event that matches a selected severity from a monitored event log as the event is
created. The agent records its place in each event log that it collects from. If the agent goes offline for a period of
time, then it collects events from where it last left off, even if those events were created while the agent was
offline. There is a potential for these events to not be collected if the event log wraps with uncollected events
being overwritten while the agent is offline.
NOTE
Azure Monitor does not collect audit events created by SQL Server from source MSSQLSERVER with event ID 18453 that
contains keywords - Classic or Audit Success and keyword 0xa0000000000000.
PROPERTY DESCRIPTION
Computer Name of the computer that the event was collected from.
EventLog Name of the event log that the event was collected from.
QUERY DESCRIPTION
Event | where EventLevelName == "error" All Windows events with severity of error.
Event | where EventLevelName == "error" | summarize Count of Windows error events by source.
count() by Source
Next steps
Configure Log Analytics to collect other data sources for analysis.
Learn about log queries to analyze the data collected from data sources and solutions.
Configure collection of performance counters from your Windows agents.
Collecting custom JSON data sources with the Log
Analytics agent for Linux in Azure Monitor
12/13/2019 • 2 minutes to read • Edit Online
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
Custom JSON data sources can be collected into Azure Monitor using the Log Analytics agent for Linux. These
custom data sources can be simple scripts returning JSON such as curl or one of FluentD's 300+ plugins. This
article describes the configuration required for this data collection.
NOTE
Log Analytics agent for Linux v1.1.0-217+ is required for Custom JSON Data
Configuration
Configure input plugin
To collect JSON data in Azure Monitor, add oms.api. to the start of a FluentD tag in an input plugin.
For example, following is a separate configuration file in
exec-json.conf
/etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.d/ . This uses the FluentD plugin exec to run a curl
command every 30 seconds. The output from this command is collected by the JSON output plugin.
<source>
type exec
command 'curl localhost/json.output'
format json
tag oms.api.httpresponse
run_interval 30s
</source>
<match oms.api.httpresponse>
type out_oms_api
log_level info
buffer_chunk_limit 5m
buffer_type file
buffer_path /var/opt/microsoft/omsagent/<workspace id>/state/out_oms_api_httpresponse*.buffer
buffer_queue_limit 10
flush_interval 20s
retry_limit 10
retry_wait 30s
</match>
<match oms.api.**>
type out_oms_api
log_level info
buffer_chunk_limit 5m
buffer_type file
buffer_path /var/opt/microsoft/omsagent/<workspace id>/state/out_oms_api*.buffer
buffer_queue_limit 10
flush_interval 20s
retry_limit 10
retry_wait 30s
</match>
Output
The data will be collected in Azure Monitor with a record type of <FLUENTD_TAG>_CL .
For example, the custom tag tag oms.api.tomcat in Azure Monitor with a record type of tomcat_CL . You could
retrieve all records of this type with the following log query.
Type=tomcat_CL
Nested JSON data sources are supported, but are indexed based off of parent field. For example, the following
JSON data is returned from a log query as tag_s : "[{ "a":"1", "b":"2" }] .
{
"tag": [{
"a":"1",
"b":"2"
}]
}
Next steps
Learn about log queries to analyze the data collected from data sources and solutions.
Collect data from CollectD on Linux agents in Azure
Monitor
12/13/2019 • 2 minutes to read • Edit Online
CollectD is an open source Linux daemon that periodically collects performance metrics from applications and
system level information. Example applications include the Java Virtual Machine (JVM ), MySQL Server, and Nginx.
This article provides information on collecting performance data from CollectD in Azure Monitor.
A full list of available plugins can be found at Table of Plugins.
The following CollectD configuration is included in the Log Analytics agent for Linux to route CollectD data to the
Log Analytics agent for Linux.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
LoadPlugin write_http
<Plugin write_http>
<Node "oms">
URL "127.0.0.1:26000/oms.collectd"
Format "JSON"
StoreRates true
</Node>
</Plugin>
Additionally, if using an versions of collectD before 5.5 use the following configuration instead.
LoadPlugin write_http
<Plugin write_http>
<URL "127.0.0.1:26000/oms.collectd">
Format "JSON"
StoreRates true
</URL>
</Plugin>
The CollectD configuration uses the default write_http plugin to send performance metric data over port 26000 to
Log Analytics agent for Linux.
NOTE
This port can be configured to a custom-defined port if needed.
The Log Analytics agent for Linux also listens on port 26000 for CollectD metrics and then converts them to Azure
Monitor schema metrics. The following is the Log Analytics agent for Linux configuration collectd.conf .
<source>
type http
port 26000
bind 127.0.0.1
</source>
<filter oms.collectd>
type filter_collectd
</filter>
NOTE
CollectD by default is set to read values at a 10-second interval. As this directly affects the volume of data sent to Azure
Monitor Logs, you might need to tune this interval within the CollectD configuration to strike a good balance between the
monitoring requirements and associated costs and usage for Azure Monitor Logs.
Versions supported
Azure Monitor currently supports CollectD version 4.8 and above.
Log Analytics agent for Linux v1.1.0-217 or above is required for CollectD metric collection.
Configuration
The following are basic steps to configure collection of CollectD data in Azure Monitor.
1. Configure CollectD to send data to the Log Analytics agent for Linux using the write_http plugin.
2. Configure the Log Analytics agent for Linux to listen for the CollectD data on the appropriate port.
3. Restart CollectD and Log Analytics agent for Linux.
Configure CollectD to forward data
1. To route CollectD data to the Log Analytics agent for Linux, oms.conf needs to be added to CollectD's
configuration directory. The destination of this file depends on the Linux distro of your machine.
If your CollectD config directory is located in /etc/collectd.d/:
sudo cp /etc/opt/microsoft/omsagent/sysconf/omsagent.d/collectd.conf
/etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.d/
sudo chown omsagent:omiusers /etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.d/collectd.conf
3. Restart CollectD and Log Analytics agent for Linux with the following commands.
sudo service collectd restart sudo /opt/microsoft/omsagent/bin/service_control restart
host Computer
plugin None
type ObjectName
type_instance CounterName
If type_instance is null then CounterName=blank
dsnames[] CounterName
dstypes None
values[] CounterValue
Next steps
Learn about log queries to analyze the data collected from data sources and solutions.
Use Custom Fields to parse data from syslog records into individual fields.
Syslog data sources in Azure Monitor
12/13/2019 • 6 minutes to read • Edit Online
Syslog is an event logging protocol that is common to Linux. Applications will send messages that may be stored
on the local machine or delivered to a Syslog collector. When the Log Analytics agent for Linux is installed, it
configures the local Syslog daemon to forward messages to the agent. The agent then sends the message to
Azure Monitor where a corresponding record is created.
NOTE
Azure Monitor supports collection of messages sent by rsyslog or syslog-ng, where rsyslog is the default daemon. The
default syslog daemon on version 5 of Red Hat Enterprise Linux, CentOS, and Oracle Linux version (sysklog) is not
supported for syslog event collection. To collect syslog data from this version of these distributions, the rsyslog daemon
should be installed and configured to replace sysklog.
Configuring Syslog
The Log Analytics agent for Linux will only collect events with the facilities and severities that are specified in its
configuration. You can configure Syslog through the Azure portal or by managing configuration files on your
Linux agents.
Configure Syslog in the Azure portal
Configure Syslog from the Data menu in Advanced Settings. This configuration is delivered to the configuration
file on each Linux agent.
You can add a new facility by first selecting the option Apply below configuration to my machines and then
typing in its name and clicking +. For each facility, only messages with the selected severities will be collected.
Check the severities for the particular facility that you want to collect. You cannot provide any additional criteria to
filter messages.
By default, all configuration changes are automatically pushed to all agents. If you want to configure Syslog
manually on each Linux agent, then uncheck the box Apply below configuration to my machines.
Configure Syslog on Linux agent
When the Log Analytics agent is installed on a Linux client, it installs a default syslog configuration file that
defines the facility and severity of the messages that are collected. You can modify this file to change the
configuration. The configuration file is different depending on the Syslog daemon that the client has installed.
NOTE
If you edit the syslog configuration, you must restart the syslog daemon for the changes to take effect.
rsyslog
The configuration file for rsyslog is located at /etc/rsyslog.d/95-omsagent.conf. Its default contents are shown
below. This collects syslog messages sent from the local agent for all facilities with a level of warning or higher.
kern.warning @127.0.0.1:25224
user.warning @127.0.0.1:25224
daemon.warning @127.0.0.1:25224
auth.warning @127.0.0.1:25224
syslog.warning @127.0.0.1:25224
uucp.warning @127.0.0.1:25224
authpriv.warning @127.0.0.1:25224
ftp.warning @127.0.0.1:25224
cron.warning @127.0.0.1:25224
local0.warning @127.0.0.1:25224
local1.warning @127.0.0.1:25224
local2.warning @127.0.0.1:25224
local3.warning @127.0.0.1:25224
local4.warning @127.0.0.1:25224
local5.warning @127.0.0.1:25224
local6.warning @127.0.0.1:25224
local7.warning @127.0.0.1:25224
You can remove a facility by removing its section of the configuration file. You can limit the severities that are
collected for a particular facility by modifying that facility's entry. For example, to limit the user facility to
messages with a severity of error or higher you would modify that line of the configuration file to the following:
user.error @127.0.0.1:25224
syslog-ng
The configuration file for syslog-ng is location at /etc/syslog-ng/syslog-ng.conf. Its default contents are shown
below. This collects syslog messages sent from the local agent for all facilities and all severities.
#
# Warnings (except iptables) in one file:
#
destination warn { file("/var/log/warn" fsync(yes)); };
log { source(src); filter(f_warn); destination(warn); };
#OMS_Destination
destination d_oms { udp("127.0.0.1" port(25224)); };
#OMS_facility = auth
filter f_auth_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(auth); };
log { source(src); filter(f_auth_oms); destination(d_oms); };
#OMS_facility = authpriv
filter f_authpriv_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(authpriv); };
log { source(src); filter(f_authpriv_oms); destination(d_oms); };
#OMS_facility = cron
filter f_cron_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(cron); };
log { source(src); filter(f_cron_oms); destination(d_oms); };
#OMS_facility = daemon
filter f_daemon_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(daemon); };
log { source(src); filter(f_daemon_oms); destination(d_oms); };
#OMS_facility = kern
filter f_kern_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(kern); };
log { source(src); filter(f_kern_oms); destination(d_oms); };
#OMS_facility = local0
filter f_local0_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(local0); };
log { source(src); filter(f_local0_oms); destination(d_oms); };
#OMS_facility = local1
filter f_local1_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(local1); };
log { source(src); filter(f_local1_oms); destination(d_oms); };
#OMS_facility = mail
filter f_mail_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(mail); };
log { source(src); filter(f_mail_oms); destination(d_oms); };
#OMS_facility = syslog
filter f_syslog_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(syslog); };
log { source(src); filter(f_syslog_oms); destination(d_oms); };
#OMS_facility = user
filter f_user_oms { level(alert,crit,debug,emerg,err,info,notice,warning) and facility(user); };
log { source(src); filter(f_user_oms); destination(d_oms); };
You can remove a facility by removing its section of the configuration file. You can limit the severities that are
collected for a particular facility by removing them from its list. For example, to limit the user facility to just alert
and critical messages, you would modify that section of the configuration file to the following:
#OMS_facility = user
filter f_user_oms { level(alert,crit) and facility(user); };
log { source(src); filter(f_user_oms); destination(d_oms); };
You can change the port number by creating two configuration files: a FluentD config file and a rsyslog-or-syslog-
ng file depending on the Syslog daemon you have installed.
The FluentD config file should be a new file located in: /etc/opt/microsoft/omsagent/conf/omsagent.d and
replace the value in the port entry with your custom port number.
<source>
type syslog
port %SYSLOG_PORT%
bind 127.0.0.1
protocol_type udp
tag oms.syslog
</source>
<filter oms.syslog.**>
type filter_syslog
</filter>
For rsyslog, you should create a new configuration file located in: /etc/rsyslog.d/ and replace the value
%SYSLOG_PORT% with your custom port number.
NOTE
If you modify this value in the configuration file 95-omsagent.conf , it will be overwritten when the agent applies a
default configuration.
The syslog-ng config should be modified by copying the example configuration shown below and adding
the custom modified settings to the end of the syslog-ng.conf configuration file located in /etc/syslog-ng/ .
Do not use the default label %WORKSPACE_ID%_oms or %WORKSPACE_ID_OMS, define a custom
label to help distinguish your changes.
NOTE
If you modify the default values in the configuration file, they will be overwritten when the agent applies a default
configuration.
filter f_custom_filter { level(warning) and facility(auth; };
destination d_custom_dest { udp("127.0.0.1" port(%SYSLOG_PORT%)); };
log { source(s_src); filter(f_custom_filter); destination(d_custom_dest); };
After completing the changes, the Syslog and the Log Analytics agent service needs to be restarted to ensure the
configuration changes take effect.
PROPERTY DESCRIPTION
Facility Defines the part of the system that generated the message.
QUERY DESCRIPTION
Syslog | where SeverityLevel == "error" All Syslog records with severity of error.
Next steps
Learn about log queries to analyze the data collected from data sources and solutions.
Use Custom Fields to parse data from syslog records into individual fields.
Configure Linux agents to collect other types of data.
Windows and Linux performance data sources in
Azure Monitor
12/13/2019 • 8 minutes to read • Edit Online
Performance counters in Windows and Linux provide insight into the performance of hardware components,
operating systems, and applications. Azure Monitor can collect performance counters at frequent intervals for
Near Real Time (NRT) analysis in addition to aggregating performance data for longer term analysis and
reporting.
* All instances
<source>
type oms_omi
object_name "Processor"
instance_regex ".*"
counter_name_regex ".*"
interval 30s
</source>
PARAMETERS DESCRIPTION
The following table lists the objects and counters that you can specify in the configuration file. There are
additional counters available for certain applications as described in Collect performance counters for Linux
applications in Azure Monitor.
Memory Pages/sec
System Processes
System Uptime
System Users
<source>
type oms_omi
object_name "Logical Disk"
instance_regex ".*
counter_name_regex ".*"
interval 5m
</source>
<source>
type oms_omi
object_name "Processor"
instance_regex ".*
counter_name_regex ".*"
interval 30s
</source>
<source>
type oms_omi
object_name "Memory"
instance_regex ".*"
counter_name_regex ".*"
interval 30s
</source>
Data collection
Azure Monitor collects all specified performance counters at their specified sample interval on all agents that
have that counter installed. The data is not aggregated, and the raw data is available in all log query views for
the duration specified by your subscription.
PROPERTY DESCRIPTION
Sizing estimates
A rough estimate for collection of a particular counter at 10-second intervals is about 1 MB per day per
instance. You can estimate the storage requirements of a particular counter with the following formula.
QUERY DESCRIPTION
Perf | where Computer == "MyComputer" All Performance data from a particular computer
Perf | where CounterName == "Current Disk Queue Length" All Performance data for a particular counter
Perf | where ObjectName == "Processor" and CounterName Average CPU Utilization across all computers
== "% Processor Time" and InstanceName == "_Total" |
summarize AVGCPU = avg(CounterValue) by Computer
Perf | where CounterName == "% Processor Time" | Maximum CPU Utilization across all computers
summarize AggregatedValue = max(CounterValue) by
Computer
Perf | where ObjectName == "LogicalDisk" and Average Current Disk Queue length across all the instances
CounterName == "Current Disk Queue Length" and of a given computer
Computer == "MyComputerName" | summarize
AggregatedValue = avg(CounterValue) by InstanceName
Perf | where CounterName == "Disk Transfers/sec" | 95th Percentile of Disk Transfers/Sec across all computers
summarize AggregatedValue = percentile(CounterValue, 95)
by Computer
Perf | where CounterName == "% Processor Time" and Hourly average of CPU usage across all computers
InstanceName == "_Total" | summarize AggregatedValue =
avg(CounterValue) by bin(TimeGenerated, 1h), Computer
QUERY DESCRIPTION
Perf | where Computer == "MyComputer" and Hourly 70 percentile of every % percent counter for a
CounterName startswith_cs "%" and InstanceName == particular computer
"_Total" | summarize AggregatedValue =
percentile(CounterValue, 70) by bin(TimeGenerated, 1h),
CounterName
Perf | where CounterName == "% Processor Time" and Hourly average, minimum, maximum, and 75-percentile CPU
InstanceName == "_Total" and Computer == usage for a specific computer
"MyComputer" | summarize ["min(CounterValue)"] =
min(CounterValue), ["avg(CounterValue)"] =
avg(CounterValue), ["percentile75(CounterValue)"] =
percentile(CounterValue, 75), ["max(CounterValue)"] =
max(CounterValue) by bin(TimeGenerated, 1h), Computer
Perf | where ObjectName == "MSSQL$INST2:Databases" All Performance data from the Database performance object
and InstanceName == "master" for the master database from the named SQL Server
instance INST2.
Next steps
Collect performance counters from Linux applications including MySQL and Apache HTTP Server.
Learn about log queries to analyze the data collected from data sources and solutions.
Export collected data to Power BI for additional visualizations and analysis.
Collect performance counters for Linux applications
in Azure Monitor
12/13/2019 • 6 minutes to read • Edit Online
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
This article provides details for configuring the Log Analytics agent for Linux to collect performance counters for
specific applications into Azure Monitor. The applications included in this article are:
MySQL
Apache HTTP Server
MySQL
If MySQL Server or MariaDB Server is detected on the computer when the Log Analytics agent is installed, a
performance monitoring provider for MySQL Server will be automatically installed. This provider connects to the
local MySQL/MariaDB server to expose performance statistics. MySQL user credentials must be configured so
that the provider can access the MySQL Server.
Configure MySQL credentials
The MySQL OMI provider requires a preconfigured MySQL user and installed MySQL client libraries in order to
query the performance and health information from the MySQL instance. These credentials are stored in an
authentication file that's stored on the Linux agent. The authentication file specifies what bind-address and port the
MySQL instance is listening on and what credentials to use to gather metrics.
During installation of the Log Analytics agent for Linux the MySQL OMI provider will scan MySQL my.cnf
configuration files (default locations) for bind-address and port and partially set the MySQL OMI authentication
file.
The MySQL authentication file is stored at /var/opt/microsoft/mysql-cimprov/auth/omsagent/mysql-auth .
Authentication file format
Following is the format for the MySQL OMI authentication file
The entries in the authentication file are described in the following table.
PROPERTY DESCRIPTION
PROPERTY DESCRIPTION
Base64 encoded Password Password of the MySQL monitoring user encoded in Base64.
AutoUpdate Specifies whether to rescan for changes in the my.cnf file and
overwrite the MySQL OMI Authentication file when the
MySQL OMI Provider is upgraded.
Default instance
The MySQL OMI authentication file can define a default instance and port number to make managing multiple
MySQL instances on one Linux host easier. The default instance is denoted by an instance with port 0. All
additional instances will inherit properties set from the default instance unless they specify different values. For
example, if MySQL instance listening on port ‘3308’ is added, the default instance’s bind-address, username, and
Base64 encoded password will be used to try and monitor the instance listening on 3308. If the instance on 3308
is bound to another address and uses the same MySQL username and password pair only the bind-address is
needed, and the other properties will be inherited.
The following table has example instance settings
DESCRIPTION FILE
Default instance and instance with port 3308. 0=127.0.0.1, myuser, cnBwdA==
3308=, ,
AutoUpdate=true
Default instance and instance with port 3308 and different 0=127.0.0.1, myuser, cnBwdA==
user name and password. 3308=127.0.1.1, myuser2,cGluaGVhZA==
AutoUpdate=true
/opt/microsoft/mysql-cimprov/bin/mycimprovauth
NOTE
The credentials file must be readable by the omsagent account. Running the mycimprovauth command as omsgent is
recommended.
The following table provides details on the syntax for using mycimprovauth.
OPERATION EXAMPLE DESCRIPTION
autoupdate false or true mycimprovauth autoupdate false Sets whether or not the authentication
file will be automatically updated on
restart or update.
default bind-address username mycimprovauth default 127.0.0.1 root Sets the default instance in the MySQL
password pwd OMI authentication file.
The password field should be entered in
plain text - the password in the MySQL
OMI authentication file will be Base 64
encoded.
delete default or port_num mycimprovauth 3308 Deletes the specified instance by either
default or by port number.
update port_num bind-address mycimprov update 3307 127.0.0.1 root Updates the specified instance or adds
username password pwd the instance if it does not exist.
The following example commands define a default user account for the MySQL server on localhost. The password
field should be entered in plain text - the password in the MySQL OMI authentication file will be Base 64 encoded
The MySQL user also requires SELECT access to the following default tables.
information_schema
mysql.
These privileges can be granted by running the following grant commands.
NOTE
To grant permissions to a MySQL monitoring user the granting user must have the ‘GRANT option’ privilege as well as the
privilege being granted.
sudo /opt/microsoft/apache-cimprov/bin/apache_config.sh -c
Next steps
Collect performance counters from Linux agents.
Learn about log queries to analyze the data collected from data sources and solutions.
Collect IIS logs in Azure Monitor
10/25/2019 • 2 minutes to read • Edit Online
Internet Information Services (IIS ) stores user activity in log files that can be collected by Azure Monitor and
stored as log data.
Data collection
Azure Monitor collects IIS log entries from each agent each time the log timestamp changes. The log is read every
5 minutes. If for any reason IIS doesn't update the timestamp before the rollover time when a new file is created,
entries will be collected following creation of the new file. The frequency of new file creation is controlled by the
Log File Rollover Schedule setting for the IIS site, which is once a day by default. If the setting is Hourly, Azure
Monitor collects the log each hour. If the setting is Daily, Azure Monitor collects the log every 24 hours.
PROPERTY DESCRIPTION
Computer Name of the computer that the event was collected from.
csReferer Site that the user followed a link from to the current site.
SourceSystem OpsMgr
QUERY DESCRIPTION
W3CIISLog | where scStatus==500 All IIS log records with a return status of 500.
W3CIISLog | summarize count() by cIP Count of IIS log entries by client IP address.
W3CIISLog | where csHost=="www.contoso.com" | summarize Count of IIS log entries by URL for the host
count() by csUriStem www.contoso.com.
W3CIISLog | summarize sum(csBytes) by Computer | take Total bytes received by each IIS computer.
500000
Next steps
Configure Azure Monitor to collect other data sources for analysis.
Learn about log queries to analyze the data collected from data sources and solutions.
Custom logs in Azure Monitor
12/13/2019 • 8 minutes to read • Edit Online
The Custom Logs data source in Azure Monitor allows you to collect events from text files on both Windows and
Linux computers. Many applications log information to text files instead of standard logging services such as
Windows Event log or Syslog. Once collected, you can either parse the data into individual fields in your queries
or extract the data during collection to individual fields.
NOTE
If there are duplicate entries in the log file, Azure Monitor will collect them. However, the query results will be inconsistent
where the filter results show more events than the result count. It will be important that you validate the log to determine if
the application that creates it is causing this behavior and address it if possible before creating the custom log collection
definition.
NOTE
A Log Analytics workspace supports the following limits:
Only 500 custom logs can be created.
A table only supports up to 500 columns.
The maximum number of characters for the column name is 500.
IMPORTANT
Custom log collection requires that the application writing the log file flushes the log content to the disk periodically. This is
because the custom log collection relies on filesystem change notifications for the log file being tracked.
DESCRIPTION PATH
All files in C:\Logs with a name starting with log and a .txt C:\Logs\log*.txt
extension on Windows agent
All files in /var/log/audit with a name starting with log and a /var/log/audit/log*.txt
.txt extension on Linux agent
1. Select Windows or Linux to specify which path format you are adding.
2. Type in the path and click the + button.
3. Repeat the process for any additional paths.
Step 4. Provide a name and description for the log
The name that you specify will be used for the log type as described above. It will always end with _CL to
distinguish it as a custom log.
1. Type in a name for the log. The _CL suffix is automatically provided.
2. Add an optional Description.
3. Click Next to save the custom log definition.
Step 5. Validate that the custom logs are being collected
It may take up to an hour for the initial data from a new custom log to appear in Azure Monitor. It will start
collecting entries from the logs found in the path you specified from the point that you defined the custom log. It
will not retain the entries that you uploaded during the custom log creation, but it will collect already existing
entries in the log files that it locates.
Once Azure Monitor starts collecting from the custom log, its records will be available with a log query. Use the
name that you gave the custom log as the Type in your query.
NOTE
If the RawData property is missing from the query, you may need to close and reopen your browser.
Data collection
Azure Monitor will collect new entries from each custom log approximately every 5 minutes. The agent will record
its place in each log file that it collects from. If the agent goes offline for a period of time, then Azure Monitor will
collect entries from where it last left off, even if those entries were created while the agent was offline.
The entire contents of the log entry are written to a single property called RawData. See Parse text data in Azure
Monitor for methods to parse each imported log entry into multiple properties.
PROPERTY DESCRIPTION
TimeGenerated Date and time that the record was collected by Azure
Monitor. If the log uses a time-based delimiter then this is the
time collected from the entry.
RawData Full text of the collected entry. You will most likely want to
parse this data into individual properties.
Next steps
See Parse text data in Azure Monitor for methods to parse each imported log entry into multiple properties.
Learn about log queries to analyze the data collected from data sources and solutions.
Create custom fields in a Log Analytics workspace in
Azure Monitor (Preview)
12/23/2019 • 7 minutes to read • Edit Online
NOTE
This article describes how to parse text data in a Log Analytics workspace as it's collected. We recommend parsing text data
in a query filter after it's collected following the guidance described in Parse text data in Azure Monitor. It provides several
advantages over using custom fields.
The Custom Fields feature of Azure Monitor allows you to extend existing records in your Log Analytics
workspace by adding your own searchable fields. Custom fields are automatically populated from data extracted
from other properties in the same record.
For example, the sample record below has useful data buried in the event description. Extracting this data into a
separate property makes it available for such actions as sorting and filtering.
NOTE
In the Preview, you are limited to 100 custom fields in your workspace. This limit will be expanded when this feature reaches
general availability.
NOTE
The custom field is populated as records matching the specified criteria are added to the Log Analytics workspace, so it will
only appear on records collected after the custom field is created. The custom field will not be added to records that are
already in the data store when it’s created.
Sample walkthrough
The following section walks through a complete example of creating a custom field. This example extracts the
service name in Windows events that indicate a service changing state. This relies on events created by Service
Control Manager during system startup on Windows computers. If you want to follow this example, you must be
collecting Information events for the System log.
We enter the following query to return all events from Service Control Manager that have an Event ID of 7036
which is the event that indicates a service starting or stopping.
The Field Extraction Wizard is opened, and the EventLog and EventID fields are selected in the Main
Example column. This indicates that the custom field will be defined for events from the System log with an event
ID of 7036. This is sufficient so we don’t need to select any other fields.
We highlight the name of the service in the RenderedDescription property and use Service to identify the
service name. The custom field will be called Service_CF. The field type in this case is a string, so we can leave
that unchanged.
We see that the service name is identified properly for some records but not for others. The Search Results show
that part of the name for the WMI Performance Adapter wasn’t selected. The Summary shows that one record
identified Modules Installer instead of Windows Modules Installer.
We start with the WMI Performance Adapter record. We click its edit icon and then Modify this highlight.
We increase the highlight to include the word WMI and then rerun the extract.
We can see that the entries for WMI Performance Adapter have been corrected, and Log Analytics also used
that information to correct the records for Windows Module Installer.
We can now run a query that verifies Service_CF is created but is not yet added to any records. That's because
the custom field doesn't work against existing records so we need to wait for new records to be collected.
After some time has passed so new events are collected, we can see that the Service_CF field is now being added
to records that match our criteria.
We can now use the custom field like any other record property. To illustrate this, we create a query that groups by
the new Service_CF field to inspect which services are the most active.
Next steps
Learn about log queries to build queries using custom fields for criteria.
Monitor custom log files that you parse using custom fields.
What is Azure Monitor for VMs (preview)?
12/13/2019 • 2 minutes to read • Edit Online
Azure Monitor for VMs monitors your Azure virtual machines (VM ) and virtual machine scale sets at scale. It
analyzes the performance and health of your Windows and Linux VMs, and monitors their processes and
dependencies on other resources and external processes.
It includes support for monitoring performance and application dependencies for VMs that are hosted on-
premises or in another cloud provider. The following key features deliver in-depth insight:
Pre-defined trending performance charts: Display core performance metrics from the guest VM
operating system.
Dependency map: Displays the interconnected components with the VM from various resource groups
and subscriptions.
NOTE
We recently announced changes we are making to the Health feature based on the feedback we have received from our
public preview customers. Given the number of changes we will be making, we are going to stop offering the Health
feature for new customers. Existing customers can continue to use the health feature. For more details, please refer to our
General Availability FAQ.
Integration with Azure Monitor logs delivers powerful aggregation and filtering, and it can analyze data trends
over time. Such comprehensive workload monitoring can't be achieved with Azure Monitor or Service Map
alone.
You can view this data in a single VM from the virtual machine directly, or you can use Azure Monitor to deliver
an aggregated view of your VMs where the view supports Azure resource-context or workspace-context
modes. For more information, see access modes overview.
Azure Monitor for VMs can deliver predictable performance and availability of vital applications. It identifies
performance bottlenecks and network issues. Azure Monitor for VMs can also help you understand whether an
issue is related to other dependencies.
Data usage
When you deploy Azure Monitor for VMs, the data that's collected by your VMs is ingested and stored in Azure
Monitor. Performance and dependency data collected are stored in a Log Analytics workspace. Based on the
pricing that's published on the Azure Monitor pricing page, Azure Monitor for VMs is billed for:
The data that's ingested and stored.
The alert rules that are created.
The notifications that are sent.
The log size varies by the string lengths of performance counters, and it can increase with the number of logical
disks and network adapters allocated to the VM. If you already have a workspace and are collecting these
counters, no duplicate charges are applied. If you're already using Service Map, the only change you’ll see is the
additional connection data that's sent to Azure Monitor.
Next steps
To understand the requirements and methods that help you monitor your virtual machines, review Deploy
Azure Monitor for VMs.
Known issues with Azure Monitor for VMs (preview)
12/13/2019 • 2 minutes to read • Edit Online
This article covers known issues with Azure Monitor for VMs, a solution in Azure that combines health, discovery
of application components, and performance monitoring of the Azure VM operating system.
Health
The following are known issues with the current release of the Health feature:
If an Azure VM is removed or deleted, it's displayed in the VM list view for sometime. Additionally, clicking the
state of a removed or deleted VM opens the Health Diagnostics view and then initiates a loading loop.
Selecting the name of the deleted VM opens a pane with a message stating that the VM has been deleted.
Configuration changes, such as updating a threshold, take up to 30 minutes even if the portal or Workload
Monitor API might update them immediately.
The Health Diagnostics experience updates faster than the other views. The information might be delayed when
you switch between them.
For Linux VMs, the title of the page listing the health criteria for a single VM view has the entire domain name
of the VM instead of the user-defined VM name.
After you disable monitoring for a VM using one of the supported methods and you try deploying it again, you
should deploy it in the same workspace. If you choose a different workspace and try to view the health state for
that VM, it might show inconsistent behavior.
After removing the solution components from your workspace, you may continue to see health state from your
Azure VMs; specifically performance and map data when you navigate to either view in the portal. Data will
eventually stop appearing from the Performance and Map view after sometime; however the Health view will
continue to show health status for your VMs. The Try now option will be available to re-onboard from
Performance and Map views only.
Next steps
To understand the requirements and methods for enabling monitoring of your virtual machines, review Enable
Azure Monitor for VMs.
Azure Monitor for VMs Generally Available (GA)
Frequently Asked Questions
12/13/2019 • 7 minutes to read • Edit Online
This General Availability FAQ covers changes that are happening in Azure Monitor for VMs as we prepare for our
GA release.
What is changing?
Currently when you complete the onboarding process for Azure Monitor for VMs, you enable the Service Map
solution in the workspace you selected to store your monitoring data, and then configure performance counters for
the data we collect from your VMs. We will be releasing a new solution, named VMInsights, that includes
additional capabilities for data collection along with a new location for storing this data in your Log Analytics
workspace.
Our current process of using performance counters in your Log Analytics workspace sends the data to the Perf
table. This new solution sends the data to a table named InsightsMetrics that is also used by Azure Monitor for
containers. This table schema allows us to store additional metrics and service data sets that are not compatible
with the Perf table format.
NOTE
If you have Alert Rules that reference these counters in the Perf table, you need to update them to reference new data
stored in the InsightsMetrics table. Refer to our documentation for example log queries that you can use that refer to this
table.
If you decide to keep the performance counters enabled, you will be billed for the data ingested and stored in the
Perf table based on [ Log Analytics pricing[(https://azure.microsoft.com/pricing/details/monitor/).
I use VM Health now with one environment and would like to deploy it
to a new one
If you are an existing customer that is using the Health feature and want to use it for a new roll-out, please contact
us at vminsights@microsoft.com to request instructions.
Next steps
To understand the requirements and methods that help you monitor your virtual machines, review Deploy Azure
Monitor for VMs.
Azure Monitor Frequently Asked Questions
1/24/2020 • 39 minutes to read • Edit Online
This Microsoft FAQ is a list of commonly asked questions about Azure Monitor.
General
What is Azure Monitor?
Azure Monitor is a service in Azure that provides performance and availability monitoring for applications and
services in Azure, other cloud environments, or on-premises. Azure Monitor collects data from multiple sources
into a common data platform where it can be analyzed for trends and anomalies. Rich features in Azure Monitor
assist you in quickly identifying and responding to critical situations that may affect your application.
What's the difference between Azure Monitor, Log Analytics, and Application Insights?
In September 2018, Microsoft combined Azure Monitor, Log Analytics, and Application Insights into a single
service to provide powerful end-to-end monitoring of your applications and the components they rely on. Features
in Log Analytics and Application Insights have not changed, although some features have been rebranded to Azure
Monitor in order to better reflect their new scope. The log data engine and query language of Log Analytics is now
referred to as Azure Monitor Logs. See Azure Monitor terminology updates.
What does Azure Monitor cost?
Features of Azure Monitor that are automatically enabled such as collection of metrics and activity logs are
provided at no cost. There is a cost associated with other features such as log queries and alerting. See the Azure
Monitor pricing page for detailed pricing information.
How do I enable Azure Monitor?
Azure Monitor is enabled the moment that you create a new Azure subscription, and Activity log and platform
metrics are automatically collected. Create diagnostic settings to collect more detailed information about the
operation of your Azure resources, and add monitoring solutions and insights to provide additional analysis on
collected data for particular services.
How do I access Azure Monitor?
Access all Azure Monitor features and data from the Monitor menu in the Azure portal. The Monitoring section
of the menu for different Azure services provides access to the same tools with data filtered to a particular
resource. Azure Monitor data is also accessible for a variety of scenarios using CLI, PowerShell, and a REST API.
Is there an on-premises version of Azure Monitor?
No. Azure Monitor is a scalable cloud service that processes and stores large amounts of data, although Azure
Monitor can monitor resources that are on-premises and in other clouds.
Can Azure Monitor monitor on-premises resources?
Yes, in addition to collecting monitoring data from Azure resources, Azure Monitor can collect data from virtual
machines and applications in other clouds and on-premises. See Sources of monitoring data for Azure Monitor.
Does Azure Monitor integrate with System Center Operations Manager?
You can connect your existing System Center Operations Manager management group to Azure Monitor to collect
data from agents into Azure Monitor Logs. This allows you to use log queries and solution to analyze data collected
from agents. You can also configure existing System Center Operations Manager agents to send data directly to
Azure Monitor. See Connect Operations Manager to Azure Monitor.
What IP addresses does Azure Monitor use?
See IP addresses used by Application Insights and Log Analytics for a listing of the IP addresses and ports required
for agents and other external resources to access Azure Monitor.
Monitoring data
Where does Azure Monitor get its data?
Azure Monitor collects data from a variety of sources including logs and metrics from Azure platform and
resources, custom applications, and agents running on virtual machines. Other services such as Azure Security
Center and Network Watcher collect data into a Log Analytics workspace so it can be analyzed with Azure Monitor
data. You can also send custom data to Azure Monitor using the REST API for logs or metrics. See Sources of
monitoring data for Azure Monitor.
What data is collected by Azure Monitor?
Azure Monitor collects data from a variety of sources into logs or metrics. Each type of data has its own relative
advantages, and each supports a particular set of features in Azure Monitor. There is a single metrics database for
each Azure subscription, while you can create multiple Log Analytics workspaces to collect logs depending on your
requirements. See Azure Monitor data platform.
Is there a maximum amount of data that I can collect in Azure Monitor?
There is no limit to the amount of metric data you can collect, but this data is stored for a maximum of 93 days. See
Retention of Metrics. There is no limit on the amount of log data that you can collected, but it may be affected by
the pricing tier you choose for the Log Analytics workspace. See pricing details.
How do I access data collected by Azure Monitor?
Insights and solutions provide a custom experience for working with data stored in Azure Monitor. You can work
directly with log data using a log query written in Kusto Query Language (KQL ). In the Azure portal, you can write
and run queries and interactively analyze data using Log Analytics. Analyze metrics in the Azure portal with the
Metrics Explorer. See Analyze log data in Azure Monitor and Getting started with Azure Metrics Explorer.
Logs
What's the difference between Azure Monitor Logs and Azure Data Explorer?
Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. Azure Monitor
Logs is built on top of Azure Data Explorer and uses the same Kusto Query Language (KQL ) with some minor
differences. See Azure Monitor log query language differences.
How do I retrieve log data?
All data is retrieved from a Log Analytics workspace using a log query written using Kusto Query Language (KQL ).
You can write your own queries or use solutions and insights that include log queries for a particular application or
service. See Overview of log queries in Azure Monitor.
What is a Log Analytics workspace?
All log data collected by Azure Monitor is stored in a Log Analytics workspace. A workspace is essentially a
container where log data is collected from a variety of sources. You may have a single Log Analytics workspace for
all your monitoring data or may have requirements for multiple workspaces. See Designing your Azure Monitor
Logs deployment.
Can you move an existing Log Analytics workspace to another Azure subscription?
You can move a workspace between resource groups or subscriptions but not to a different region. See Move a Log
Analytics workspace to different subscription or resource group.
Why can't I see Query Explorer and Save buttons in Log Analytics?
Query Explorer, Save and New alert rule buttons are not available when the query scope is set to a specific
resource. To create alerts, save or load a query, Log Analytics must be scoped to a workspace. To open Log
Analytics in workspace context, select Logs from the Azure Monitor menu. The last used workspace is selected,
but you can select any other workspace. See Log query scope and time range in Azure Monitor Log Analytics
Why am I getting the error: "Register resource provider 'Microsoft.Insights' for this subscription to enable this
query" when opening Log Analytics from a VM?
Many resource providers are automatically registered, but you may need to manually register some resource
providers. The scope for registration is always the subscription. See Resource providers and types for more
information.
Why am I am getting no access error message when opening Log Analytics from a VM?
To view VM Logs, you need to be granted with read permission to the workspaces that stores the VM logs. In these
cases, your administrator must grant you with to permissions in Azure.
Alerts
What is an alert in Azure Monitor?
Alerts proactively notify you when important conditions are found in your monitoring data. They allow you to
identify and address issues before the users of your system notice them. There are multiple kinds of alerts:
Metric - Metric value exceeds a threshold.
Log query - Results of a log query match defined criteria.
Activity log - Activity log event matches defined criteria.
Web test - Results of availability test match defined criteria.
See Overview of alerts in Microsoft Azure.
What is an action group?
An action group is a collection of notifications and actions that can be triggered by an alert. Multiple alerts can use
a single action group allowing you to leverage common sets of notifications and actions. See Create and manage
action groups in the Azure portal.
What is an action rule?
An action rule allows you to modify the behavior of a set of alerts that match a certain criteria. This allows you to to
perform such requirements as disable alert actions during a maintenance window. You can also apply an action
group to a set of alerts rather than applying them directly to the alert rules. See Action rules.
Agents
Does Azure Monitor require an agent?
An agent is only required to collect data from the operating system and workloads in virtual machines. The virtual
machines can be located in Azure, another cloud environment, or on-premises. See Overview of the Azure Monitor
agents.
What's the difference between the Azure Monitor agents?
Azure Diagnostic extension is for Azure virtual machines and collects data to Azure Monitor Metrics, Azure
Storage, and Azure Event Hubs. The Log Analytics agent is for virtual machines in Azure, another cloud
environment, or on-premises and collects data to Azure Monitor Logs. The Dependency agent requires the Log
Analytics agent and collected process details and dependencies. See Overview of the Azure Monitor agents.
Does my agent traffic use my ExpressRoute connection?
Traffic to Azure Monitor uses the Microsoft peering ExpressRoute circuit. See ExpressRoute documentation for a
description of the different types of ExpressRoute traffic.
How can I confirm that the Log Analytics agent is able to communicate with Azure Monitor?
From Control Panel on the agent computer, select Security & Settings, Microsoft Monitoring Agent . Under
the Azure Log Analytics (OMS ) tab, a green check mark icon confirms that the agent is able to communicate with
Azure Monitor. A yellow warning icon means the agent is having issues. One common reason is the Microsoft
Monitoring Agent service has stopped. Use service control manager to restart the service.
How do I stop the Log Analytics agent from communicating with Azure Monitor?
For agents connected to Log Analytics directly, open the Control Panel and select Security & Settings, Microsoft
Monitoring Agent. Under the Azure Log Analytics (OMS ) tab, remove all workspaces listed. In System Center
Operations Manager, remove the computer from the Log Analytics managed computers list. Operations Manager
updates the configuration of the agent to no longer report to Log Analytics.
How much data is sent per agent?
The amount of data sent per agent depends on:
The solutions you have enabled
The number of logs and performance counters being collected
The volume of data in the logs
See Manage usage and costs with Azure Monitor Logs for details.
For computers that are able to run the WireData agent, use the following query to see how much data is being sent:
WireData
| where ProcessName == "C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe"
| where Direction == "Outbound"
| summarize sum(TotalBytes) by Computer
How much network bandwidth is used by the Microsoft Management Agent (MMA ) when sending data to Azure
Monitor?
Bandwidth is a function on the amount of data sent. Data is compressed as it is sent over the network.
How can I be notified when data collection from the Log Analytics agent stops?
Use the steps described in create a new log alert to be notified when data collection stops. Use the following
settings for the alert rule:
Define alert condition: Specify your Log Analytics workspace as the resource target.
Alert criteria
Signal Name: Custom log search
Search query:
Heartbeat | summarize LastCall = max(TimeGenerated) by Computer | where LastCall < ago(15m)
Alert logic: Based on number of results, Condition Greater than, Threshold value 0
Evaluated based on: Period (in minutes) 30, Frequency (in minutes) 10
Define alert details
Name: Data collection stopped
Severity: Warning
Specify an existing or new Action Group so that when the log alert matches criteria, you are notified if you have a
heartbeat missing for more than 15 minutes.
What are the firewall requirements for Azure Monitor agents?
See Network firewall requirementsfor details on firewall requirements.
Visualizations
Why can't I can’t see View Designer?
View Designer is only available for users assigned with Contributor permissions or higher in the Log Analytics
workspace.
Application Insights
Configuration problems
I'm having trouble setting up my:
.NET app
Monitoring an already-running app
Azure diagnostics
Java web app
I get no data from my server
Set firewall exceptions
Set up an ASP.NET server
Set up a Java server
Can I use Application Insights with ...?
Web apps on an IIS server in Azure VM or Azure virtual machine scale set
Web apps on an IIS server - on-premises or in a VM
Java web apps
Node.js apps
Web apps on Azure
Cloud Services on Azure
App servers running in Docker
Single-page web apps
SharePoint
Windows desktop app
Other platforms
Is it free?
Yes, for experimental use. In the basic pricing plan, your application can send a certain allowance of data each
month free of charge. The free allowance is large enough to cover development, and publishing an app for a small
number of users. You can set a cap to prevent more than a specified amount of data from being processed.
Larger volumes of telemetry are charged by the Gb. We provide some tips on how to limit your charges.
The Enterprise plan incurs a charge for each day that each web server node sends telemetry. It is suitable if you
want to use Continuous Export on a large scale.
Read the pricing plan.
How much does it cost?
Open the Usage and estimated costs page in an Application Insights resource. There's a chart of recent
usage. You can set a data volume cap, if you want.
Open the Azure Billing blade to see your bills across all resources.
What does Application Insights modify in my project?
The details depend on the type of project. For a web application:
Adds these files to your project:
ApplicationInsights.config
ai.js
Installs these NuGet packages:
Application Insights API - the core API
Application Insights API for Web Applications - used to send telemetry from the server
Application Insights API for JavaScript Applications - used to send telemetry from the client
The packages include these assemblies:
Microsoft.ApplicationInsights
Microsoft.ApplicationInsights.Platform
Inserts items into:
Web.config
packages.config
(New projects only - if you add Application Insights to an existing project, you have to do this manually.) Inserts
snippets into the client and server code to initialize them with the Application Insights resource ID. For example,
in an MVC app, code is inserted into the master page Views/Shared/_Layout.cshtml
How do I upgrade from older SDK versions?
See the release notes for the SDK appropriate to your type of application.
How can I change which Azure resource my project sends data to?
In Solution Explorer, right-click ApplicationInsights.config and choose Update Application Insights. You can
send the data to an existing or new resource in Azure. The update wizard changes the instrumentation key in
ApplicationInsights.config, which determines where the server SDK sends your data. Unless you deselect "Update
all," it will also change the key where it appears in your web pages.
What is Status Monitor?
A desktop app that you can use in your IIS web server to help configure Application Insights in web apps. It doesn't
collect telemetry: you can stop it when you are not configuring an app.
Learn more.
What telemetry is collected by Application Insights?
From server web apps:
HTTP requests
Dependencies. Calls to: SQL Databases; HTTP calls to external services; Azure Cosmos DB, table, blob storage,
and queue.
Exceptions and stack traces.
Performance Counters - If you use Status Monitor, Azure monitoring for App Services, Azure monitoring for
VM or virtual machine scale set, or the Application Insights collectd writer.
Custom events and metrics that you code.
Trace logs if you configure the appropriate collector.
From client web pages:
Page view counts
AJAX calls Requests made from a running script.
Page view load data
User and session counts
Authenticated user IDs
From other sources, if you configure them:
Azure diagnostics
Import to Analytics
Log Analytics
Logstash
Can I filter out or modify some telemetry?
Yes, in the server you can write:
Telemetry Processor to filter or add properties to selected telemetry items before they are sent from your app.
Telemetry Initializer to add properties to all items of telemetry.
Learn more for ASP.NET or Java.
How are city, country/region, and other geo location data calculated?
We look up the IP address (IPv4 or IPv6) of the web client using GeoLite2.
Browser telemetry: We collect the sender's IP address.
Server telemetry: The Application Insights module collects the client IP address. It is not collected if
X-Forwarded-For is set.
To learn more about how IP address and geolocation data is collected in Application Insights refer to this article.
You can configure the ClientIpHeaderTelemetryInitializer to take the IP address from a different header. In some
systems, for example, it is moved by a proxy, load balancer, or CDN to X-Originating-IP . Learn more.
You can use Power BI to display your request telemetry on a map.
How long is data retained in the portal? Is it secure?
Take a look at Data Retention and Privacy.
Could personal data be sent in the telemetry?
This is possible if your code sends such data. It can also happen if variables in stack traces include personal data.
Your development team should conduct risk assessments to ensure that personal data is properly handled. Learn
more about data retention and privacy.
All octets of the client web address are always set to 0 after the geo location attributes are looked up.
My Instrumentation Key is visible in my web page source.
This is common practice in monitoring solutions.
It can't be used to steal your data.
It could be used to skew your data or trigger alerts.
We have not heard that any customer has had such problems.
You could:
Use two separate Instrumentation Keys (separate Application Insights resources), for client and server data. Or
Write a proxy that runs in your server, and have the web client send data through that proxy.
How do I see POST data in Diagnostic search?
We don't log POST data automatically, but you can use a TrackTrace call: put the data in the message parameter.
This has a longer size limit than the limits on string properties, though you can't filter on it.
Should I use single or multiple Application Insights resources?
Use a single resource for all the components or roles in a single business system. Use separate resources for
development, test, and release versions, and for independent applications.
See the discussion here
Example - cloud service with worker and web roles
How do I dynamically change the instrumentation key?
Discussion here
Example - cloud service with worker and web roles
What are the User and Session counts?
The JavaScript SDK sets a user cookie on the web client, to identify returning users, and a session cookie to
group activities.
If there is no client-side script, you can set cookies at the server.
If one real user uses your site in different browsers, or using in-private/incognito browsing, or different
machines, then they will be counted more than once.
To identify a logged-in user across machines and browsers, add a call to setAuthenticatedUserContext().
Have I enabled everything in Application Insights?
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Server app perf: response times, ... Add Application Insights to your project Detect perf issues
or Install AI Status Monitor on server
(or write your own code to track
dependencies)
Dependency telemetry Install AI Status Monitor on server Diagnose issues with databases or other
external components
Get stack traces from exceptions Insert TrackException calls in your code Detect and diagnose exceptions
(but some are reported automatically)
Search log traces Add a logging adapter Diagnose exceptions, perf issues
Client usage basics: page views, JavaScript initializer in web pages Usage analytics
sessions, ...
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Client custom metrics Tracking calls in web pages Enhance user experience
Automation
Configuring Application Insights
You can write PowerShell scripts using Azure Resource Monitor to:
Create and update Application Insights resources.
Set the pricing plan.
Get the instrumentation key.
Add a metric alert.
Add an availability test.
You can't set up a Metric Explorer report or set up continuous export.
Querying the telemetry
Use the REST API to run Analytics queries.
How can I set an alert on an event?
Azure alerts are only on metrics. Create a custom metric that crosses a value threshold whenever your event
occurs. Then set an alert on the metric. You'll get a notification whenever the metric crosses the threshold in either
direction; you won't get a notification until the first crossing, no matter whether the initial value is high or low; there
is always a latency of a few minutes.
Are there data transfer charges between an Azure web app and Application Insights?
If your Azure web app is hosted in a data center where there is an Application Insights collection endpoint, there
is no charge.
If there is no collection endpoint in your host data center, then your app's telemetry will incur Azure outgoing
charges.
This doesn't depend on where your Application Insights resource is hosted. It just depends on the distribution of
our endpoints.
Can I send telemetry to the Application Insights portal?
We recommend you use our SDKs and use the SDK API. There are variants of the SDK for various platforms.
These SDKs handle buffering, compression, throttling, retries, and so on. However, the ingestion schema and
endpoint protocol are public.
Can I monitor an intranet web server?
Yes, but you will need to allow traffic to our services by either firewall exceptions or proxy redirects.
QuickPulse https://rt.services.visualstudio.com:443
ApplicationIdProvider https://dc.services.visualstudio.com:443
TelemetryChannel https://dc.services.visualstudio.com:443
<ApplicationInsights>
...
<TelemetryModules>
<Add
Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule,
Microsoft.AI.PerfCounterCollector">
<QuickPulseServiceEndpoint>https://rt.services.visualstudio.com/QuickPulseService.svc</QuickPulseServiceEndpoin
t>
</Add>
</TelemetryModules>
...
<TelemetryChannel>
<EndpointAddress>https://dc.services.visualstudio.com/v2/track</EndpointAddress>
</TelemetryChannel>
...
<ApplicationIdProvider
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplicationId
Provider, Microsoft.ApplicationInsights">
<ProfileQueryEndpoint>https://dc.services.visualstudio.com/api/profiles/{0}/appId</ProfileQueryEndpoint>
</ApplicationIdProvider>
...
</ApplicationInsights>
NOTE
ApplicationIdProvider is available starting in v2.6.0.
Proxy passthrough
Proxy passthrough can be achieved by configuring either a machine level or application level proxy. For more
information see dotnet's article on DefaultProxy.
Example Web.config:
<system.net>
<defaultProxy>
<proxy proxyaddress="http://xx.xx.xx.xx:yyyy" bypassonlocal="true"/>
</defaultProxy>
</system.net>
Option 2
Re-enable collection for these properties for every container log line.
If the first option is not convenient due to query changes involved, you can re-enable collecting these fields by
enabling the setting log_collection_settings.enrich_container_logs in the agent config map as described in the
data collection configuration settings.
NOTE
The second option is not recommended with large clusters that have more than 50 nodes because it generates API server
calls from every node in the cluster to perform this enrichment. This option also increases data size for every log line collected.
console.log(json.stringify({
"Hello": "This example has multiple lines:",
"Docker/Moby": "will not break this into multiple lines",
"and you will receive":"all of them in log analytics",
"as one": "log entry"
}));
This data will look like the following example in Azure Monitor for logs when you query for it:
LogEntry : ({“Hello": "This example has multiple lines:","Docker/Moby": "will not break this into multiple
lines", "and you will receive":"all of them in log analytics", "as one": "log entry"}
For a detailed look at the issue, review the following GitHub link.
How do I resolve Azure AD errors when I enable live logs?
You may see the following error: The reply url specified in the request does not match the reply urls
configured for the application: '<application ID>'. The solution to solve it can be found in the article How to
view container data in real time with Azure Monitor for containers.
Why can't I upgrade cluster after onboarding?
If after you enable Azure Monitor for containers for an AKS cluster, you delete the Log Analytics workspace the
cluster was sending its data to, when attempting to upgrade the cluster it will fail. To work around this, you will have
to disable monitoring and then re-enable it referencing a different valid workspace in your subscription. When you
try to perform the cluster upgrade again, it should process and complete successfully.
Which ports and domains do I need to open/whitelist for the agent?
See the Network firewall requirements for the proxy and firewall configuration information required for the
containerized agent with Azure, Azure US Government, and Azure China 21Vianet clouds.
NOTE
We configure performance counters for the workspace that affects all VMs that report to the workspace, whether or not you
have chosen to onboard them to Azure Monitor for VMs. For more details on how performance counters are configured for
the workspace, please refer to our documentation. For information about the counters configured for Azure Monitor for VMs,
please refer to our enable Azure Monitor for VMs article.
Next steps
If your question isn't answered here, you can refer to the following forums to additional questions and answers.
Log Analytics
Application Insights
For general feedback on Azure Monitor please visit the feedback forum.
Enable Azure Monitor for VMs (preview) overview
12/13/2019 • 8 minutes to read • Edit Online
This article provides an overview of the options available to set up Azure Monitor for VMs. Use Azure Monitor for
VMs to monitor health and performance. Discover application dependencies that run on Azure virtual machines
(VMs) and virtual machine scale sets, on-premises VMs, or VMs hosted in another cloud environment.
To set up Azure Monitor for VMs:
Enable a single Azure VM or virtual machine scale set by selecting Insights (preview) directly from the VM or
virtual machine scale set.
Enable two or more Azure VMs and virtual machine scale sets by using Azure Policy. This method ensures that
on existing and new VMs and scale sets, the required dependencies are installed and properly configured.
Noncompliant VMs and scale sets are reported, so you can decide whether to enable them and to remediate
them.
Enable two or more Azure VMs or virtual machine scale sets across a specified subscription or resource group
by using PowerShell.
Enable Azure Monitor for VMs to monitor VMs or physical computers hosted in your corporate network or
other cloud environment.
Prerequisites
Before you start, make sure that you understand the information in the following sections.
NOTE
The following information described in this section is also applicable to the Service Map solution.
Log Analytics
Azure Monitor for VMs supports a Log Analytics workspace in the following regions:
West Central US
West US
West US 2
South Central US
East US
East US2
Central US
North Central US
Canada Central
UK South
North Europe
West Europe
East Asia
Southeast Asia
Central India
Japan East
Australia East
Australia Southeast
NOTE
You can deploy Azure VMs from any region. These VMs aren't limited to the regions supported by the Log Analytics
workspace.
If you don't have a workspace, you can create one by using one of these resources:
The Azure CLI
PowerShell
The Azure portal
Azure Resource Manager
You can also create a workspace while you're enabling monitoring for a single Azure VM or virtual machine scale
set in the Azure portal.
To set up an at-scale scenario that uses Azure Policy, Azure PowerShell, or Azure Resource Manager templates, in
your Log Analytics workspace:
Install the ServiceMap and InfrastructureInsights solutions. You can complete this installation by using a
provided Azure Resource Manager template. Or on the Get Started tab, select Configure Workspace.
Configure the Log Analytics workspace to collect performance counters.
To configure your workspace for the at-scale scenario, use one of the following methods:
Use Azure PowerShell.
On the Azure Monitor for VMs Policy Coverage page, select Configure Workspace.
Supported operating systems
The following table lists the Windows and Linux operating systems that Azure Monitor for VMs supports. Later in
this section, you'll find a full list that details the major and minor Linux OS release and supported kernel versions.
Windows 10 1803 X X
Windows 8.1 X X
Windows 8 X X
OS VERSION PERFORMANCE MAPS
Windows 7 SP1 X X
CentOS Linux 7, 6 X X
Debian 9.4, 8 X1
1 The Performance feature of Azure Monitor for VMs is available only from Azure Monitor. It isn't available
directly from the left pane of the Azure VM.
NOTE
In the Linux operating system:
Only default and SMP Linux kernel releases are supported.
Nonstandard kernel releases, such as Physical Address Extension (PAE) and Xen, aren't supported for any Linux
distribution. For example, a system with the release string of 2.6.16.21-0.8-xen isn't supported.
Custom kernels, including recompilations of standard kernels, aren't supported.
CentOSPlus kernel is supported.
The Linux kernel must be patched for the Spectre vulnerability. Please consult your Linux distribution vendor for more
details.
7.6 3.10.0-957
7.5 3.10.0-862
7.4 3.10.0-693
6.10 2.6.32-754
6.9 2.6.32-696
CentOSPlus
6.10 2.6.32-754.3.5
2.6.32-696.30.1
OS VERSION KERNEL VERSION
6.9 2.6.32-696.30.1
2.6.32-696.18.7
Ubuntu Server
16.04.3 4.15.*
16.04 4.13.*
4.11.*
4.10.*
4.8.*
4.4.*
12 SP3 4.4.*
12 SP2 4.4.*
Debian
9 4.9
NOTE
The following information described in this section is also applicable to the Service Map solution.
In a hybrid environment, you can download and install the Dependency agent manually or using an automated
method.
The following table describes the connected sources that the Map feature supports in a hybrid environment.
CONNECTED SOURCE SUPPORTED DESCRIPTION
Windows agents Yes Along with the Log Analytics agent for
Windows, Windows agents need the
Dependency agent. For more
information, see supported operating
systems.
Linux agents Yes Along with the Log Analytics agent for
Linux, Linux agents need the
Dependency agent. For more
information, see supported operating
systems.
Single Azure VM or virtual machine Enable from the VM You can enable a single Azure VM by
scale set selecting Insights (preview) directly
from the VM or virtual machine scale
set.
Multiple Azure VMs or virtual machine Enable through Azure Policy You can enable multiple Azure VMs by
scale sets using Azure Policy and available policy
definitions.
DEPLOYMENT STATE METHOD DESCRIPTION
Multiple Azure VMs or virtual machine Enable through Azure PowerShell or You can enable multiple Azure VMs or
scale sets Azure Resource Manager templates virtual machine scale sets across a
specified subscription or resource group
by using Azure PowerShell or Azure
Resource Manager templates.
Hybrid cloud Enable for the hybrid environment You can deploy to VMs or physical
computers that are hosted in your
datacenter or other cloud
environments.
NOTE
The following list of performance counters enabled by Azure Monitor for VMs does not limit you from enabling additional
counters you need to collect from VMs reporting to the workspace. Also, if you disable these counters, it will prevent the set
of performance charts included with the Performance feature from showing resource utilization from your VMs.
Management packs
When Azure Monitor for VMs is enabled and configured with a Log Analytics workspace, a management pack is
forwarded to all the Windows computers reporting to that workspace. If you have integrated your System Center
Operations Manager management group with the Log Analytics workspace, the Service Map management pack is
deployed from the management group to the Windows computers reporting to the management group.
The management pack is named Microsoft.IntelligencePacks.ApplicationDependencyMonitor. Its written to
%Programfiles%\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs\ folder. The data source
that the management pack uses is
%Program files%\Microsoft Monitoring Agent\Agent\Health Service State\Resources\
<AutoGeneratedID>\Microsoft.EnterpriseManagement.Advisor.ApplicationDependencyMonitorDataSource.dll
.
NOTE
For information about viewing or deleting personal data, see Azure Data Subject Requests for the GDPR. For more
information about GDPR, see the GDPR section of the Service Trust portal.
Now that you've enabled monitoring for your VM, monitoring information is available for analysis in Azure
Monitor for VMs.
Next steps
To learn how to use the Performance monitoring feature, see View Azure Monitor for VMs Performance. To view
discovered application dependencies, see View Azure Monitor for VMs Map.
Enable Azure Monitor for VMs (preview) for
evaluation
12/13/2019 • 2 minutes to read • Edit Online
You can evaluate Azure Monitor for VMs (preview ) on a small number of Azure virtual machines (VMs) or on a
single VM or virtual machine scale set. The easiest and most direct way to enable monitoring is from the Azure
portal. Your goal is to monitor your VMs and discover any performance or availability issues.
Before you begin, review the prerequisites and make sure your subscription and resources meet the requirements.
6. On the Azure Monitor Insights Onboarding page, if you have an existing Log Analytics workspace in the
same subscription, select it in the drop-down list.
The list preselects the default workspace and location where the VM is deployed in the subscription.
NOTE
To create a new Log Analytics workspace to store the monitoring data from the VM, see Create a Log Analytics
workspace. Your Log Analytics workspace must belong to one of the supported regions.
After you've enabled monitoring, you might need to wait about 10 minutes before you can view the health metrics
for the VM.
After you've enabled monitoring, you might need to wait about 10 minutes before you can view the monitoring
data for the scale set.
NOTE
If you use a manual upgrade model for your scale set, upgrade the instances to complete the setup. You can start the
upgrades from the Instances page, in the Settings section.
Now that you've enabled monitoring for your VM or virtual machine scale set, the monitoring information is
available for analysis in Azure Monitor for VMs.
Next steps
To view discovered application dependencies, see Use Azure Monitor for VMs Map.
To identify bottlenecks, overall utilization, and your VM's performance, see View Azure VM performance.
Enable Azure Monitor for VMs (preview) by using
Azure Policy
12/13/2019 • 12 minutes to read • Edit Online
This article explains how to enable Azure Monitor for VMs (preview ) for Azure virtual machines or virtual machine
scale sets by using Azure Policy. At the end of this process, you will have successfully configured enabling the Log
Analytics and Dependency agents and identified virtual machines that aren't compliant.
To discover, manage, and enable Azure Monitor for VMs for all of your Azure virtual machines or virtual machine
scale sets, you can use either Azure Policy or Azure PowerShell. Azure Policy is the method we recommend
because you can manage policy definitions to effectively govern your subscriptions to ensure consistent
compliance and automatic enabling of newly provisioned VMs. These policy definitions:
Deploy the Log Analytics agent and the Dependency agent.
Report on compliance results.
Remediate for noncompliant VMs.
If you're interested in accomplishing these tasks with Azure PowerShell or an Azure Resource Manager template,
see Enable Azure Monitor for VMs (preview ) using Azure PowerShell or Azure Resource Manager templates.
This information is useful to help you plan and execute your governance scenario for Azure Monitor for VMs from
one central location. While Azure Policy provides a compliance view when a policy or initiative is assigned to a
scope, with this new page you can discover where the policy or initiative isn't assigned and assign it in place. All
actions like assign, view, and edit redirect to Azure Policy directly. The Azure Monitor for VMs Policy Coverage
page is an expanded and integrated experience for only the initiative Enable Azure Monitor for VMs.
From this page, you also can configure your Log Analytics workspace for Azure Monitor for VMs, which:
Installs the Service Map solution.
Enables the operating system performance counters used by the performance charts, workbooks, and your
custom log queries and alerts.
This option isn't related to any policy actions. It's available to provide an easy way to satisfy the prerequisites
required for enabling Azure Monitor for VMs.
What information is available on this page?
The following table provides a breakdown of the information that's presented on the policy coverage page and
how to interpret it.
FUNCTION DESCRIPTION
Total VMs Number of VMs under that scope. For a management group,
it's a sum of VMs nested under the subscriptions or child
management group.
Assignment Coverage Percent of VMs that are covered by the policy or initiative.
Compliant VMs Number of VMs that are compliant under the policy or
initiative. For the initiative Enable Azure Monitor for VMs,
this is the number of VMs that have both Log Analytics agent
and Dependency agent. In some cases, it might appear blank
because of no assignment, no VMs, or not enough
permissions. Information is provided under Compliance
State.
When you assign the policy or initiative, the scope selected in the assignment could be the scope listed or a subset
of it. For instance, you might have created an assignment for a subscription (policy scope) and not a management
group (coverage scope). In this case, the value of Assignment Coverage indicates the VMs in the policy or
initiative scope divided by the VMs in coverage scope. In another case, you might have excluded some VMs,
resource groups, or a subscription from policy scope. If the value is blank, it indicates that either the policy or
initiative doesn't exist or you don't have permission. Information is provided under Assignment Status.
[Preview]: Enable Azure Monitor for Enable Azure Monitor for the virtual Initiative
VMs machines in the specified scope
(management group, subscription, or
resource group). Takes Log Analytics
workspace as a parameter.
[Preview]: Audit Log Analytics agent Reports VMs as noncompliant if the VM Policy
deployment – VM image (OS) unlisted image (OS) isn't defined in the list and
the agent isn't installed.
[Preview]: Deploy Dependency agent for Deploy Dependency agent for Linux Policy
Linux VMs VMs if the VM image (OS) is defined in
the list and the agent isn't installed.
[Preview]: Deploy Dependency agent for Deploy Dependency agent for Windows Policy
Windows VMs VMs if the VM image (OS) is defined in
the list and the agent isn't installed.
[Preview]: Deploy Log Analytics agent Deploy Log Analytics agent for Linux Policy
for Linux VMs VMs if the VM image (OS) is defined in
the list and the agent isn't installed.
[Preview]: Deploy Log Analytics agent Deploy Log Analytics agent for Policy
for Windows VMs Windows VMs if the VM image (OS) is
defined in the list and the agent isn't
installed.
[Preview]: Enable Azure Monitor for Enable Azure Monitor for the virtual Initiative
virtual machine scale sets machine scale sets in the specified
scope (management group,
subscription, or resource group). Takes
Log Analytics workspace as a parameter.
Note: If your scale set upgrade policy is
set to Manual, apply the extension to all
the VMs in the set by calling upgrade
on them. In the CLI, this is
az vmss update-instances .
[Preview]: Audit Dependency agent Reports virtual machine scale set as Policy
deployment in virtual machine scale noncompliant if the VM image (OS) isn't
sets – VM image (OS) unlisted defined in the list and the agent isn't
installed.
[Preview]: Audit Log Analytics agent Reports virtual machine scale set as Policy
deployment in virtual machine scale noncompliant if the VM image (OS) isn't
sets – VM image (OS) unlisted defined in the list and the agent isn't
installed.
NAME DESCRIPTION TYPE
[Preview]: Deploy Dependency agent for Deploy Dependency agent for Linux Policy
Linux virtual machine scale sets virtual machine scale sets if the VM
image (OS) is defined in the list and the
agent isn't installed.
[Preview]: Deploy Dependency agent for Deploy Dependency agent for Windows Policy
Windows virtual machine scale sets virtual machine scale sets if the VM
image (OS) is defined in the list and the
agent isn't installed.
[Preview]: Deploy Log Analytics agent Deploy Log Analytics agent for Linux Policy
for Linux virtual machine scale sets virtual machine scale sets if the VM
Image (OS) is defined in the list and the
agent isn't installed.
[Preview]: Deploy Log Analytics agent Deploy Log Analytics agent for Policy
for Windows virtual machine scale sets Windows virtual machine scale sets if
the VM image (OS) is defined in the list
and the agent isn't installed.
NOTE
If the workspace is beyond the scope of the assignment, grant Log Analytics Contributor permissions to the policy
assignment's Principal ID. If you don't do this, you might see a deployment failure like
The client '343de0fe-e724-46b8-b1fb-97090f7054ed' with object id '343de0fe-e724-46b8-b1fb-
97090f7054ed' does not have authorization to perform action
'microsoft.operationalinsights/workspaces/read' over scope ...
To grant access, review how to manually configure the managed identity.
The Managed Identity check box is selected because the initiative being assigned includes a policy with the
deployIfNotExists effect.
9. In the Manage Identity location drop-down list, select the appropriate region.
10. Select Assign.
After you create the assignment, the Azure Monitor for VMs Policy Coverage page updates Assignment
Coverage, Assignment Status, Compliant VMs, and Compliance State to reflect the changes.
The following matrix maps each possible compliance state for the initiative.
Compliant All VMs in the scope have the Log Analytics and Dependency
agents deployed to them.
Not Compliant Not all VMs in the scope have the Log Analytics and
Dependency agents deployed to them and might require
remediation.
Success All VMs in the scope have the Log Analytics and Dependency
agents deployed to them.
Based on the results of the policies included with the initiative, VMs are reported as noncompliant in the following
scenarios:
Log Analytics agent or Dependency agent isn't deployed.
This scenario is typical for a scope with existing VMs. To mitigate it, deploy the required agents by creating
remediation tasks on a noncompliant policy.
[Preview ]: Deploy Dependency agent for Linux VMs
[Preview ]: Deploy Dependency agent for Windows VMs
[Preview ]: Deploy Log Analytics agent for Linux VMs
[Preview ]: Deploy Log Analytics agent for Windows VMs
VM image (OS ) isn't identified in the policy definition.
The criteria of the deployment policy include only VMs that are deployed from well-known Azure VM
images. Check the documentation to see whether the VM OS is supported. If it isn't supported, duplicate
the deployment policy and update or modify it to make the image compliant.
[Preview ]: Audit Dependency agent deployment – VM image (OS ) unlisted
[Preview ]: Audit Log Analytics agent deployment – VM image (OS ) unlisted
VMs aren't logging in to the specified Log Analytics workspace.
It's possible that some VMs in the initiative scope are logging in to a Log Analytics workspace other than
the one that's specified in the policy assignment. This policy is a tool to identify which VMs are reporting to
a noncompliant workspace.
[Preview ]: Audit Log Analytics workspace for VM – Report mismatch
Next steps
Now that monitoring is enabled for your virtual machines, this information is available for analysis with Azure
Monitor for VMs.
To view discovered application dependencies, see View Azure Monitor for VMs Map.
To identify bottlenecks and overall utilization with your VM's performance, see View Azure VM
performance.
Enable Azure Monitor for VMs (preview) using Azure
PowerShell or Resource Manager templates
1/14/2020 • 9 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
This article explains how to enable Azure Monitor for VMs (preview ) for Azure virtual machines or virtual
machine scale sets by using Azure PowerShell or Azure Resource Manager templates. At the end of this process,
you will have successfully begun monitoring all of your virtual machines and learn if any are experiencing
performance or availability issues.
"plan": {
"name": "[concat('ServiceMap', '(', parameters('WorkspaceName'),')')]",
"publisher": "Microsoft",
"product": "[Concat('OMSGallery/', 'ServiceMap')]",
"promotionCode": ""
}
}
]
}
]
}
The configuration change can take a few minutes to finish. When it's finished, a message displays
that's similar to the following and includes the result:
provisioningState : Succeeded
To run the following command by using the Azure CLI:
az login
az account set --subscription "Subscription Name"
az group deployment create --name DeploySolutions --resource-group <ResourceGroupName> --
template-file InstallSolutionsForVMInsights.json --parameters WorkspaceName=<workspaceName>
WorkspaceLocation=<WorkspaceLocation - example: eastus>
The configuration change can take a few minutes to finish. When it's finished, a message is displayed
that's similar to the following and includes the result:
provisioningState : Succeeded
NOTE
The template needs to be deployed in the same resource group as the resource to be brought on board.
The configuration change can take a few minutes to finish. When it's finished, a message displays that's similar to
the following and includes the result:
provisioningState : Succeeded
az login
az account set --subscription "Subscription Name"
az group deployment create --resource-group <ResourceGroupName> --template-file <Template.json> --parameters
<Parameters.json>
provisioningState : Succeeded
SYNOPSIS
This script installs VM extensions for Log Analytics and the Dependency agent as needed for VM Insights.
This script installs VM extensions for Log Analytics and the Dependency agent as needed for VM Insights.
SYNTAX
.\Install-VMInsights.ps1 [-WorkspaceId] <String> [-WorkspaceKey] <String> [-SubscriptionId] <String> [[-
ResourceGroup]
<String>] [[-Name] <String>] [[-PolicyAssignmentName] <String>] [-ReInstall] [-TriggerVmssManualVMUpdate]
[-Approve] [-WorkspaceRegion] <String>
[-WhatIf] [-Confirm] [<CommonParameters>]
DESCRIPTION
This script installs or reconfigures the following on VMs and virtual machine scale sets:
- Log Analytics VM extension configured to supplied Log Analytics workspace
- Dependency agent VM extension
Script will show you a list of VMs or virtual machine scale sets that will apply to and let you confirm to
continue.
Use -Approve switch to run without prompting, if all required parameters are provided.
If the extensions are already installed, they will not install again.
Use -ReInstall switch if you need to, for example, update the workspace.
Use -WhatIf if you want to see what would happen in terms of installs, what workspace configured to, and
status of the extension.
PARAMETERS
-WorkspaceId <String>
Log Analytics WorkspaceID (GUID) for the data to be sent to
-WorkspaceKey <String>
Log Analytics Workspace primary or secondary key
-SubscriptionId <String>
SubscriptionId for the VMs/VM Scale Sets
If using PolicyAssignmentName parameter, subscription that VMs are in
-ResourceGroup <String>
<Optional> Resource Group to which the VMs or VM Scale Sets belong
-Name <String>
<Optional> To install to a single VM/VM Scale Set
-PolicyAssignmentName <String>
<Optional> Take the input VMs to operate on as the Compliance results from this Assignment
If specified will only take from this source.
-ReInstall [<SwitchParameter>]
<Optional> If VM/VM Scale Set is already configured for a different workspace, set this to change to
the new workspace
-TriggerVmssManualVMUpdate [<SwitchParameter>]
<Optional> Set this flag to trigger update of VM instances in a scale set whose upgrade policy is set
to Manual
-Approve [<SwitchParameter>]
<Optional> Gives the approval for the installation to start with no confirmation prompt for the listed
VMs/VM Scale Sets
-WorkspaceRegion <String>
Region the Log Analytics Workspace is in
Supported values: "East US","eastus","Southeast Asia","southeastasia","West Central
US","westcentralus","West Europe","westeurope"
For Health supported is: "East US","eastus","West Central US","westcentralus"
-WhatIf [<SwitchParameter>]
<Optional> See what would happen in terms of installs.
If extension is already installed will show what workspace is currently configured, and status of the
VM extension
-Confirm [<SwitchParameter>]
<Optional> Confirm every action
<CommonParameters>
This cmdlet supports the common parameters: Verbose, Debug,
ErrorAction, ErrorVariable, WarningAction, WarningVariable,
OutBuffer, PipelineVariable, and OutVariable. For more information, see
about_CommonParameters (https:/go.microsoft.com/fwlink/?LinkID=113216).
Specify to reinstall extensions even if already installed, for example, to update to a different workspace
Specify to use a PolicyAssignmentName for source and to reinstall (move to a new workspace)
The following example demonstrates using the PowerShell commands in the folder to enable Azure Monitor for
VMs and understand the expected output:
$WorkspaceId = "<GUID>"
$WorkspaceKey = "<Key>"
$SubscriptionId = "<GUID>"
.\Install-VMInsights.ps1 -WorkspaceId $WorkspaceId -WorkspaceKey $WorkspaceKey -SubscriptionId $SubscriptionId
-WorkspaceRegion eastus
Getting list of VMs or virtual machine scale sets matching criteria specified
db-ws-1 VM running
db-ws2012 VM running
This operation will install the Log Analytics and Dependency agent extensions on the previous two VMs or
virtual machine scale sets.
VMs in a non-running state will be skipped.
Extension will not be reinstalled if already installed. Use -ReInstall if desired, for example, to update
workspace.
Confirm
Continue?
[Y] Yes [N] No [S] Suspend [?] Help (default is "Y"): y
Summary:
Succeeded: (4)
db-ws-1 : Successfully deployed DependencyAgentWindows
db-ws-1 : Successfully deployed MicrosoftMonitoringAgent
db-ws2012 : Successfully deployed DependencyAgentWindows
db-ws2012 : Successfully deployed MicrosoftMonitoringAgent
Failed: (0)
Next steps
Now that monitoring is enabled for your virtual machines, this information is available for analysis with Azure
Monitor for VMs.
To view discovered application dependencies, see View Azure Monitor for VMs Map.
To identify bottlenecks and overall utilization with your VM's performance, see View Azure VM
Performance.
Enable Azure Monitor for VMs (preview) for a hybrid
environment
1/14/2020 • 7 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
This article explains how to enable Azure Monitor for VMs (preview ) for virtual machines or physical computers
hosted in your datacenter or other cloud environment. At the end of this process, you will have successfully begun
monitoring your virtual machines in your environment and learn if they are experiencing any performance or
availability issues.
Before you get started, be sure to review the prerequisites and verify that your subscription and resources meet
the requirements. Review the requirements and deployment methods for the Log Analytics Linux and Windows
agent.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
NOTE
The Azure Monitor for VMs Map Dependency agent doesn't transmit any data itself, and it doesn't require any changes to
firewalls or ports. The Map data is always transmitted by the Log Analytics agent to the Azure Monitor service, either directly
or through the Operations Management Suite gateway if your IT security policies don't allow computers on the network to
connect to the internet.
NOTE
The information described in this article for deploying the Dependency agent is also applicable to the Service Map solution.
Install the Dependency agent on Windows
You can install the Dependency agent manually on Windows computers by running
InstallDependencyAgent-Windows.exe . If you run this executable file without any options, it starts a setup wizard that
you can follow to install the agent interactively.
NOTE
Administrator privileges are required to install or uninstall the agent.
The following table highlights the parameters that are supported by setup for the agent from the command line.
PARAMETER DESCRIPTION
For example, to run the installation program with the /? parameter, enter InstallDependencyAgent-
Windows.exe /?.
Files for the Windows Dependency agent are installed in C:\Program Files\Microsoft Dependency Agent by
default. If the Dependency agent fails to start after setup is finished, check the logs for detailed error information.
The log directory is %Programfiles%\Microsoft Dependency Agent\logs.
NOTE
Root access is required to install or configure the agent.
PARAMETER DESCRIPTION
--check Check permissions and the operating system, but don't install
the agent.
For example, to run the installation program with the -help parameter, enter InstallDependencyAgent-
Linux64.bin -help.
Install the Linux Dependency agent as root by running the command sh InstallDependencyAgent-Linux64.bin .
If the Dependency agent fails to start, check the logs for detailed error information. On Linux agents, the log
directory is /var/opt/microsoft/dependency-agent/log.
Files for the Dependency agent are placed in the following directories:
FILES LOCATION
.\InstallDependencyAgent-Windows.exe /S
$DAPackageLocalPath = "C:\InstallDependencyAgent-Windows.exe"
Node localhost
{
# Download and install the Dependency agent
xRemoteFile DAPackage
{
Uri = "https://aka.ms/dependencyagentwindows"
DestinationPath = $DAPackageLocalPath
}
xPackage DA
{
Ensure="Present"
Name = "Dependency Agent"
Path = $DAPackageLocalPath
Arguments = '/S'
ProductId = ""
InstalledCheckRegKey =
"HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\DependencyAgent"
InstalledCheckRegValueName = "DisplayName"
InstalledCheckRegValueData = "Dependency Agent"
DependsOn = "[xRemoteFile]DAPackage"
}
}
}
"plan": {
"name": "[concat('ServiceMap', '(', parameters('WorkspaceName'),')')]",
"publisher": "Microsoft",
"product": "[Concat('OMSGallery/', 'ServiceMap')]",
"promotionCode": ""
}
}
]
}
]
}
The configuration change can take a few minutes to finish. When it's finished, a message displays that's
similar to the following and includes the result:
provisioningState : Succeeded
After you've enabled monitoring, it might take about 10 minutes before you can view the health state and
metrics for the hybrid computer.
Troubleshooting
VM doesn't appear on the map
If your Dependency agent installation succeeded, but you don't see your computer on the map, diagnose the
problem by following these steps.
1. Is the Dependency agent installed successfully? You can validate this by checking to see if the service is
installed and running.
Windows: Look for the service named "Microsoft Dependency agent."
Linux: Look for the running process "microsoft-dependency-agent."
2. Are you on the Free pricing tier of Log Analytics? The Free plan allows for up to five unique computers. Any
subsequent computers won't show up on the map, even if the prior five are no longer sending data.
3. Is the computer sending log and perf data to Azure Monitor Logs? Perform the following query for your
computer:
Did it return one or more results? Is the data recent? If so, your Log Analytics agent is operating correctly
and communicating with the service. If not, check the agent on your server: Log Analytics agent for
Windows troubleshooting or Log Analytics agent for Linux troubleshooting.
Computer appears on the map but has no processes
If you see your server on the map, but it has no process or connection data, that indicates that the Dependency
agent is installed and running, but the kernel driver didn't load.
Check the C:\Program Files\Microsoft Dependency Agent\logs\wrapper.log file (Windows) or
/var/opt/microsoft/dependency-agent/log/service.log file (Linux). The last lines of the file should indicate why the
kernel didn't load. For example, the kernel might not be supported on Linux if you updated your kernel.
Next steps
Now that monitoring is enabled for your virtual machines, this information is available for analysis with Azure
Monitor for VMs.
To view discovered application dependencies, see View Azure Monitor for VMs Map.
To identify bottlenecks and overall utilization with your VM's performance, see View Azure VM
performance.
Integrate System Center Operations Manager with
Azure Monitor for VMs Map feature
12/13/2019 • 5 minutes to read • Edit Online
In Azure Monitor for VMs, you can view discovered application components on Windows and Linux virtual
machines (VMs) that run in Azure or your environment. With this integration between the Map feature and System
Center Operations Manager, you can automatically create distributed application diagrams in Operations Manager
that are based on the dynamic dependency maps in Azure Monitor for VMs. This article describes how to configure
your System Center Operations Manager management group to support this feature.
NOTE
If you have already deployed Service Map, you can view your maps in Azure Monitor for VMs, which includes additional
features to monitor VM health and performance. The Map feature of Azure Monitor for VMs is intended to replace the
standalone Service Map solution. To learn more, see Azure Monitor for VMs overview.
Prerequisites
A System Center Operations Manager management group (2012 R2 or later).
A Log Analytics workspace configured to support Azure Monitor for VMs.
One or more Windows and Linux virtual machines or physical computers that are monitored by Operations
Manager and sending data to your Log Analytics workspace. Linux servers reporting to an Operations Manager
management group need to be configured to directly connect to Azure Monitor. For more information, review
the overview in Collect log data with the Log Analytics agent.
A service principal with access to the Azure subscription that is associated with the Log Analytics workspace.
For more information, go to Create a service principal.
Configure integration
After you install the Service Map management pack, a new node, Service Map, is displayed under Operations
Management Suite in the Administration pane of your Operations Manager Operations console.
NOTE
Operations Management Suite was a collection of services that included Log Analytics, is now part of Azure Monitor.
To configure Azure Monitor for VMs Map integration, do the following:
1. To open the configuration wizard, in the Service Map Overview pane, click Add workspace.
2. In the Connection Configuration window, enter the tenant name or ID, application ID (also known as the
username or clientID ), and password of the service principal, and then click Next. For more information, go
to Create a service principal.
3. In the Subscription Selection window, select the Azure subscription, Azure resource group (the one that
contains the Log Analytics workspace), and Log Analytics workspace, and then click Next.
4. In the Machine Group Selection window, you choose which Service Map Machine Groups you want to
sync to Operations Manager. Click Add/Remove Machine Groups, choose groups from the list of
Available Machine Groups, and click Add. When you are finished selecting groups, click Ok to finish.
5. In the Server Selection window, you configure the Service Map Servers Group with the servers that you
want to sync between Operations Manager and the Map feature. Click Add/Remove Servers.
For the integration to build a distributed application diagram for a server, the server must be:
Monitored by Operations Manager
Configured to report to the Log Analytics workspace configured with Azure Monitor for VMs
Listed in the Service Map Servers Group
6. Optional: Select the All Management Servers Resource Pool to communicate with Log Analytics, and then
click Add Workspace.
It might take a minute to configure and register the Log Analytics workspace. After it is configured,
Operations Manager initiates the first Map sync.
Monitor integration
After the Log Analytics workspace is connected, a new folder, Service Map, is displayed in the Monitoring pane of
the Operations Manager Operations console.
NOTE
These alerts are not Log Analytics alerts synced with Operations Manager, they are generated in the management
group based on workflows defined in the Service Map management pack.
Servers: Lists the monitored servers that are configured to sync from Azure Monitor for VMs Map feature.
Machine Group Dependency Views: Lists all machine groups that are synced from the Map feature. You
can click any group to view its distributed application diagram.
Server Dependency Views: Lists all servers that are synced from the Map feature. You can click any server
to view its distributed application diagram.
NOTE
Operations Management Suite was a collection of services that included Log Analytics, which is now part of Azure Monitor.
You can configure only one Log Analytics workspace in this current release.
Azure includes services for specific roles or tasks in the monitoring space, but it doesn't provide in-depth health
perspectives of operating systems (OSs) hosted on Azure virtual machines (VMs). Although you can use Azure
Monitor for different conditions, it's not designed to model and represent the health of core components, or the
overall health of VMs.
By using Azure Monitor for VMs health, you can actively monitor the availability and performance of a Windows
or Linux guest OS. The health feature uses a model that represents key components and their relationships,
provides criteria that specifies how to measure component health, and sends an alert when it detects an unhealthy
condition.
Viewing the overall health state of an Azure VM and the underlying OS can be observed from two perspectives:
directly from a VM, or across all VMs in a resource group from Azure Monitor.
This article shows how to quickly assess, investigate, and resolve health issues when they are detected by the
Azure Monitor for VMs health feature.
For information about configuring Azure Monitor for VMs, see Enable Azure Monitor for VMs.
NOTE
We recently announced changes we are making to the Health feature based on the feedback we have received from our
public preview customers. Given the number of changes we will be making, we are going to stop offering the Health feature
for new customers. Existing customers can continue to use the health feature. For more details, please refer to our General
Availability FAQ.
LOOKBACK
MONITOR FREQUENCY DURATION ALERT ON WORKLOAD
NAME (MIN) (MIN) OPERATOR THRESHOLD STATE SEVERITY CATEGORY
NOTE
Lookback Duration represents how often the look back window checks the metric values, such as over the last five minutes.
NOTE
Frequency represents how often the metric alert checks if the conditions are met, such as every one minute. It is the rate at
which health criterion is executed, and lookback is the duration over which health criterion is evaluated. For example, health
criterion is evaluating if the condition CPU utilization is greater than 95 percent with a frequency of 5 minutes and remains
greater than 95% for 15 minutes (3 consecutive evaluation cycles), then the state is updated to critical severity if it wasn't
already.
In the Guest VM health section, the table shows the health rollup of performance components monitored by
health criteria for the VM, and the total number of VM health alerts raised by unhealthy components. These
components include CPU, Memory, Disk, and Network. Expand the chevron next to Guest VM health to view the
health its components.
Selecting the state next to the component will open the Health Diagnostics experience in the context of the selected
component. It shows the composition of the state of that component, describing what health criteria are used to
compute its health. For more information, see Health Diagnostics and working with health criteria. For more
information about alerts, see Alerts.
The health states defined for a VM are described in the following table:
Select any column including VM count, Critical, Warning, Healthy, or Unknown. View the list of filtered results
in the Virtual Machines page that match the column selected.
For example, to review all VMs that run Red Hat Enterprise Linux release 7.5, select the VM count value for that
OS, and it will list the VMs matching that filter and their current health state.
You click Show Health check box and the health state is returned for the filtered results in the table.
For any one of the items in the list, you can click the corresponding health state to launch Health Diagnostics,
which shows how health is evaluated for the selected VM.
In the Virtual Machines page, if you select the name of a VM under the column VM Name, you're directed to the
VM instance page. This page provides more details of the alerts and health criteria issues that are affecting the
selected VM. Filter the health state details by selecting Health State icon in the upper-left corner of the page to
see which components are unhealthy. You can also view VM Health alerts raised by an unhealthy component
categorized by alert severity.
From the VM list view, select the name of a VM to open the Health page for that VM, similarly as if you selected
Insights (preview) from the VM directly.
The Virtual Machines (preview) in Azure Monitor page shows a rollup health status for the VM and alerts.
This health status is categorized by severity, which represents VM health alerts raised when the health state
changed from healthy to unhealthy, based on criteria. Selecting VMs in critical condition opens a page with a list
of one or more VMs in a critical health state.
Selecting the health status for one of the VMs shows the Health Diagnostics view of the VM. In this view, you
can determine which health criteria is reflecting a health-state issue. When the Health Diagnostics page opens, it
shows all the VM components and their associated health criteria with the current health state.
For more information, see Health diagnostics.
Selecting View all health criteria opens a page showing a list of all the health criteria available with this feature.
The information can be further filtered based on the following options:
Type. There are three types of health criteria to assess conditions and roll up the overall health state of a
monitored VM:
Unit. Measures some aspect of a VM. This health criteria type might be checking a performance counter
to determine the performance of the component, running a script to perform a synthetic transaction, or
watching for an event that indicates an error. The filter is set to unit by default.
Dependency. Provides a health rollup between different entities. This health criteria allows the health of
an entity to depend on the health of another type of entity that it relies on for successful operation.
Aggregate. Provides a combined health state of similar health criteria. Unit and dependency health
criterion are typically configured under an aggregate health criterion. In addition to providing better
general organization of the many different health criteria targeted at an entity, aggregate health criterion
provides a unique health state for distinct categories of the entities.
Category. The type of health criteria used to group similar criteria for reporting purposes. These categories
are Availability and Performance.
To see which instances are unhealthy, select a value under the Unhealthy Component column. In this page, a
table lists the components that are in a critical health state.
Health diagnostics
The Health Diagnostics page allows you to visualize the health model of a VM. This page lists all VM
components, associated health criteria, state changes, and other significant issues identified by monitored
components related to the VM.
Component model
The leftmost column in the Health Diagnostics page is Component Model. All components, which are
associated with the VM, are displayed in this column along with their current health state.
In the following example, the discovered components are Disk, Logical Disk, Processor, Memory, and
Operating System. Multiple instances of these components are discovered and displayed in this column.
For example, the following figure shows that the VM has two instances of logical disks, C: and D:, which are in a
healthy state:
Health criteria
The center column in the Health Diagnostics page is Health Criteria. The health model defined for the VM is
displayed in a hierarchical tree. The health model for a VM consists of unit and aggregate health criteria.
A health criterion measures the health of a monitored instance, which could be a threshold value, state of an entity,
and so on. A health criterion has either two or three configurable health state thresholds, as described earlier. At
any point, the health criterion can be in only one of its potential states.
The health model defines criteria that determine the health of the overall target and components of the target. The
hierarchy of criteria is shown in the Health Criteria section on the Health Diagnostics page.
The health-rollup policy is part of the configuration of aggregate health criteria (the default is set to worst-of). You
can find a default set of health criteria running as part of this feature in the Monitoring configuration details
section of this article.
You can also use the Azure Monitor REST API monitor instances list by resource to get a list of all health criteria.
This criteria includes configuration details running against the Azure VM resource.
The Unit health criteria type can have its configuration modified by selecting the ellipsis link to the right side.
Select Show Details to open the configuration pane.
In the configuration pane for the selected health criteria, if you use the example Average Disk Seconds Per
Write, the threshold can be configured with a different numeric value. It's a two-state monitor, meaning it can
change only from Healthy to Warning.
Other health criteria sometimes use three states, where you can configure the value for warning and critical health-
state thresholds. You can also modify a threshold by using Azure Monitor REST API monitor configuration.
NOTE
Applying health criteria configuration changes to one instance applies them to all monitored instances. For example, if you
select Disk -1 D: and then modify the Average Disk Seconds Per Write threshold, the change applies to all instances
discovered and monitored on the VM.
If you want to learn more about health criteria, we've included knowledge articles to help you identify problems,
causes, and resolutions. Select View information on the page to see the related knowledge article.
To review all the knowledge articles included with Azure Monitor for VMs health, see Azure Monitor health
knowledge documentation.
State changes
The far-right column of the Health Diagnostics page is State Changes. This column lists all the state changes
associated with the health criteria selected in the Health Criteria section, or the state change of the VM if a VM
was selected from the Component Model or Health Criteria column of the table.
The following section shows the health-criteria state and the associated time. This information shows the latest
state at the top of the column.
Association of Component Model, Health Criteria, and State Changes columns
The three columns are interlinked with each other. When you select an instance in the Component Model
column, the Health Criteria column is filtered to that component view. Correspondingly, the State Changes
column is updated based on the selected health criteria.
For example, if you select Disk - 1 D: from the list under Component Model, Health Criteria filters to Disk - 1D:,
and State Changes shows the state change based on the availability of Disk - 1 D:.
To see an updated health state, you can refresh the Health Diagnostics page by selecting the Refresh link. If there
is an update to the health criterion's health state based on the pre-defined polling interval, this task allows you to
avoid waiting and reflects the latest health state. The Health Criteria State is a filter that lets you scope the results
based on the selected health state: Healthy, Warning, Critical, Unknown, and All. The Last Updated time in the
upper-right corner represents the last time the Health Diagnostics page was refreshed.
Alerts
Azure Monitor for VMs health integrates with Azure Alerts. It raises an alert when predefined criteria, when
detected, change from a healthy state to an unhealthy state. Alerts are categorized by severity, from Sev 0 through
Sev 4, with Sev 0 as the highest level.
Alerts aren't associated with an action group to notify you when the alert has been triggered. A user with Owner
role at the subscription scope must configure notifications by following the steps in the Configure alerts section.
The total number of VM Health alerts categorized by severity is available on the Health dashboard under the
Alerts section. When you select either the total number of alerts or the number corresponding to a severity level,
the Alerts page opens and lists all alerts matching your selection.
For example, if you select the row corresponding to Sev level 1, you'll see the following view:
The All Alerts page isn't scoped to show only alerts matching your selection. It's also filtered by Resource type to
show only health alerts raised by a VM resource. This format is reflected in the alert list, under the column Target
Resource, where it shows the Azure VM the raised alert when an unhealthy condition was met.
Alerts from other resource types or services are not intended to be included in this view. These alerts include log
alerts, which are based on log queries or metric alerts that you'd normally view from the default Azure Monitor All
Alerts page.
You can filter this view by selecting values in the drop-down menus at the top of the page.
COLUMN DESCRIPTION
Resource Group Select a single resource group. Only alerts with targets in the
selected resource group are included in the view.
Resource type Select one or more resource types. By default, only alerts of
target Virtual machines is selected and included in this view.
This column is only available after a resource group has been
specified.
Monitor Condition Select a monitor condition to filter alerts if they have been
fired or resolved by the system if the condition is no longer
active. Or, select All to include alerts of all conditions.
Monitor service Select a service or select All to include all services. Only alerts
from VM Insights are supported for this feature.
Time range Only alerts fired within the selected time window are included
in the view. Supported values are the past hour, the past 24
hours, the past 7 days, and the past 30 days.
When you select an alert, the Alert detail page is displayed. This page provides details of the alert and allows you
to change its state.
To learn more about managing alerts, see Create, view, and manage alerts using Azure Monitor.
NOTE
Creating new alerts based on health criteria or modifying existing health alert rules in Azure Monitor from the portal isn't
currently supported.
You can change an alert state for one or multiple alerts by selecting them, and then selecting Change state from
the All Alerts page in the upper-left corner. Select one of the states on the Change alert state pane, add a
description of the change in the Comment field, and then select Ok to commit your changes. When the
information is verified and the changes are applied, track the progress under Notifications in the menu.
Configure alerts
You can't manage certain alert-management tasks from the Azure portal. These tasks must be performed by using
the Azure Monitor REST API. Specifically:
Enabling or disabling an alert for health criteria
Setting up notifications for health criteria alerts
Each example uses ARMClient on your Windows machine. If you are not familiar with this method, see Using
ARMClient.
Enable or disable an alert rule
To enable or disable an alert for specific health criteria, the property alertGeneration must be modified with a
value of either Disabled or Enabled.
To identify the monitorId for specific health criteria, the following example shows how to query for that value for
the criteria LogicalDisk\Avg Disk Seconds Per Transfer:
1. In a terminal window, type armclient.exe login. Doing so prompts you to sign in to Azure.
2. Enter the following command to retrieve all the health criterion active on a specific VM and identify the
value for monitorId property:
armclient GET
"subscriptions/subscriptionId/resourceGroups/resourcegroupName/providers/Microsoft.Compute/virtualMachin
es/vmName/providers/Microsoft.WorkloadMonitor/monitors?api-version=2018-08-31-preview"
The following example shows the output of the armclient GET command. Take note of the value of
MonitorId. This value is required for the next step, where we must specify the ID of the health criteria and
modify its property to create an alert.
"id": "/subscriptions/a7f23fdb-e626-4f95-89aa-
3a360a90861e/resourcegroups/Lab/providers/Microsoft.Compute/virtualMachines/SVR01/providers/Microsoft.Wo
rkloadMonitor/monitors/ComponentTypeId='LogicalDisk',MonitorId='Microsoft_LogicalDisk_AvgDiskSecPerRead'
",
"name": "ComponentTypeId='LogicalDisk',MonitorId='Microsoft_LogicalDisk_AvgDiskSecPerRead'",
"type": "Microsoft.WorkloadMonitor/virtualMachines/monitors"
},
{
"properties": {
"description": "Monitor the performance counter LogicalDisk\\Avg Disk Sec Per Transfer",
"monitorId": "Microsoft_LogicalDisk_AvgDiskSecPerTransfer",
"monitorName": "Microsoft.LogicalDisk.AvgDiskSecPerTransfer",
"monitorDisplayName": "Average Logical Disk Seconds Per Transfer",
"parentMonitorName": null,
"parentMonitorDisplayName": null,
"monitorType": "Unit",
"monitorCategory": "PerformanceHealth",
"componentTypeId": "LogicalDisk",
"componentTypeName": "LogicalDisk",
"componentTypeDisplayName": "Logical Disk",
"monitorState": "Enabled",
"criteria": [
{
"healthState": "Warning",
"comparisonOperator": "GreaterThan",
"threshold": 0.1
}
],
"alertGeneration": "Enabled",
"frequency": 1,
"lookbackDuration": 17,
"documentationURL": "https://aka.ms/Ahcs1r",
"configurable": true,
"signalType": "Metrics",
"signalName": "VMHealth_Avg. Logical Disk sec/Transfer"
},
"etag": null,
armclient patch
subscriptions/subscriptionId/resourceGroups/resourcegroupName/providers/Microsoft.Compute/virtualMachine
s/vmName/providers/Microsoft.WorkloadMonitor/monitors/Microsoft_LogicalDisk_AvgDiskSecPerTransfer?api-
version=2018-08-31-preview "{'properties':{'alertGeneration':'Disabled'}}"
4. Enter the GET command used in step 2 to verify that the property value is set to Disabled.
Associate an action group with health criteria
Azure Monitor for VMs health supports SMS and email notifications when alerts are generated from unhealthy
health criteria. To configure notifications, note the name of the configured action group to send SMS or email
notifications.
NOTE
This action must be performed against each monitored VM that you want to receive a notification for. It doesn't apply to all
VMs in a resource group.
1. In a terminal window, enter armclient.exe login. Doing so prompts you to sign in to Azure.
2. Enter the following command to associate an action group with alert rules:
$payload = "{'properties':{'ActionGroupResourceIds':
['/subscriptions/subscriptionId/resourceGroups/resourcegroupName/providers/microsoft.insights/actionGrou
ps/actiongroupName']}}"
armclient PUT
https://management.azure.com/subscriptions/subscriptionId/resourceGroups/resourcegroupName/providers/Mic
rosoft.Compute/virtualMachines/vmName/providers/Microsoft.WorkloadMonitor/notificationSettings/default?
api-version=2018-08-31-preview $payload
3. To verify that the value of the property actionGroupResourceIds was successfully updated, enter the
following command:
armclient GET
"subscriptions/subscriptionName/resourceGroups/resourcegroupName/providers/Microsoft.Compute/virtualMach
ines/vmName/providers/Microsoft.WorkloadMonitor/notificationSettings?api-version=2018-08-31-preview"
{
"value": [
{
"properties": {
"actionGroupResourceIds": [
"/subscriptions/a7f23fdb-e626-4f95-89aa-
3a360a90861e/resourceGroups/Lab/providers/microsoft.insights/actionGroups/Lab-IT%20Ops%20Notify"
]
},
"etag": null,
"id": "/subscriptions/a7f23fdb-e626-4f95-89aa-
3a360a90861e/resourcegroups/Lab/providers/Microsoft.Compute/virtualMachines/SVR01/providers/Microsoft.Wo
rkloadMonitor/notificationSettings/default",
"name": "notificationSettings/default",
"type": "Microsoft.WorkloadMonitor/virtualMachines/notificationSettings"
}
],
"nextLink": null
}
Next steps
To identify limitations and overall VM performance, see View Azure VM Performance.
To learn about discovered application dependencies, see View Azure Monitor for VMs Map.
Use the Map feature of Azure Monitor for VMs
(preview) to understand application components
12/13/2019 • 6 minutes to read • Edit Online
In Azure Monitor for VMs, you can view discovered application components on Windows and Linux virtual
machines (VMs) that run in Azure or your environment. You can observe the VMs in two ways. View a map
directly from a VM or view a map from Azure Monitor to see the components across groups of VMs. This article
will help you understand these two viewing methods and how to use the Map feature.
For information about configuring Azure Monitor for VMs, see Enable Azure Monitor for VMs.
Sign in to Azure
Sign in to the Azure portal.
Close the Logs page and return to the Properties pane. There, select Alerts to view VM health-criteria alerts.
The Map feature integrates with Azure Alerts to show alerts for the selected server in the selected time range.
The server displays an icon for current alerts, and the Machine Alerts pane lists the alerts.
To make the Map feature display relevant alerts, create an alert rule that applies to a specific computer:
Include a clause to group alerts by computer (for example, by Computer interval 1 minute).
Base the alert on a metric.
For more information about Azure Alerts and creating alert rules, see Unified alerts in Azure Monitor.
In the upper-right corner, the Legend option describes the symbols and roles on the map. For a closer look at
your map and to move it around, use the zoom controls in the lower-right corner. You can set the zoom level and
fit the map to the size of the page.
Connection metrics
The Connections pane displays standard metrics for the selected connection from the VM over the TCP port.
The metrics include response time, requests per minute, traffic throughput, and links.
Failed connections
The map shows failed connections for processes and computers. A dashed red line indicates a client system is
failing to reach a process or port. For systems that use the Dependency agent, the agent reports on failed
connection attempts. The Map feature monitors a process by observing TCP sockets that fail to establish a
connection. This failure could result from a firewall, a misconfiguration in the client or server, or an unavailable
remote service.
Understanding failed connections can help you troubleshoot, validate migration, analyze security, and
understand the overall architecture of the service. Failed connections are sometimes harmless, but they often
point to a problem. Connections might fail, for example, when a failover environment suddenly becomes
unreachable or when two application tiers can't communicate with each other after a cloud migration.
Client groups
On the map, client groups represent client machines that connect to the mapped machine. A single client group
represents the clients for an individual process or machine.
To see the monitored clients and IP addresses of the systems in a client group, select the group. The contents of
the group appear below.
If the group includes monitored and unmonitored clients, you can select the appropriate section of the group's
doughnut chart to filter the clients.
Server-port groups
Server-port groups represent ports on servers that have inbound connections from the mapped machine. The
group contains the server port and a count of the number of servers that have connections to that port. Select
the group to see the individual servers and connections.
If the group includes monitored and unmonitored servers, you can select the appropriate section of the group's
doughnut chart to filter the servers.
Choose a workspace by using the Workspace selector at the top of the page. If you have more than one Log
Analytics workspace, choose the workspace that's enabled with the solution and that has VMs reporting to it.
The Group selector returns subscriptions, resource groups, computer groups, and virtual machine scale sets of
computers that are related to the selected workspace. Your selection applies only to the Map feature and doesn't
carry over to Performance or Health.
By default, the map shows the last 30 minutes. If you want to see how dependencies looked in the past, you can
query for historical time ranges of up to one hour. To run the query, use the TimeRange selector. You might run
a query, for example, during an incident or to see the status before a change.
Next steps
To identify bottlenecks, check performance, and understand overall utilization of your VMs, see View
performance status for Azure Monitor for VMs.
How to chart performance with Azure Monitor for
VMs (preview)
12/20/2019 • 7 minutes to read • Edit Online
Azure Monitor for VMs includes a set of performance charts that target several key performance indicators
(KPIs) to help you determine how well a virtual machine is performing. The charts show resource utilization over
a period of time so you can identify bottlenecks, anomalies, or switch to a perspective listing each machine to
view resource utilization based on the metric selected. While there are numerous elements to consider when
dealing with performance, Azure Monitor for VMs monitors key operating system performance indicators related
to processor, memory, network adapter, and disk utilization. Performance complements the health monitoring
feature and helps expose issues that indicate a possible system component failure, support tuning and
optimization to achieve efficiency, or support capacity planning.
On the Top N Charts tab, if you have more than one Log Analytics workspace, choose the workspace enabled
with the solution from the Workspace selector at the top of the page. The Group selector will return
subscriptions, resource groups, computer groups, and virtual machine scale sets of computers related to the
selected workspace that you can use to further filter results presented in the charts on this page and across the
other pages. Your selection only applies to the Performance feature and does not carry over to Health or Map.
By default, the charts show the last 24 hours. Using the TimeRange selector, you can query for historical time
ranges of up to 30 days to show how performance looked in the past.
The five capacity utilization charts shown on the page are:
CPU Utilization % - shows the top five machines with the highest average processor utilization
Available Memory - shows the top five machines with the lowest average amount of available memory
Logical Disk Space Used % - shows the top five machines with the highest average disk space used % across
all disk volumes
Bytes Sent Rate - shows the top five machines with highest average of bytes sent
Bytes Receive Rate - shows the top five machines with highest average of bytes received
Clicking on the pin icon at the upper right-hand corner of any one of the five charts will pin the selected chart to
the last Azure dashboard you last viewed. From the dashboard, you can resize and reposition the chart. Selecting
the chart from the dashboard will redirect you to Azure Monitor for VMs and load the correct scope and view.
Clicking on the icon located to the left of the pin icon on any one of the five charts opens the Top N List view.
Here you see the resource utilization for that performance metric by individual VM in a list view and which
machine is trending highest.
When you click on the virtual machine, the Properties pane is expanded on the right to show the properties of
the item selected, such as system information reported by the operating system, properties of the Azure VM, etc.
Clicking on one of the options under the Quick Links section will redirect you to that feature directly from the
selected VM.
Switch to the Aggregated Charts tab to view the performance metrics filtered by average or percentiles
measures.
NOTE
The list cannot show more than 500 machines at a time.
To filter the results on a specific virtual machine in the list, enter its computer name in the Search by name
textbox.
If you would rather view utilization from a different performance metric, from the Metric drop-down list select
Available Memory, Logical Disk Space Used %, Network Received Byte/s, or Network Sent Byte/s and
the list updates to show utilization scoped to that metric.
Selecting a virtual machine from the list opens the Properties panel on the right-side of the page and from here
you can select Performance detail. The Virtual Machine Detail page opens and is scoped to that VM, similar
in experience when accessing VM Insights Performance directly from the Azure VM.
Alerts
Performance metrics enabled as part of Azure Monitor for VMs do not include pre-configured alert rules. There
are health alerts corresponding to performance issues detected on your Azure VM, such as high CPU utilization,
low memory available, low disk space, etc. However, these health alerts only apply to all VMs enabled for Azure
Monitor for VMs.
However, we may only collect and store a subset of the performance metrics you require in the Log Analytics
workspace. If your monitoring strategy requires analysis or alerting that includes other performance metrics in
order to effectively evaluate capacity or health of the virtual machine, or you need the flexibility to specify your
own alerting criteria or logic, you can configure collection of those performance counters in Log Analytics and
define log alerts. While Log Analytics allows you to perform complex analysis with other data types, and provide
longer retention to support trend analysis, metrics on the other hand, are lightweight and capable of supporting
near real-time scenarios. They are collected by the Azure Diagnostic agent and stored in the Azure Monitor
metrics store, allowing you to create alerts with lower latency and at a lower cost.
Review the overview of collection of metrics and logs with Azure Monitor to further understand the fundamental
differences and other considerations before configuring collection of these additional metrics and alert rules.
Next steps
Learn how to use Workbooks that are included with Azure Monitor for VMs to further analyze
performance and network metrics.
To learn about discovered application dependencies, see View Azure Monitor for VMs Map.
Create interactive reports Azure Monitor for VMs
with workbooks
12/13/2019 • 12 minutes to read • Edit Online
Workbooks combine text,log queries, metrics, and parameters into rich interactive reports. Workbooks are editable
by any other team members who have access to the same Azure resources.
Workbooks are helpful for scenarios such as:
Exploring the usage of your virtual machine when you don't know the metrics of interest in advance: CPU
utilization, disk space, memory, network dependencies, etc. Unlike other usage analytics tools, workbooks let
you combine multiple kinds of visualizations and analyses, making them great for this kind of free-form
exploration.
Explaining to your team how a recently provisioned VM is performing, by showing metrics for key counters and
other log events.
Sharing the results of a resizing experiment of your VM with other members of your team. You can explain the
goals for the experiment with text, then show each usage metric and analytics queries used to evaluate the
experiment, along with clear call-outs for whether each metric was above- or below -target.
Reporting the impact of an outage on the usage of your VM, combining data, text explanation, and a discussion
of next steps to prevent outages in the future.
Azure Monitor for VMs includes several workbooks to get you started, and the following table summarizes them.
It launches the workbook gallery with a number of prebuilt workbooks to help you get started.
7. We'll start with the Default Template, which is located under the heading Quick start.
Editing workbook sections
Workbooks have two modes: editing mode, and reading mode. When the default template workbook is first
launched, it opens in editing mode. It shows all the content of the workbook, including any steps and parameters
that are otherwise hidden. Reading mode presents a simplified report style view. Reading mode allows you to
abstract away the complexity that went into creating a report while still having the underlying mechanics only a
few clicks away when needed for modification.
1. When you're done editing a section, click Done Editing in the bottom-left corner of the section.
2. To create a duplicate of a section, click the Clone this section icon. Creating duplicate sections is a great to
way to iterate on a query without losing previous iterations.
3. To move up a section in a workbook, click the Move up or Move down icon.
4. To remove a section permanently, click the Remove icon.
Custom width Makes an item an arbitrary size, so you can fit many items on
a single line allowing you to better organize your charts and
tables into rich interactive reports.
Conditionally visible Specify to hide steps based on a parameter when in reading
mode.
Export a parameter Allow a selected row in the grid or chart to cause later steps
to change values or become visible.
Show query when not editing Displays the query above the chart or table even when in
reading mode.
Show open in analytics button when not editing Adds the blue Analytics icon to the right-hand corner of the
chart to allow one-click access.
Most of these settings are fairly intuitive, but to understand Export a parameter it is better to examine a
workbook that makes use of this functionality.
One of the prebuilt workbooks - TCP Traffic, provides information on connection metrics from a VM.
The first section of the workbook is based on log query data. The second section is also based on log query data,
but selecting a row in the first table will interactively update the contents of the charts:
The behavior is possible through use of the When an item is selected, export a parameter advanced settings,
which are enabled in the table's log query.
The second log query then utilizes the exported values when a row is selected to create a set of values that are then
used by the section heading and charts. If no row is selected, it hides the section heading and charts.
For example, the hidden parameter in the second section uses the following reference from the row selected in the
grid:
VMConnection
| where TimeGenerated {TimeRange}
| where Computer in ("{ComputerName}") or '*' in ("{ComputerName}")
| summarize Sent = sum(BytesSent), Received = sum(BytesReceived) by bin(TimeGenerated, {TimeRange:grain})
Text Allows the user to edit a text box, and you can optionally
supply a query to fill in the default value.
Time range picker Allows the user to choose from a predefined set of time range
values, or pick from a custom time range.
Resource picker Allows the user to choose from the resources selected for the
workbook.
Using a text parameter
The value a user types in the text box is replaced directly in the query, with no escaping or quoting. If the value you
need is a string, the query should have quotes around the parameter (like '{parameter}').
The text parameter allows the value in a text box to be used anywhere. It can be a table name, column name,
function name, operator, etc. The text parameter type has a setting Get default value from analytics query,
which allows the workbook author to use a query to populate the default value for that text box.
When using the default value from a log query, only the first value of the first row (row 0, column 0) is used as the
default value. Therefore it is recommended to limit your query to return just one row and one column. Any other
data returned by the query is ignored.
Whatever value the query returns will be replaced directly with no escaping or quoting. If the query returns no
rows, the result of the parameter is either an empty string (if the parameter is not required) or undefined (if the
parameter is required).
Using a drop-down
The dropdown parameter type lets you create a drop-down control, allowing the selection of one or many values.
The dropdown is populated by a log query or JSON. If the query returns one column, the values in that column
are both the value and the label in the drop-down control. If the query returns two columns, the first column is the
value, and the second column is the label shown in the drop-down. If the query returns three columns, the third
column is used to indicate the default selection in that drop-down. This column can be any type, but the simplest is
to use bool or numeric types, where 0 is false, and 1 is true.
If the column is a string type, null/empty string is considered false, and any other value is considered true. For
single selection drop-downs, the first value with a true value is used as the default selection. For multiple selection
drop-downs, all values with a true value are used as the default selected set. The items in the drop-down are shown
in whatever order the query returned rows.
Let's look at the parameters present in the Connections Overview report. Click the edit symbol next to Direction.
[
{ "value": "inbound", "label": "Inbound"},
{ "value": "outbound", "label": "Outbound"}
]
A more applicable example is using a drop-down to pick from a set of performance counters by name:
Perf
| summarize by CounterName, ObjectName
| order by ObjectName asc, CounterName asc
| project Counter = pack('counter', CounterName, 'object', ObjectName), CounterName, group = ObjectName
Next steps
To identify limitations and overall VM performance, see View Azure VM Performance.
To learn about discovered application dependencies, see View Azure Monitor for VMs Map.
How to query logs from Azure Monitor for VMs
(preview)
12/20/2019 • 16 minutes to read • Edit Online
Azure Monitor for VMs collects performance and connection metrics, computer and process inventory data, and
health state information and forwards it to the Log Analytics workspace in Azure Monitor. This data is available for
query in Azure Monitor. You can apply this data to scenarios that include migration planning, capacity analysis,
discovery, and on-demand performance troubleshooting.
Map records
One record is generated per hour for each unique computer and process, in addition to the records that are
generated when a process or computer starts or is on-boarded to Azure Monitor for VMs Map feature. These
records have the properties in the following tables. The fields and values in the ServiceMapComputer_CL events
map to fields of the Machine resource in the ServiceMap Azure Resource Manager API. The fields and values in
the ServiceMapProcess_CL events map to the fields of the Process resource in the ServiceMap Azure Resource
Manager API. The ResourceName_s field matches the name field in the corresponding Resource Manager
resource.
There are internally generated properties you can use to identify unique processes and computers:
Computer: Use ResourceId or ResourceName_s to uniquely identify a computer within a Log Analytics
workspace.
Process: Use ResourceId to uniquely identify a process within a Log Analytics workspace. ResourceName_s is
unique within the context of the machine on which the process is running (MachineResourceName_s)
Because multiple records can exist for a specified process and computer in a specified time range, queries can
return more than one record for the same computer or process. To include only the most recent record, add
| summarize arg_max(TimeGenerated, *) by ResourceId to the query.
PROPERTY DESCRIPTION
To account for the impact of grouping, information about the number of grouped physical connections is provided
in the following properties of the record:
PROPERTY DESCRIPTION
Metrics
In addition to connection count metrics, information about the volume of data sent and received on a given logical
connection or network port are also included in the following properties of the record:
PROPERTY DESCRIPTION
BytesSent Total number of bytes that have been sent during the
reporting time window
BytesReceived Total number of bytes that have been received during the
reporting time window
The third type of data being reported is response time - how long does a caller spend waiting for a request sent
over a connection to be processed and responded to by the remote endpoint. The response time reported is an
estimation of the true response time of the underlying application protocol. It is computed using heuristics based
on the observation of the flow of data between the source and destination end of a physical network connection.
Conceptually, it is the difference between the time the last byte of a request leaves the sender, and the time when
the last byte of the response arrives back to it. These two timestamps are used to delineate request and response
events on a given physical connection. The difference between them represents the response time of a single
request.
In this first release of this feature, our algorithm is an approximation that may work with varying degree of success
depending on the actual application protocol used for a given network connection. For example, the current
approach works well for request-response based protocols such as HTTP (S ), but does not work with one-way or
message queue-based protocols.
Here are some important points to consider:
1. If a process accepts connections on the same IP address but over multiple network interfaces, a separate record
for each interface will be reported.
2. Records with wildcard IP will contain no activity. They are included to represent the fact that a port on the
machine is open to inbound traffic.
3. To reduce verbosity and data volume, records with wildcard IP will be omitted when there is a matching record
(for the same process, port, and protocol) with a specific IP address. When a wildcard IP record is omitted, the
IsWildcardBind record property with the specific IP address, will be set to "True" to indicate that the port is
exposed over every interface of the reporting machine.
4. Ports that are bound only on a specific interface have IsWildcardBind set to False.
Naming and Classification
For convenience, the IP address of the remote end of a connection is included in the RemoteIp property. For
inbound connections, RemoteIp is the same as SourceIp, while for outbound connections, it is the same as
DestinationIp. The RemoteDnsCanonicalNames property represents the DNS canonical names reported by the
machine for RemoteIp. The RemoteDnsQuestions and RemoteClassification properties are reserved for future use.
Geolocation
VMConnection also includes geolocation information for the remote end of each connection record in the
following properties of the record:
PROPERTY DESCRIPTION
Malicious IP
Every RemoteIp property in VMConnection table is checked against a set of IPs with known malicious activity. If
the RemoteIp is identified as malicious the following properties will be populated (they are empty, when the IP is
not considered malicious) in the following properties of the record:
PROPERTY DESCRIPTION
TLPLevel Traffic Light Protocol (TLP) Level is one of the defined values,
White, Green, Amber, Red.
Ports
Ports on a machine that actively accept incoming traffic or could potentially accept traffic, but are idle during the
reporting time window, are written to the VMBoundPort table.
Every record in VMBoundPort is identified by the following fields:
PROPERTY DESCRIPTION
The identity a port is derived from the above five fields and is stored in the PortId property. This property can be
used to quickly find records for a specific port across time.
Metrics
Port records include metrics representing the connections associated with them. Currently, the following metrics
are reported (the details for each metric are described in the previous section):
BytesSent and BytesReceived
LinksEstablished, LinksTerminated, LinksLive
ResposeTime, ResponseTimeMin, ResponseTimeMax, ResponseTimeSum
Here are some important points to consider:
If a process accepts connections on the same IP address but over multiple network interfaces, a separate record
for each interface will be reported.
Records with wildcard IP will contain no activity. They are included to represent the fact that a port on the
machine is open to inbound traffic.
To reduce verbosity and data volume, records with wildcard IP will be omitted when there is a matching record
(for the same process, port, and protocol) with a specific IP address. When a wildcard IP record is omitted, the
IsWildcardBind property for the record with the specific IP address, will be set to True. This indicates the port is
exposed over every interface of the reporting machine.
Ports that are bound only on a specific interface have IsWildcardBind set to False.
VMComputer records
Records with a type of VMComputer have inventory data for servers with the Dependency agent. These records
have the properties in the following table:
PROPERTY DESCRIPTION
SourceSystem Insights
HypervisorType hyperv
HostingProvider azure
PROPERTY DESCRIPTION
AzureVmScaleSetResourceId The unique identifier of the virtual machine scale set resource.
VMProcess record
Records with a type of VMProcess have inventory data for TCP -connected processes on servers with the
Dependency agent. These records have the properties in the following table:
PROPERTY DESCRIPTION
SourceSystem Insights
Group Process group name. Processes in the same group are logically
related, e.g., part of the same product or system component.
let Today = now(); VMComputer | extend DaysSinceBoot = Today - BootTime | summarize by Computer,
DaysSinceBoot, BootTime | sort by BootTime asc
Bound Ports
VMBoundPort
| where TimeGenerated >= ago(24hr)
| where Computer == 'admdemo-appsvr'
| distinct Port, ProcessName
VMBoundPort
| where Ip != "127.0.0.1"
| summarize by Computer, Machine, Port, Protocol
| summarize OpenPorts=count() by Computer, Machine
| order by OpenPorts desc
Score processes in your workspace by the number of ports they have open
VMBoundPort
| where Ip != "127.0.0.1"
| summarize by ProcessName, Port, Protocol
| summarize OpenPorts=count() by ProcessName
| order by OpenPorts desc
//
VMBoundPort
| where Ip != "127.0.0.1"
| summarize BytesSent=sum(BytesSent), BytesReceived=sum(BytesReceived),
LinksEstablished=sum(LinksEstablished), LinksTerminated=sum(LinksTerminated), arg_max(TimeGenerated,
LinksLive) by Machine, Computer, ProcessName, Ip, Port, IsWildcardBind
| project-away TimeGenerated
| order by Machine, Computer, Port, Ip, ProcessName
Next steps
If you are new to writing log queries in Azure Monitor, review how to use Log Analytics in the Azure portal
to write log queries.
Learn about writing search queries.
How to upgrade the Azure Monitor for VMs
Dependency agent
12/13/2019 • 2 minutes to read • Edit Online
After initial deployment of the Azure Monitor for VMs Dependency agent, updates are released that include bug
fixes or support of new features or functionality. This article helps you understand the methods available and how
to perform the upgrade manually or through automation.
Upgrade options
The Dependency agent for Windows and Linux can be upgraded to the latest release manually or automatically
depending on the deployment scenario and environment the machine is running in. The following methods can be
used to upgrade the agent.
Custom Azure VM images Manual install of Dependency agent for Updating VMs to the newest version of
Windows/Linux the agent needs to be performed from
the command line running the Windows
installer package or Linux self-extracting
and installable shell script bundle.
Non-Azure VMs Manual install of Dependency agent for Updating VMs to the newest version of
Windows/Linux the agent needs to be performed from
the command line running the Windows
installer package or Linux self-extracting
and installable shell script bundle.
InstallDependencyAgent-Windows.exe /S /RebootMode=manual
The /RebootMode=manual parameter prevents the upgrade from automatically rebooting the machine if some
processes are using files from the previous version and have a lock on them.
3. To confirm the upgrade was successful, check the install.log for detailed setup information. The log
directory is %Programfiles%\Microsoft Dependency Agent\logs.
Next steps
If you want to stop monitoring your VMs for a period of time or remove Azure Monitor for VMs entirely, see
Disable monitoring of your VMs in Azure Monitor for VMs.
Disable monitoring of your VMs in Azure Monitor for
VMs (preview)
12/13/2019 • 3 minutes to read • Edit Online
After you enable monitoring of your virtual machines (VMs), you can later choose to disable monitoring in Azure
Monitor for VMs. This article shows how to disable monitoring for one or more VMs.
Currently, Azure Monitor for VMs doesn't support selective disabling of VM monitoring. Your Log Analytics
workspace might support Azure Monitor for VMs and other solutions. It might also collect other monitoring data.
If your Log Analytics workspace provides these services, you need to understand the effect and methods of
disabling monitoring before you start.
Azure Monitor for VMs relies on the following components to deliver its experience:
A Log Analytics workspace, which stores monitoring data from VMs and other sources.
A collection of performance counters configured in the workspace. The collection updates the monitoring
configuration on all VMs connected to the workspace.
InfrastructureInsights and ServiceMap , which are monitoring solutions configured in the workspace. These
solutions update the monitoring configuration on all VMs connected to the workspace.
MicrosoftMonitoringAgent and DependencyAgent , which are Azure VM extensions. These extensions collect and
send data to the workspace.
As you prepare to disable monitoring of your VMs, keep these considerations in mind:
If you evaluated with a single VM and used the preselected default Log Analytics workspace, you can disable
monitoring by uninstalling the Dependency agent from the VM and disconnecting the Log Analytics agent from
this workspace. This approach is appropriate if you intend to use the VM for other purposes and decide later to
reconnect it to a different workspace.
If you selected a preexisting Log Analytics workspace that supports other monitoring solutions and data
collection from other sources, you can remove solution components from the workspace without interrupting
or affecting your workspace.
NOTE
After removing the solution components from your workspace, you might continue to see health state from your Azure VMs;
specifically, you'll see performance and map data when you go to either view in the portal. Data will eventually stop
appearing in the Performance and Map views. But the Health view will continue to show health status for your VMs. The
Try now option will be available from the selected Azure VM so you can re-enable monitoring in the future.
NOTE
If you used the Service Map monitoring solution before you enabled Azure Monitor for VMs and you still rely on it, don't
remove that solution as described in the last step of the following procedure.
1. Sign in to the Azure portal.
2. In the Azure portal, select All services. In the list of resources, type Log Analytics. As you begin typing, the list
filters suggestions based on your input. Select Log Analytics.
3. In your list of Log Analytics workspaces, select the workspace you chose when you enabled Azure Monitor for
VMs.
4. On the left, select Solutions.
5. In the list of solutions, select InfrastructureInsights(workspace name). On the Overview page for the
solution, select Delete. When prompted to confirm, select Yes.
6. In the list of solutions, select ServiceMap(workspace name). On the Overview page for the solution, select
Delete. When prompted to confirm, select Yes.
Before you enabled Azure Monitor for VMs, if you didn't collect performance counters for the Windows-based or
Linux-based VMs in your workspace, disable those rules for Windows and for Linux.
NOTE
Don't remove the Log Analytics agent if:
Azure Automation manages the VM to orchestrate processes or to manage configuration or updates.
Azure Security Center manages the VM for security and threat detection.
If you do remove the Log Analytics agent, you will prevent those services and solutions from proactively managing your VM.
NOTE
The Capacity and Performance solution has been deprecated. Customers who have already installed the solution can
continue to use it, but Capacity and Performance can not be added to any new workspaces.
You can use the Capacity and Performance solution in Monitor to help you understand the capacity of your Hyper-
V servers. The solution provides insights into your Hyper-V environment by showing you the overall utilization
(CPU, memory, and disk) of the hosts and the VMs running on those Hyper-V hosts. Metrics are collected for CPU,
memory, and disks across all your hosts and the VMs running on them.
The solution:
Shows hosts with highest and lowest CPU and memory utilization
Shows VMs with highest and lowest CPU and memory utilization
Shows VMs with highest and lowest IOPS and throughput utilization
Shows which VMs are running on which hosts
Shows the top disks with high throughput, IOPS, and latency in cluster shared volumes
Allows you to customize and filter based on groups
NOTE
The previous version of the Capacity and Performance solution called Capacity Management required both System Center
Operations Manager and System Center Virtual Machine Manager. This updated solution doesn't have those dependencies.
Connected sources
The following table describes the connected sources that are supported by this solution.
Prerequisites
Windows or Operations Manager agents must be installed on Windows Server 2012 or higher Hyper-V hosts,
not virtual machines.
Configuration
Perform the following step to add the Capacity and Performance solution to your workspace.
Add the Capacity and Performance solution to your Log Analytics workspace using the process described in
Add Log Analytics solutions from the Solutions Gallery.
Management packs
If your SCOM management group is connected to your Log Analytics workspace, then the following management
packs will be installed in SCOM when you add this solution. There is no configuration or maintenance of these
management packs required.
Microsoft.IntelligencePacks.CapacityPerformance
The 1201 event resembles:
When the Capacity and Performance solution is updated, the version number will change.
For more information on how solution management packs are updated, see Connect Operations Manager to Log
Analytics.
Evaluate performance
Production computing environments differ greatly from one organization to another. Also, capacity and
performance workloads might depend on how your VMs are running, and what you consider normal. Specific
procedures to help you measure performance would probably not apply to your environment. So, more
generalized prescriptive guidance is better suited to help. Microsoft publishes a variety of prescriptive guidance
articles to help you measure performance.
To summarize, the solution collects capacity and performance data from a variety of sources including performance
counters. Use that capacity and performance data that presented in various surfaces in the solution and compare
your results to those at the Measuring Performance on Hyper-V article. Although the article was published some
time ago, the metrics, considerations, and guidelines are still valid. The article contains links to other useful
resources.
All host memory configurations Perf | where ObjectName == "Capacity and Performance" and
CounterName == "Host Assigned Memory MB" | summarize
MB = avg(CounterValue) by InstanceName
All VM memory configurations Perf | where ObjectName == "Capacity and Performance" and
CounterName == "VM Assigned Memory MB" | summarize
MB = avg(CounterValue) by InstanceName
Breakdown of Total Disk IOPS across all VMs Perf | where ObjectName == "Capacity and Performance" and
(CounterName == "VHD Reads/s" or CounterName == "VHD
Writes/s") | summarize AggregatedValue = avg(CounterValue)
by bin(TimeGenerated, 1h), CounterName, InstanceName
Breakdown of Total Disk Throughput across all VMs Perf | where ObjectName == "Capacity and Performance" and
(CounterName == "VHD Read MB/s" or CounterName ==
"VHD Write MB/s") | summarize AggregatedValue =
avg(CounterValue) by bin(TimeGenerated, 1h), CounterName,
InstanceName
Breakdown of Total IOPS across all CSVs Perf | where ObjectName == "Capacity and Performance" and
(CounterName == "CSV Reads/s" or CounterName == "CSV
Writes/s") | summarize AggregatedValue = avg(CounterValue)
by bin(TimeGenerated, 1h), CounterName, InstanceName
Breakdown of Total Throughput across all CSVs Perf | where ObjectName == "Capacity and Performance" and
(CounterName == "CSV Reads/s" or CounterName == "CSV
Writes/s") | summarize AggregatedValue = avg(CounterValue)
by bin(TimeGenerated, 1h), CounterName, InstanceName
Breakdown of Total Latency across all CSVs Perf | where ObjectName == "Capacity and Performance" and
(CounterName == "CSV Read Latency" or CounterName ==
"CSV Write Latency") | summarize AggregatedValue =
avg(CounterValue) by bin(TimeGenerated, 1h), CounterName,
InstanceName
Next steps
Use Log searches in Log Analytics to view detailed Capacity and Performance data.
VMware Monitoring (Deprecated) solution in Azure
Monitor
12/13/2019 • 8 minutes to read • Edit Online
NOTE
The VMware Monitoring solution has been deprecated. Customers who have already installed the solution can continue to
use it, but VMware Monitoring can not be added to any new workspaces.
The VMware Monitoring solution in Azure Monitor is a solution that helps you create a centralized logging and
monitoring approach for large VMware logs. This article describes how you can troubleshoot, capture, and
manage the ESXi hosts in a single location using the solution. With the solution, you can see detailed data for all
your ESXi hosts in a single location. You can see top event counts, status, and trends of VM and ESXi hosts
provided through the ESXi host logs. You can troubleshoot by viewing and searching centralized ESXi host logs.
And, you can create alerts based on log search queries.
The solution uses native syslog functionality of the ESXi host to push data to a target VM, which has the Log
Analytics agent. However, the solution doesn't write files into syslog within the target VM. The Log Analytics agent
opens port 1514 and listens to this. Once it receives the data, the Log Analytics agent pushes the data into Azure
Monitor.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
Configure syslog collection
1. Set up syslog forwarding for VSphere. For detailed information to help set up syslog forwarding, see
Configuring syslog on ESXi 5.0 and higher (2003322). Go to ESXi Host Configuration > Software >
Advanced Settings > Syslog.
2. In the Syslog.global.logHost field, add your Linux server and the port number 1514. For example,
tcp://hostname:1514 or tcp://123.456.789.101:1514
3. Open the ESXi host firewall for syslog. ESXi Host Configuration > Software > Security Profile >
Firewall and open Properties.
4. Check the vSphere Console to verify that syslog is properly set up. Confirm on the ESXI host that port 1514
is configured.
5. Download and install the Log Analytics agent for Linux on the Linux server. For more information, see the
Documentation for Log Analytics agent for Linux.
6. After the Log Analytics agent for Linux is installed, go to the
/etc/opt/microsoft/omsagent/sysconf/omsagent.d directory and copy the vmware_esxi.conf file to the
/etc/opt/microsoft/omsagent/conf/omsagent.d directory and the change the owner/group and permissions
of the file. For example:
sudo cp /etc/opt/microsoft/omsagent/sysconf/omsagent.d/vmware_esxi.conf
/etc/opt/microsoft/omsagent/conf/omsagent.d
sudo chown omsagent:omiusers /etc/opt/microsoft/omsagent/conf/omsagent.d/vmware_esxi.conf
9. In the Azure portal, perform a log query for VMware_CL . When Azure Monitor collects the syslog data, it
retains the syslog format. In the portal, some specific fields are captured, such as Hostname and
ProcessName.
If your view log search results are similar to the image above, you're set to use the VMware Monitoring
solution dashboard.
Linux • every 3
minutes
The following table show examples of data fields collected by the VMware Monitoring solution:
ResourceLocation_s VMware
ResourceName_s VMware
ResourceType_s Hyper-V
VMName_s VM name
If this is not successful, vSphere settings in the Advanced Configuration are likely not correct. See
Configure syslog collection for information about how to set up the ESXi host for syslog forwarding.
2. If syslog port connectivity is successful, but you don't still see any data, then reload the syslog on the
ESXi host by using ssh to run the following command: esxcli system syslog reload
The VM with Log Analytics agent is not set correctly. To test this, perform the following steps:
1. Log Analytics listens to the port 1514. To verify that it is open, run the following command:
netstat -a | grep 1514
2. You should see port 1514/tcp open. If you do not, verify that the omsagent is installed correctly. If you
do not see the port information, then the syslog port is not open on the VM.
a. Verify that the Log Analytics agent is running by using ps -ef | grep oms . If it is not running, start the
process by running the command sudo /opt/microsoft/omsagent/bin/service_control start
b. Open the /etc/opt/microsoft/omsagent/conf/omsagent.d/vmware_esxi.conf file.
c. Verify that the proper user and group setting is valid, similar to:
-rw-r--r-- 1 omsagent omiusers 677 Sep 20 16:46 vmware_esxi.conf
d. If the file does not exist or the user and group setting is wrong, take corrective action by Preparing a Linux
server.
Next steps
Use log queries in Log Analytics to view detailed VMware host data.
Create your own dashboards showing VMware host data.
Create alerts when specific VMware host events occur.
Wire Data 2.0 (Preview) solution in Azure Monitor
12/13/2019 • 15 minutes to read • Edit Online
Wire data is consolidated network and performance data collected from Windows-connected and Linux-connected
computers with the Log Analytics agent, including those monitored by Operations Manager in your environment.
Network data is combined with your other log data to help you correlate data.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log
Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology
to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
In addition to the Log Analytics agent, the Wire Data solution uses Microsoft Dependency Agents that you install
on computers in your IT infrastructure. Dependency Agents monitor network data sent to and from your
computers for network levels 2-3 in the OSI model, including the various protocols and ports used. Data is then
sent to Azure Monitor using agents.
NOTE
If you have already deployed Service Map, or are considering Service Map or Azure Monitor for VMs, there is a new
connection metrics data set they collect and store in Azure Monitor that provides comparable information to Wire Data.
By default, Azure Monitor logs data for CPU, memory, disk, and network performance data from counters built
into Windows and Linux, as well as other performance counters that you can specify. Network and other data
collection is done in real-time for each agent, including subnets and application-level protocols being used by the
computer. Wire Data looks at network data at the application level, not down at the TCP transport layer. The
solution doesn't look at individual ACKs and SYNs. Once the handshake is completed, it is considered a live
connection and marked as Connected. That connection stays live as long as both sides agree the socket is open
and data can pass back and forth. Once either side closes the connection, it is marked as Disconnected. Therefore,
it only counts the bandwidth of successfully completed packets, it doesn't report on resends or failed packets.
If you've used sFlow or other software with Cisco's NetFlow protocol, then the statistics and data you see from
wire data will be familiar to you.
Some of the types of built-in Log search queries include:
Agents that provide wire data
IP address of agents providing wire data
Outbound communications by IP addresses
Number of bytes sent by application protocols
Number of bytes sent by an application service
Bytes received by different protocols
Total bytes sent and received by IP version
Average latency for connections that were measured reliably
Computer processes that initiated or received network traffic
Amount of network traffic for a process
When you search using wire data, you can filter and group data to view information about the top agents and top
protocols. Or you can view when certain computers (IP addresses/MAC addresses) communicated with each other,
for how long, and how much data was sent—basically, you view metadata about network traffic, which is search-
based.
However, since you're viewing metadata, it's not necessarily useful for in-depth troubleshooting. Wire data in
Azure Monitor is not a full capture of network data. It is not intended for deep packet-level troubleshooting. The
advantage of using the agent, compared to other collection methods, is that you don't have to install appliances,
reconfigure your network switches, or perform complicated configurations. Wire data is simply agent-based—you
install the agent on a computer and it will monitor its own network traffic. Another advantage is when you want to
monitor workloads running in cloud providers or hosting service provider or Microsoft Azure, where the user
doesn't own the fabric layer.
Connected sources
Wire Data gets its data from the Microsoft Dependency Agent. The Dependency Agent depends on the Log
Analytics agent for its connections to Azure Monitor. This means that a server must have the Log Analytics agent
installed and configured with the Dependency agent. The following table describes the connected sources that the
Wire Data solution supports.
System Center Operations Manager Yes Wire Data analyzes and collects data
management group from Windows and Linux agents in a
connected System Center Operations
Manager management group.
If you are a System Center Operations Manager user with a management group connected to Azure Monitor:
No additional configuration is required when your System Center Operations Manager agents can access the
internet to connect to Azure Monitor.
You need to configure the Log Analytics gateway to work with System Center Operations Manager when your
System Center Operations Manager agents cannot access Azure Monitor over the internet.
If your Windows or Linux computers cannot directly connect to the service, you need to configure the Log
Analytics agent to connect to Azure Monitor using the Log Analytics gateway. You can download the Log Analytics
gateway from the Microsoft Download Center.
Prerequisites
Requires the Insight and Analytics solution offer.
If you're using the previous version of the Wire Data solution, you must first remove it. However, all data
captured through the original Wire Data solution is still available in Wire Data 2.0 and log search.
Administrator privileges are required to install or uninstall the Dependency agent.
The Dependency agent must be installed on a computer with a 64-bit operating system.
Operating systems
The following sections list the supported operating systems for the Dependency agent. Wire Data doesn't support
32-bit architectures for any operating system.
Windows Server
Windows Server 2019
Windows Server 2016 1803
Windows Server 2016
Windows Server 2012 R2
Windows Server 2012
Windows Server 2008 R2 SP1
Windows desktop
Windows 10 1803
Windows 10
Windows 8.1
Windows 8
Windows 7
Supported Linux operating systems
The following sections list the supported operating systems for the Dependency agent on Linux.
Only default and SMP Linux kernel releases are supported.
Nonstandard kernel releases, such as PAE and Xen, are not supported for any Linux distribution. For example, a
system with the release string of "2.6.16.21-0.8-xen" is not supported.
Custom kernels, including recompiles of standard kernels, are not supported.
Red Hat Li n u x 7
7.4 3.10.0-693
7.5 3.10.0-862
7.6 3.10.0-957
Red Hat Li n u x 6
6.9 2.6.32-696
6.10 2.6.32-754
C e n t O SP l u s
6.9 2.6.32-696.18.7
2.6.32-696.30.1
6.10 2.6.32-696.30.1
2.6.32-754.3.5
U b u n t u Se r v e r
16.04 4.4.*
4.8.*
4.10.*
4.11.*
4.13.*
14.04 3.13.*
4.4.*
SU SE L i n u x 1 1 En t e r p r i se Se r v e r
11 SP4 3.0.*
SU SE L i n u x 1 2 En t e r p r i se Se r v e r
12 SP2 4.4.*
12 SP3 4.4.*
Configuration
Perform the following steps to configure the Wire Data solution for your workspaces.
1. Enable the Activity Log Analytics solution from the Azure marketplace or by using the process described in Add
monitoring solutions from the Solutions Gallery.
2. Install the Dependency agent on each computer where you want to get data. The Dependency agent can
monitor connections to immediate neighbors, so you might not need an agent on every computer.
NOTE
You cannot add the previous version of the Wire Data solution to new workspaces. If you have the original Wire Data
solution enabled, you can continue to use it. However, to use Wire Data 2.0, you must first remove the original version.
Install the Dependency agent on Windows
Administrator privileges are required to install or uninstall the agent.
The Dependency agent is installed on computers running Windows through InstallDependencyAgent-
Windows.exe. If you run this executable file without any options, it starts a wizard that you can follow to install
interactively.
Use the following steps to install the Dependency agent on each computer running Windows:
1. Install the Log Analytics agent following the steps in Collect data from Windows computers hosted in your
environment.
2. Download the Windows Dependency agent using the link in the previous section and then run it by using the
following command: InstallDependencyAgent-Windows.exe
3. Follow the wizard to install the agent.
4. If the Dependency agent fails to start, check the logs for detailed error information. For Windows agents, the
log directory is %Programfiles%\Microsoft Dependency Agent\logs.
Windows command line
Use options from the following table to install from a command line. To see a list of the installation flags, run the
installer by using the /? flag as follows.
InstallDependencyAgent-Windows.exe /?
FLAG DESCRIPTION
Files for the Windows Dependency agent are placed in C:\Program Files\Microsoft Dependency agent by default.
Install the Dependency agent on Linux
Root access is required to install or configure the agent.
The Dependency agent is installed on Linux computers through InstallDependencyAgent-Linux64.bin, a shell
script with a self-extracting binary. You can run the file by using sh or add execute permissions to the file itself.
Use the following steps to install the Dependency agent on each Linux computer:
1. Install the Log Analytics agent following the steps in Collect data from Linux computers hosted in your
environment.
2. Download the Linux Dependency agent using the link in the previous section and then install it as root by using
the following command: sh InstallDependencyAgent-Linux64.bin
3. If the Dependency agent fails to start, check the logs for detailed error information. On Linux agents, the log
directory is: /var/opt/microsoft/dependency-agent/log.
To see a list of the installation flags, run the installation program with the -help flag as follows.
InstallDependencyAgent-Linux64.bin -help
FLAG DESCRIPTION
Files for the Dependency agent are placed in the following directories:
FILES LOCATION
/opt/microsoft/dependency-agent/bin/microsoft-dependency-
agent-manager
.\InstallDependencyAgent-Windows.exe /S
sh InstallDependencyAgent-Linux64.bin -s
$DAPackageLocalPath = "C:\InstallDependencyAgent-Windows.exe"
Node $NodeName
xRemoteFile DAPackage
Uri = "https://aka.ms/dependencyagentwindows"
DestinationPath = $DAPackageLocalPath
DependsOn = "[Package]OI"
xPackage DA
Ensure = "Present"
Path = $DAPackageLocalPath
Arguments = '/S'
ProductId = ""
InstalledCheckRegKey =
"HKEY\_LOCAL\_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\DependencyAgent"
InstalledCheckRegValueName = "DisplayName"
Management packs
When Wire Data is activated in a Log Analytics workspace, a 300-KB management pack is sent to all the Windows
servers in that workspace. If you are using System Center Operations Manager agents in a connected
management group, the Dependency Monitor management pack is deployed from System Center Operations
Manager. If the agents are directly connected, Azure Monitor delivers the management pack.
The management pack is named Microsoft.IntelligencePacks.ApplicationDependencyMonitor. It's written to:
%Programfiles%\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs. The data source
that the management pack uses is: %Program files%\Microsoft Monitoring Agent\Agent\Health Service
State\Resources<AutoGeneratedID>\Microsoft.EnterpriseManagement.Advisor.ApplicationDependencyMonitorD
ataSource.dll.
BLADE DESCRIPTION
BLADE DESCRIPTION
Agents capturing network traffic Shows the number of agents that are capturing network
traffic and lists the top 10 computers that are capturing traffic.
Click the number to run a log search for
WireData | summarize sum(TotalBytes) by Computer |
take 500000
. Click a computer in the list to run a log search returning the
total number of bytes captured.
Local Subnets Shows the number of local subnets that agents have
discovered. Click the number to run a log search for
WireData | summarize sum(TotalBytes) by LocalSubnet
that lists all subnets with the number of bytes sent over each
one. Click a subnet in the list to run a log search returning the
total number of bytes sent over the subnet.
You can use the Agents capturing network traffic blade to determine how much network bandwidth is being
consumed by computers. This blade can help you easily find the chattiest computer in your environment. Such
computers could be overloaded, acting abnormally, or using more network resources than normal.
Similarly, you can use the Local Subnets blade to determine how much network traffic is moving through your
subnets. Users often define subnets around critical areas for their applications. This blade offers a view into those
areas.
The Application-level Protocols blade is useful because it's helpful know what protocols are in use. For
example, you might expect SSH to not be in use in your network environment. Viewing information available in
the blade can quickly confirm or disprove your expectation.
It's also useful to know if protocol traffic is increasing or decreasing over time. For example, if the amount of data
being transmitted by an application is increasing, that might be something you should be aware of, or that you
might find noteworthy.
Input data
Wire data collects metadata about network traffic using the agents that you have enabled. Each agent sends data
about every 15 seconds.
Output data
A record with a type of WireData is created for each type of input data. WireData records have properties shown
in the following table:
PROPERTY DESCRIPTION
IPVersion IP version
Next steps
Search logs to view detailed wire data search records.
Using Service Map solution in Azure
1/17/2020 • 27 minutes to read • Edit Online
Service Map automatically discovers application components on Windows and Linux systems and maps the
communication between services. With Service Map, you can view your servers in the way that you think of
them: as interconnected systems that deliver critical services. Service Map shows connections between servers,
processes, inbound and outbound connection latency, and ports across any TCP -connected architecture, with no
configuration required other than the installation of an agent.
This article describes the details of onboarding and using Service Map. For information about configuring the
prerequisites for this solution, see Enable the Azure Monitor for VMs overview. To summarize, you need the
following:
A Log Analytics workspace to enable this solution.
The Log Analytics agent installed on the Windows computer or Linux server configured to report the
same workspace that you enabled the solution with.
The Dependency agent installed on the Windows computer or Linux server.
NOTE
If you have already deployed Service Map, you can now also view your maps in Azure Monitor for VMs, which includes
additional features to monitor VM health and performance. To learn more, see Azure Monitor for VMs overview. To learn
about the differences between the Service Map solution and Azure Monitor for VMs Map feature, see the following FAQ.
Sign in to Azure
Sign in to the Azure portal at https://portal.azure.com.
Mapping overview
Service Map agents gather information about all TCP -connected processes on the server where they’re installed
and details about the inbound and outbound connections for each process.
From the list in the left pane, you can select machines or groups that have Service Map agents to visualize their
dependencies over a specified time range. Machine dependency maps focus on a specific machine, and they show
all the machines that are direct TCP clients or servers of that machine. Machine Group maps show sets of servers
and their dependencies.
Machines can be expanded in the map to show the running process groups and processes with active network
connections during the selected time range. When a remote machine with a Service Map agent is expanded to
show process details, only those processes that communicate with the focus machine are shown. The count of
agentless front-end machines that connect into the focus machine is indicated on the left side of the processes
they connect to. If the focus machine is making a connection to a back-end machine that has no agent, the back-
end server is included in a Server Port Group, along with other connections to the same port number.
By default, Service Map maps show the last 30 minutes of dependency information. By using the time controls at
the upper left, you can query maps for historical time ranges of up to one hour to show how dependencies
looked in the past (for example, during an incident or before a change occurred). Service Map data is stored for
30 days in paid workspaces, and for 7 days in free workspaces.
Machine Groups
Machine Groups allow you to see maps centered around a set of servers, not just one so you can see all the
members of a multi-tier application or server cluster in one map.
Users select which servers belong in a group together and choose a name for the group. You can then choose to
view the group with all of its processes and connections, or view it with only the processes and connections that
directly relate to the other members of the group.
Creating a Machine Group
To create a group, select the machine or machines you want in the Machines list and click Add to group.
There, you can choose Create new and give the group a name.
NOTE
Machine groups are limited to 10 servers.
Viewing a Group
Once you’ve created some groups, you can view them by choosing the Groups tab.
Then select the Group name to view the map for that Machine Group.
The machines that belong to the group are outlined in white in the map.
Expanding the Group will list the machines that make up the Machine Group.
Filter by processes
You can toggle the map view between showing all processes and connections in the Group and only the ones that
directly relate to the Machine Group. The default view is to show all processes. You can change the view by
clicking the filter icon above the map.
When All processes is selected, the map will include all processes and connections on each of the machines in
the Group.
If you change the view to show only group-connected processes, the map will be narrowed down to only those
processes and connections that are directly connected to other machines in the group, creating a simplified view.
Role icons
Certain processes serve particular roles on machines: web servers, application servers, database, and so on.
Service Map annotates process and machine boxes with role icons to help identify at a glance the role a process
or server plays.
Web server
Application server
Database server
LDAP server
ROLE ICON DESCRIPTION
SMB server
Failed connections
Failed connections are shown in Service Map maps for processes and computers, with a dashed red line
indicating that a client system is failing to reach a process or port. Failed connections are reported from any
system with a deployed Service Map agent if that system is the one attempting the failed connection. Service
Map measures this process by observing TCP sockets that fail to establish a connection. This failure could result
from a firewall, a misconfiguration in the client or server, or a remote service being unavailable.
Understanding failed connections can help with troubleshooting, migration validation, security analysis, and
overall architectural understanding. Failed connections are sometimes harmless, but they often point directly to a
problem, such as a failover environment suddenly becoming unreachable, or two application tiers being unable
to talk after a cloud migration.
Client Groups
Client Groups are boxes on the map that represent client machines that do not have Dependency Agents. A
single Client Group represents the clients for an individual process or machine.
To see the IP addresses of the servers in a Client Group, select the group. The contents of the group are listed in
the Client Group Properties pane.
Server Port Groups
Server Port Groups are boxes that represent server ports on servers that do not have Dependency Agents. The
box contains the server port and a count of the number of servers with connections to that port. Expand the box
to see the individual servers and connections. If there is only one server in the box, the name or IP address is
listed.
Context menu
Clicking the ellipsis (...) at the top right of any server displays the context menu for that server.
Computer summary
The Machine Summary pane includes an overview of a server's operating system, dependency counts, and data
from other solutions. Such data includes performance metrics, service desk tickets, change tracking, security, and
updates.
Alerts integration
Service Map integrates with Azure Alerts to show fired alerts for the selected server in the selected time range.
The server displays an icon if there are current alerts, and the Machine Alerts pane lists the alerts.
To enable Service Map to display relevant alerts, create an alert rule that fires for a specific computer. To create
proper alerts:
Include a clause to group by computer (for example, by Computer interval 1 minute).
Choose to alert based on metric measurement.
Security integration
Service Map integration with Security and Audit is automatic when both solutions are enabled and configured in
your Log Analytics workspace.
The Machine Security pane shows data from the Security and Audit solution for the selected server. The pane
lists a summary of any outstanding security issues for the server during the selected time range. Clicking any of
the security issues drills down into a Log Search for details about them.
Updates integration
Service Map integration with Update Management is automatic when both solutions are enabled and configured
in your Log Analytics workspace.
The Machine Updates pane displays data from the Update Management solution for the selected server. The
pane lists a summary of any missing updates for the server during the selected time range.
Log Analytics records
Service Map computer and process inventory data is available for search in Log Analytics. You can apply this data
to scenarios that include migration planning, capacity analysis, discovery, and on-demand performance
troubleshooting.
One record is generated per hour for each unique computer and process, in addition to the records that are
generated when a process or computer starts or is on-boarded to Service Map. These records have the
properties in the following tables. The fields and values in the ServiceMapComputer_CL events map to fields of
the Machine resource in the ServiceMap Azure Resource Manager API. The fields and values in the
ServiceMapProcess_CL events map to the fields of the Process resource in the ServiceMap Azure Resource
Manager API. The ResourceName_s field matches the name field in the corresponding Resource Manager
resource.
NOTE
As Service Map features grow, these fields are subject to change.
There are internally generated properties you can use to identify unique processes and computers:
Computer: Use ResourceId or ResourceName_s to uniquely identify a computer within a Log Analytics
workspace.
Process: Use ResourceId to uniquely identify a process within a Log Analytics workspace. ResourceName_s is
unique within the context of the machine on which the process is running (MachineResourceName_s)
Because multiple records can exist for a specified process and computer in a specified time range, queries can
return more than one record for the same computer or process. To include only the most recent record, add "|
dedup ResourceId" to the query.
Connections
Connection metrics are written to a new table in Log Analytics - VMConnection. This table provides information
about the connections for a machine (inbound and outbound). Connection Metrics are also exposed with APIs
that provide the means to obtain a specific metric during a time window. TCP connections resulting from
accepting on a listening socket are inbound, while those created by connecting to a given IP and port are
outbound. The direction of a connection is represented by the Direction property, which can be set to either
inbound or outbound.
Records in these tables are generated from data reported by the Dependency agent. Every record represents an
observation over a one minute time interval. The TimeGenerated property indicates the start of the time interval.
Each record contains information to identify the respective entity, that is, connection or port, as well as metrics
associated with that entity. Currently, only network activity that occurs using TCP over IPv4 is reported.
To manage cost and complexity, connection records do not represent individual physical network connections.
Multiple physical network connections are grouped into a logical connection, which is then reflected in the
respective table. Meaning, records in VMConnection table represent a logical grouping and not the individual
physical connections that are being observed. Physical network connection sharing the same value for the
following attributes during a given one minute interval, are aggregated into a single logical record in
VMConnection.
PROPERTY DESCRIPTION
To account for the impact of grouping, information about the number of grouped physical connections is
provided in the following properties of the record:
PROPERTY DESCRIPTION
Metrics
In addition to connection count metrics, information about the volume of data sent and received on a given
logical connection or network port are also included in the following properties of the record:
PROPERTY DESCRIPTION
BytesSent Total number of bytes that have been sent during the
reporting time window
BytesReceived Total number of bytes that have been received during the
reporting time window
The third type of data being reported is response time - how long does a caller spend waiting for a request sent
over a connection to be processed and responded to by the remote endpoint. The response time reported is an
estimation of the true response time of the underlying application protocol. It is computed using heuristics based
on the observation of the flow of data between the source and destination end of a physical network connection.
Conceptually, it is the difference between the time the last byte of a request leaves the sender, and the time when
the last byte of the response arrives back to it. These two timestamps are used to delineate request and response
events on a given physical connection. The difference between them represents the response time of a single
request.
In this first release of this feature, our algorithm is an approximation that may work with varying degree of
success depending on the actual application protocol used for a given network connection. For example, the
current approach works well for request-response based protocols such as HTTP (S ), but does not work with one-
way or message queue-based protocols.
Here are some important points to consider:
1. If a process accepts connections on the same IP address but over multiple network interfaces, a separate
record for each interface will be reported.
2. Records with wildcard IP will contain no activity. They are included to represent the fact that a port on the
machine is open to inbound traffic.
3. To reduce verbosity and data volume, records with wildcard IP will be omitted when there is a matching
record (for the same process, port, and protocol) with a specific IP address. When a wildcard IP record is
omitted, the IsWildcardBind record property with the specific IP address, will be set to "True" to indicate that
the port is exposed over every interface of the reporting machine.
4. Ports that are bound only on a specific interface have IsWildcardBind set to "False".
Naming and Classification
For convenience, the IP address of the remote end of a connection is included in the RemoteIp property. For
inbound connections, RemoteIp is the same as SourceIp, while for outbound connections, it is the same as
DestinationIp. The RemoteDnsCanonicalNames property represents the DNS canonical names reported by the
machine for RemoteIp. The RemoteDnsQuestions and RemoteClassification properties are reserved for future
use.
Geolocation
VMConnection also includes geolocation information for the remote end of each connection record in the
following properties of the record:
PROPERTY DESCRIPTION
Malicious IP
Every RemoteIp property in VMConnection table is checked against a set of IPs with known malicious activity. If
the RemoteIp is identified as malicious the following properties will be populated (they are empty, when the IP is
not considered malicious) in the following properties of the record:
PROPERTY DESCRIPTION
TLPLevel Traffic Light Protocol (TLP) Level is one of the defined values,
White, Green, Amber, Red.
ServiceMapComputer_CL records
Records with a type of ServiceMapComputer_CL have inventory data for servers with Service Map agents. These
records have the properties in the following table:
PROPERTY DESCRIPTION
Type ServiceMapComputer_CL
SourceSystem OpsManager
PROPERTY DESCRIPTION
Type ServiceMapProcess_CL
SourceSystem OpsManager
REST API
All the server, process, and dependency data in Service Map is available via the Service Map REST API.
Next steps
Learn more about log searches in Log Analytics to retrieve data that's collected by Service Map.
Troubleshooting
If you have any problems installing or running Service Map, this section can help you. If you still can't resolve
your problem, please contact Microsoft Support.
Dependency agent installation problems
Installer prompts for a reboot
The Dependency agent generally does not require a reboot upon installation or removal. However, in certain rare
cases, Windows Server requires a reboot to continue with an installation. This happens when a dependency,
usually the Microsoft Visual C++ Redistributable library requires a reboot because of a locked file.
Message "Unable to install Dependency agent: Visual Studio Runtime libraries failed to install (code = [code_number])" appears
The Microsoft Dependency agent is built on the Microsoft Visual Studio runtime libraries. You'll get a message if
there's a problem during installation of the libraries.
The runtime library installers create logs in the %LOCALAPPDATA%\temp folder. The file is
dd_vcredist_arch_yyyymmddhhmmss.log , where arch is x86 or amd64 and yyyymmddhhmmss is the date and time
(24-hour clock) when the log was created. The log provides details about the problem that's blocking installation.
It might be useful to install the latest runtime libraries first.
The following table lists code numbers and suggested resolutions.
0x17 The library installer requires a Windows Look in the most recent library installer
update that hasn't been installed. log.
If a reference to
Windows8.1-KB2999226-x64.msu is
followed by a line
Error 0x80240017: Failed to
execute MSU package,
you don't have the prerequisites to
install KB2999226. Follow the
instructions in the prerequisites section
in Universal C Runtime in Windows
article. You might need to run
Windows Update and reboot multiple
times in order to install the
prerequisites.
Post-installation issues
Server doesn't appear in Service Map
If your Dependency agent installation succeeded, but you don't see your machine in the Service Map solution:
Is the Dependency agent installed successfully? You can validate this by checking to see if the service is
installed and running.
Windows: Look for the service named Microsoft Dependency agent. Linux: Look for the running
process microsoft-dependency-agent.
Are you on the Log Analytics free tier? The Free plan allows for up to five unique Service Map machines.
Any subsequent machines won't appear in Service Map, even if the prior five are no longer sending data.
Is your server sending log and perf data to Azure Monitor Logs? Go to Azure Monitor\Logs and run the
following query for your computer:
Did you get a variety of events in the results? Is the data recent? If so, your Log Analytics agent is operating
correctly and communicating with the workspace. If not, check the agent on your machine: Log Analytics agent
for Windows troubleshooting or Log Analytics agent for Linux troubleshooting.
Server appears in Service Map but has no processes
If you see your machine in Service Map, but it has no process or connection data, that indicates that the
Dependency agent is installed and running, but the kernel driver didn't load.
Check the C:\Program Files\Microsoft Dependency Agent\logs\wrapper.log file (Windows) or
/var/opt/microsoft/dependency-agent/log/service.log file ( Linux). The last lines of the file should indicate why
the kernel didn't load. For example, the kernel might not be supported on Linux if you updated your kernel.
Feedback
Do you have any feedback for us about Service Map or this documentation? Visit our User Voice page, where
you can suggest features or vote up existing suggestions.
Computer groups in Azure Monitor log queries
1/22/2020 • 6 minutes to read • Edit Online
Computer groups in Azure Monitor allow you to scope log queries to a particular set of computers. Each group is
populated with computers either using a query that you define or by importing groups from different sources.
When the group is included in a log query, the results are limited to records that match the computers in the
group.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a
Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the
terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
METHOD DESCRIPTION
Log Search API Use the Log Search API to programmatically create a
computer group based on the results of a log query.
Windows Server Update Services Automatically scan WSUS servers or clients for targeting
groups and create a group in Azure Monitor for each.
Log query
Computer groups created from a log query contain all of the computers returned by a query that you define. This
query is run every time the computer group is used so that any changes since the group was created is reflected.
You can use any query for a computer group, but it must return a distinct set of computers by using
distinct Computer . Following is a typical example query that you could use for as a computer group.
Use the following procedure to create a computer group from a log search in the Azure portal.
1. Click Logs in the Azure Monitor menu in the Azure portal.
2. Create and run a query that returns the computers that you want in the group.
3. Click Save at the top of the screen.
4. Change Save as to Function and select Save this query as a computer group.
5. Provide values for each property for the computer group described in the table and click Save.
The following table describes the properties that define a computer group.
PROPERTY DESCRIPTION
Function alias A unique alias used to identify the computer group in a query.
Active Directory
When you configure Azure Monitor to import Active Directory group memberships, it analyzes the group
membership of any Windows domain joined computers with the Log Analytics agent. A computer group is
created in Azure Monitor for each security group in Active Directory, and each Windows computer is added to the
computer groups corresponding to the security groups they are members of. This membership is continuously
updated every 4 hours.
NOTE
Imported Active Directory groups only contain Windows machines.
You configure Azure Monitor to import Active Directory security groups from Advanced settings in your Log
Analytics workspace in the Azure portal. Select Computer Groups, Active Directory, and then Import Active
Directory group memberships from computers. There is no further configuration required.
When groups have been imported, the menu lists the number of computers with group membership detected and
the number of groups imported. You can click on either of these links to return the ComputerGroup records with
this information.
Windows Server Update Service
When you configure Azure Monitor to import WSUS group memberships, it analyzes the targeting group
membership of any computers with the Log Analytics agent. If you are using client-side targeting, any computer
that is connected to Azure Monitor and is part of any WSUS targeting groups has its group membership
imported to Azure Monitor. If you are using server-side targeting, the Log Analytics agent should be installed on
the WSUS server in order for the group membership information to be imported to Azure Monitor. This
membership is continuously updated every 4 hours.
You configure Azure Monitor to import WSUS groups from Advanced settings in your Log Analytics workspace
in the Azure portal. Select Computer Groups, WSUS, and then Import WSUS group memberships. There is
no further configuration required.
When groups have been imported, the menu lists the number of computers with group membership detected and
the number of groups imported. You can click on either of these links to return the ComputerGroup records with
this information.
Configuration Manager
When you configure Azure Monitor to import Configuration Manager collection memberships, it creates a
computer group for each collection. The collection membership information is retrieved every 3 hours to keep the
computer groups current.
Before you can import Configuration Manager collections, you must connect Configuration Manager to Azure
Monitor.
When collections have been imported, the menu lists the number of computers with group membership detected
and the number of groups imported. You can click on either of these links to return the ComputerGroup records
with this information.
For example, you could use the following to return UpdateSummary records for only computers in a computer
group called mycomputergroup.
UpdateSummary | where Computer in (mycomputergroup)
Imported computer groups and their included computers are stored in the ComputerGroup table. For example,
the following query would return a list of computers in the Domain Computers group from Active Directory.
ComputerGroup | where GroupSource == "ActiveDirectory" and Group == "Domain Computers" | distinct Computer
The following query would return UpdateSummary records for only computers in Domain Computers.
let ADComputers = ComputerGroup | where GroupSource == "ActiveDirectory" and Group == "Domain Computers" |
distinct Computer;
UpdateSummary | where Computer in (ADComputers)
PROPERTY DESCRIPTION
Type ComputerGroup
SourceSystem SourceSystem
GroupFullName Full path to the group including the source and source name.
PROPERTY DESCRIPTION
ActiveDirectory
WSUS
WSUSClientTargeting
GroupSourceName Name of the source that the group was collected from. For
Active Directory, this is the domain name.
ManagementGroupName Name of the management group for SCOM agents. For other
agents, this is AOI-<workspace ID>
TimeGenerated Date and time the computer group was created or updated.
Next steps
Learn about log queries to analyze the data collected from data sources and solutions.
Azure Monitor for containers overview
1/8/2020 • 3 minutes to read • Edit Online
Azure Monitor for containers is a feature designed to monitor the performance of container workloads
deployed to:
Managed Kubernetes clusters hosted on Azure Kubernetes Service (AKS )
Azure Container Instances
Self-managed Kubernetes clusters hosted on Azure Stack or on-premises
Azure Red Hat OpenShift
Azure Monitor for containers supports clusters running the Linux and Windows Server 2019 operating system.
Monitoring your containers is critical, especially when you're running a production cluster, at scale, with multiple
applications.
Azure Monitor for containers gives you performance visibility by collecting memory and processor metrics
from controllers, nodes, and containers that are available in Kubernetes through the Metrics API. Container logs
are also collected. After you enable monitoring from Kubernetes clusters, metrics and logs are automatically
collected for you through a containerized version of the Log Analytics agent for Linux. Metrics are written to the
metrics store and log data is written to the logs store associated with your Log Analytics workspace.
NOTE
Support for Azure Red Hat OpenShift is a feature in public preview at this time.
The main differences in monitoring a Windows Server cluster compared to a Linux cluster are the following:
Memory RSS metric isn't available for Windows node and containers.
Disk storage capacity information isn't available for Windows nodes.
Container logs aren't available for containers running in Windows nodes.
Live Data (preview ) feature support is available with the exception of Windows container logs.
Only pod environments are monitored, not Docker environments.
With the preview release, a maximum of 30 Windows Server containers are supported. This limitation
doesn't apply to Linux containers.
Check out the following video providing an intermediate level deep dive to help you learn about monitoring
your AKS cluster with Azure Monitor for containers.
Next steps
To begin monitoring your Kubernetes cluster, review How to enable the Azure Monitor for containers to
understand the requirements and available methods to enable monitoring.
Azure Monitor Frequently Asked Questions
1/24/2020 • 39 minutes to read • Edit Online
This Microsoft FAQ is a list of commonly asked questions about Azure Monitor.
General
What is Azure Monitor?
Azure Monitor is a service in Azure that provides performance and availability monitoring for applications and
services in Azure, other cloud environments, or on-premises. Azure Monitor collects data from multiple sources
into a common data platform where it can be analyzed for trends and anomalies. Rich features in Azure Monitor
assist you in quickly identifying and responding to critical situations that may affect your application.
What's the difference between Azure Monitor, Log Analytics, and Application Insights?
In September 2018, Microsoft combined Azure Monitor, Log Analytics, and Application Insights into a single
service to provide powerful end-to-end monitoring of your applications and the components they rely on. Features
in Log Analytics and Application Insights have not changed, although some features have been rebranded to Azure
Monitor in order to better reflect their new scope. The log data engine and query language of Log Analytics is now
referred to as Azure Monitor Logs. See Azure Monitor terminology updates.
What does Azure Monitor cost?
Features of Azure Monitor that are automatically enabled such as collection of metrics and activity logs are
provided at no cost. There is a cost associated with other features such as log queries and alerting. See the Azure
Monitor pricing page for detailed pricing information.
How do I enable Azure Monitor?
Azure Monitor is enabled the moment that you create a new Azure subscription, and Activity log and platform
metrics are automatically collected. Create diagnostic settings to collect more detailed information about the
operation of your Azure resources, and add monitoring solutions and insights to provide additional analysis on
collected data for particular services.
How do I access Azure Monitor?
Access all Azure Monitor features and data from the Monitor menu in the Azure portal. The Monitoring section
of the menu for different Azure services provides access to the same tools with data filtered to a particular
resource. Azure Monitor data is also accessible for a variety of scenarios using CLI, PowerShell, and a REST API.
Is there an on-premises version of Azure Monitor?
No. Azure Monitor is a scalable cloud service that processes and stores large amounts of data, although Azure
Monitor can monitor resources that are on-premises and in other clouds.
Can Azure Monitor monitor on-premises resources?
Yes, in addition to collecting monitoring data from Azure resources, Azure Monitor can collect data from virtual
machines and applications in other clouds and on-premises. See Sources of monitoring data for Azure Monitor.
Does Azure Monitor integrate with System Center Operations Manager?
You can connect your existing System Center Operations Manager management group to Azure Monitor to collect
data from agents into Azure Monitor Logs. This allows you to use log queries and solution to analyze data collected
from agents. You can also configure existing System Center Operations Manager agents to send data directly to
Azure Monitor. See Connect Operations Manager to Azure Monitor.
What IP addresses does Azure Monitor use?
See IP addresses used by Application Insights and Log Analytics for a listing of the IP addresses and ports
required for agents and other external resources to access Azure Monitor.
Monitoring data
Where does Azure Monitor get its data?
Azure Monitor collects data from a variety of sources including logs and metrics from Azure platform and
resources, custom applications, and agents running on virtual machines. Other services such as Azure Security
Center and Network Watcher collect data into a Log Analytics workspace so it can be analyzed with Azure Monitor
data. You can also send custom data to Azure Monitor using the REST API for logs or metrics. See Sources of
monitoring data for Azure Monitor.
What data is collected by Azure Monitor?
Azure Monitor collects data from a variety of sources into logs or metrics. Each type of data has its own relative
advantages, and each supports a particular set of features in Azure Monitor. There is a single metrics database for
each Azure subscription, while you can create multiple Log Analytics workspaces to collect logs depending on your
requirements. See Azure Monitor data platform.
Is there a maximum amount of data that I can collect in Azure Monitor?
There is no limit to the amount of metric data you can collect, but this data is stored for a maximum of 93 days. See
Retention of Metrics. There is no limit on the amount of log data that you can collected, but it may be affected by
the pricing tier you choose for the Log Analytics workspace. See pricing details.
How do I access data collected by Azure Monitor?
Insights and solutions provide a custom experience for working with data stored in Azure Monitor. You can work
directly with log data using a log query written in Kusto Query Language (KQL ). In the Azure portal, you can write
and run queries and interactively analyze data using Log Analytics. Analyze metrics in the Azure portal with the
Metrics Explorer. See Analyze log data in Azure Monitor and Getting started with Azure Metrics Explorer.
Logs
What's the difference between Azure Monitor Logs and Azure Data Explorer?
Azure Data Explorer is a fast and highly scalable data exploration service for log and telemetry data. Azure Monitor
Logs is built on top of Azure Data Explorer and uses the same Kusto Query Language (KQL ) with some minor
differences. See Azure Monitor log query language differences.
How do I retrieve log data?
All data is retrieved from a Log Analytics workspace using a log query written using Kusto Query Language (KQL ).
You can write your own queries or use solutions and insights that include log queries for a particular application or
service. See Overview of log queries in Azure Monitor.
What is a Log Analytics workspace?
All log data collected by Azure Monitor is stored in a Log Analytics workspace. A workspace is essentially a
container where log data is collected from a variety of sources. You may have a single Log Analytics workspace for
all your monitoring data or may have requirements for multiple workspaces. See Designing your Azure Monitor
Logs deployment.
Can you move an existing Log Analytics workspace to another Azure subscription?
You can move a workspace between resource groups or subscriptions but not to a different region. See Move a
Log Analytics workspace to different subscription or resource group.
Why can't I see Query Explorer and Save buttons in Log Analytics?
Query Explorer, Save and New alert rule buttons are not available when the query scope is set to a specific
resource. To create alerts, save or load a query, Log Analytics must be scoped to a workspace. To open Log
Analytics in workspace context, select Logs from the Azure Monitor menu. The last used workspace is selected,
but you can select any other workspace. See Log query scope and time range in Azure Monitor Log Analytics
Why am I getting the error: "Register resource provider 'Microsoft.Insights' for this subscription to enable this
query" when opening Log Analytics from a VM?
Many resource providers are automatically registered, but you may need to manually register some resource
providers. The scope for registration is always the subscription. See Resource providers and types for more
information.
Why am I am getting no access error message when opening Log Analytics from a VM?
To view VM Logs, you need to be granted with read permission to the workspaces that stores the VM logs. In these
cases, your administrator must grant you with to permissions in Azure.
Alerts
What is an alert in Azure Monitor?
Alerts proactively notify you when important conditions are found in your monitoring data. They allow you to
identify and address issues before the users of your system notice them. There are multiple kinds of alerts:
Metric - Metric value exceeds a threshold.
Log query - Results of a log query match defined criteria.
Activity log - Activity log event matches defined criteria.
Web test - Results of availability test match defined criteria.
See Overview of alerts in Microsoft Azure.
What is an action group?
An action group is a collection of notifications and actions that can be triggered by an alert. Multiple alerts can use
a single action group allowing you to leverage common sets of notifications and actions. See Create and manage
action groups in the Azure portal.
What is an action rule?
An action rule allows you to modify the behavior of a set of alerts that match a certain criteria. This allows you to to
perform such requirements as disable alert actions during a maintenance window. You can also apply an action
group to a set of alerts rather than applying them directly to the alert rules. See Action rules.
Agents
Does Azure Monitor require an agent?
An agent is only required to collect data from the operating system and workloads in virtual machines. The virtual
machines can be located in Azure, another cloud environment, or on-premises. See Overview of the Azure Monitor
agents.
What's the difference between the Azure Monitor agents?
Azure Diagnostic extension is for Azure virtual machines and collects data to Azure Monitor Metrics, Azure
Storage, and Azure Event Hubs. The Log Analytics agent is for virtual machines in Azure, another cloud
environment, or on-premises and collects data to Azure Monitor Logs. The Dependency agent requires the Log
Analytics agent and collected process details and dependencies. See Overview of the Azure Monitor agents.
Does my agent traffic use my ExpressRoute connection?
Traffic to Azure Monitor uses the Microsoft peering ExpressRoute circuit. See ExpressRoute documentation for a
description of the different types of ExpressRoute traffic.
How can I confirm that the Log Analytics agent is able to communicate with Azure Monitor?
From Control Panel on the agent computer, select Security & Settings, Microsoft Monitoring Agent . Under
the Azure Log Analytics (OMS ) tab, a green check mark icon confirms that the agent is able to communicate
with Azure Monitor. A yellow warning icon means the agent is having issues. One common reason is the
Microsoft Monitoring Agent service has stopped. Use service control manager to restart the service.
How do I stop the Log Analytics agent from communicating with Azure Monitor?
For agents connected to Log Analytics directly, open the Control Panel and select Security & Settings, Microsoft
Monitoring Agent. Under the Azure Log Analytics (OMS ) tab, remove all workspaces listed. In System Center
Operations Manager, remove the computer from the Log Analytics managed computers list. Operations Manager
updates the configuration of the agent to no longer report to Log Analytics.
How much data is sent per agent?
The amount of data sent per agent depends on:
The solutions you have enabled
The number of logs and performance counters being collected
The volume of data in the logs
See Manage usage and costs with Azure Monitor Logs for details.
For computers that are able to run the WireData agent, use the following query to see how much data is being
sent:
WireData
| where ProcessName == "C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost.exe"
| where Direction == "Outbound"
| summarize sum(TotalBytes) by Computer
How much network bandwidth is used by the Microsoft Management Agent (MMA ) when sending data to
Azure Monitor?
Bandwidth is a function on the amount of data sent. Data is compressed as it is sent over the network.
How can I be notified when data collection from the Log Analytics agent stops?
Use the steps described in create a new log alert to be notified when data collection stops. Use the following
settings for the alert rule:
Define alert condition: Specify your Log Analytics workspace as the resource target.
Alert criteria
Signal Name: Custom log search
Search query:
Heartbeat | summarize LastCall = max(TimeGenerated) by Computer | where LastCall < ago(15m)
Alert logic: Based on number of results, Condition Greater than, Threshold value 0
Evaluated based on: Period (in minutes) 30, Frequency (in minutes) 10
Define alert details
Name: Data collection stopped
Severity: Warning
Specify an existing or new Action Group so that when the log alert matches criteria, you are notified if you have a
heartbeat missing for more than 15 minutes.
What are the firewall requirements for Azure Monitor agents?
See Network firewall requirementsfor details on firewall requirements.
Visualizations
Why can't I can’t see View Designer?
View Designer is only available for users assigned with Contributor permissions or higher in the Log Analytics
workspace.
Application Insights
Configuration problems
I'm having trouble setting up my:
.NET app
Monitoring an already-running app
Azure diagnostics
Java web app
I get no data from my server
Set firewall exceptions
Set up an ASP.NET server
Set up a Java server
Can I use Application Insights with ...?
Web apps on an IIS server in Azure VM or Azure virtual machine scale set
Web apps on an IIS server - on-premises or in a VM
Java web apps
Node.js apps
Web apps on Azure
Cloud Services on Azure
App servers running in Docker
Single-page web apps
SharePoint
Windows desktop app
Other platforms
Is it free?
Yes, for experimental use. In the basic pricing plan, your application can send a certain allowance of data each
month free of charge. The free allowance is large enough to cover development, and publishing an app for a small
number of users. You can set a cap to prevent more than a specified amount of data from being processed.
Larger volumes of telemetry are charged by the Gb. We provide some tips on how to limit your charges.
The Enterprise plan incurs a charge for each day that each web server node sends telemetry. It is suitable if you
want to use Continuous Export on a large scale.
Read the pricing plan.
How much does it cost?
Open the Usage and estimated costs page in an Application Insights resource. There's a chart of recent
usage. You can set a data volume cap, if you want.
Open the Azure Billing blade to see your bills across all resources.
What does Application Insights modify in my project?
The details depend on the type of project. For a web application:
Adds these files to your project:
ApplicationInsights.config
ai.js
Installs these NuGet packages:
Application Insights API - the core API
Application Insights API for Web Applications - used to send telemetry from the server
Application Insights API for JavaScript Applications - used to send telemetry from the client
The packages include these assemblies:
Microsoft.ApplicationInsights
Microsoft.ApplicationInsights.Platform
Inserts items into:
Web.config
packages.config
(New projects only - if you add Application Insights to an existing project, you have to do this manually.) Inserts
snippets into the client and server code to initialize them with the Application Insights resource ID. For example,
in an MVC app, code is inserted into the master page Views/Shared/_Layout.cshtml
How do I upgrade from older SDK versions?
See the release notes for the SDK appropriate to your type of application.
How can I change which Azure resource my project sends data to?
In Solution Explorer, right-click ApplicationInsights.config and choose Update Application Insights. You can
send the data to an existing or new resource in Azure. The update wizard changes the instrumentation key in
ApplicationInsights.config, which determines where the server SDK sends your data. Unless you deselect "Update
all," it will also change the key where it appears in your web pages.
What is Status Monitor?
A desktop app that you can use in your IIS web server to help configure Application Insights in web apps. It doesn't
collect telemetry: you can stop it when you are not configuring an app.
Learn more.
What telemetry is collected by Application Insights?
From server web apps:
HTTP requests
Dependencies. Calls to: SQL Databases; HTTP calls to external services; Azure Cosmos DB, table, blob storage,
and queue.
Exceptions and stack traces.
Performance Counters - If you use Status Monitor, Azure monitoring for App Services, Azure monitoring for
VM or virtual machine scale set, or the Application Insights collectd writer.
Custom events and metrics that you code.
Trace logs if you configure the appropriate collector.
From client web pages:
Page view counts
AJAX calls Requests made from a running script.
Page view load data
User and session counts
Authenticated user IDs
From other sources, if you configure them:
Azure diagnostics
Import to Analytics
Log Analytics
Logstash
Can I filter out or modify some telemetry?
Yes, in the server you can write:
Telemetry Processor to filter or add properties to selected telemetry items before they are sent from your app.
Telemetry Initializer to add properties to all items of telemetry.
Learn more for ASP.NET or Java.
How are city, country/region, and other geo location data calculated?
We look up the IP address (IPv4 or IPv6) of the web client using GeoLite2.
Browser telemetry: We collect the sender's IP address.
Server telemetry: The Application Insights module collects the client IP address. It is not collected if
X-Forwarded-For is set.
To learn more about how IP address and geolocation data is collected in Application Insights refer to this article.
You can configure the ClientIpHeaderTelemetryInitializer to take the IP address from a different header. In some
systems, for example, it is moved by a proxy, load balancer, or CDN to X-Originating-IP . Learn more.
You can use Power BI to display your request telemetry on a map.
How long is data retained in the portal? Is it secure?
Take a look at Data Retention and Privacy.
Could personal data be sent in the telemetry?
This is possible if your code sends such data. It can also happen if variables in stack traces include personal data.
Your development team should conduct risk assessments to ensure that personal data is properly handled. Learn
more about data retention and privacy.
All octets of the client web address are always set to 0 after the geo location attributes are looked up.
My Instrumentation Key is visible in my web page source.
This is common practice in monitoring solutions.
It can't be used to steal your data.
It could be used to skew your data or trigger alerts.
We have not heard that any customer has had such problems.
You could:
Use two separate Instrumentation Keys (separate Application Insights resources), for client and server data. Or
Write a proxy that runs in your server, and have the web client send data through that proxy.
How do I see POST data in Diagnostic search?
We don't log POST data automatically, but you can use a TrackTrace call: put the data in the message parameter.
This has a longer size limit than the limits on string properties, though you can't filter on it.
Should I use single or multiple Application Insights resources?
Use a single resource for all the components or roles in a single business system. Use separate resources for
development, test, and release versions, and for independent applications.
See the discussion here
Example - cloud service with worker and web roles
How do I dynamically change the instrumentation key?
Discussion here
Example - cloud service with worker and web roles
What are the User and Session counts?
The JavaScript SDK sets a user cookie on the web client, to identify returning users, and a session cookie to
group activities.
If there is no client-side script, you can set cookies at the server.
If one real user uses your site in different browsers, or using in-private/incognito browsing, or different
machines, then they will be counted more than once.
To identify a logged-in user across machines and browsers, add a call to setAuthenticatedUserContext().
Have I enabled everything in Application Insights?
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Server app perf: response times, ... Add Application Insights to your project Detect perf issues
or Install AI Status Monitor on server
(or write your own code to track
dependencies)
Dependency telemetry Install AI Status Monitor on server Diagnose issues with databases or other
external components
Get stack traces from exceptions Insert TrackException calls in your code Detect and diagnose exceptions
(but some are reported automatically)
Search log traces Add a logging adapter Diagnose exceptions, perf issues
WHAT YOU SHOULD SEE HOW TO GET IT WHY YOU WANT IT
Client usage basics: page views, JavaScript initializer in web pages Usage analytics
sessions, ...
Client custom metrics Tracking calls in web pages Enhance user experience
Automation
Configuring Application Insights
You can write PowerShell scripts using Azure Resource Monitor to:
Create and update Application Insights resources.
Set the pricing plan.
Get the instrumentation key.
Add a metric alert.
Add an availability test.
You can't set up a Metric Explorer report or set up continuous export.
Querying the telemetry
Use the REST API to run Analytics queries.
How can I set an alert on an event?
Azure alerts are only on metrics. Create a custom metric that crosses a value threshold whenever your event
occurs. Then set an alert on the metric. You'll get a notification whenever the metric crosses the threshold in either
direction; you won't get a notification until the first crossing, no matter whether the initial value is high or low; there
is always a latency of a few minutes.
Are there data transfer charges between an Azure web app and Application Insights?
If your Azure web app is hosted in a data center where there is an Application Insights collection endpoint, there
is no charge.
If there is no collection endpoint in your host data center, then your app's telemetry will incur Azure outgoing
charges.
This doesn't depend on where your Application Insights resource is hosted. It just depends on the distribution of
our endpoints.
Can I send telemetry to the Application Insights portal?
We recommend you use our SDKs and use the SDK API. There are variants of the SDK for various platforms.
These SDKs handle buffering, compression, throttling, retries, and so on. However, the ingestion schema and
endpoint protocol are public.
Can I monitor an intranet web server?
Yes, but you will need to allow traffic to our services by either firewall exceptions or proxy redirects.
QuickPulse https://rt.services.visualstudio.com:443
ApplicationIdProvider https://dc.services.visualstudio.com:443
TelemetryChannel https://dc.services.visualstudio.com:443
<ApplicationInsights>
...
<TelemetryModules>
<Add
Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule,
Microsoft.AI.PerfCounterCollector">
<QuickPulseServiceEndpoint>https://rt.services.visualstudio.com/QuickPulseService.svc</QuickPulseServiceEndpoin
t>
</Add>
</TelemetryModules>
...
<TelemetryChannel>
<EndpointAddress>https://dc.services.visualstudio.com/v2/track</EndpointAddress>
</TelemetryChannel>
...
<ApplicationIdProvider
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplicationId
Provider, Microsoft.ApplicationInsights">
<ProfileQueryEndpoint>https://dc.services.visualstudio.com/api/profiles/{0}/appId</ProfileQueryEndpoint>
</ApplicationIdProvider>
...
</ApplicationInsights>
NOTE
ApplicationIdProvider is available starting in v2.6.0.
Proxy passthrough
Proxy passthrough can be achieved by configuring either a machine level or application level proxy. For more
information see dotnet's article on DefaultProxy.
Example Web.config:
<system.net>
<defaultProxy>
<proxy proxyaddress="http://xx.xx.xx.xx:yyyy" bypassonlocal="true"/>
</defaultProxy>
</system.net>
Option 2
Re-enable collection for these properties for every container log line.
If the first option is not convenient due to query changes involved, you can re-enable collecting these fields by
enabling the setting log_collection_settings.enrich_container_logs in the agent config map as described in the
data collection configuration settings.
NOTE
The second option is not recommended with large clusters that have more than 50 nodes because it generates API server
calls from every node in the cluster to perform this enrichment. This option also increases data size for every log line
collected.
console.log(json.stringify({
"Hello": "This example has multiple lines:",
"Docker/Moby": "will not break this into multiple lines",
"and you will receive":"all of them in log analytics",
"as one": "log entry"
}));
This data will look like the following example in Azure Monitor for logs when you query for it:
LogEntry : ({“Hello": "This example has multiple lines:","Docker/Moby": "will not break this into multiple
lines", "and you will receive":"all of them in log analytics", "as one": "log entry"}
For a detailed look at the issue, review the following GitHub link.
How do I resolve Azure AD errors when I enable live logs?
You may see the following error: The reply url specified in the request does not match the reply urls
configured for the application: '<application ID>'. The solution to solve it can be found in the article How to
view container data in real time with Azure Monitor for containers.
Why can't I upgrade cluster after onboarding?
If after you enable Azure Monitor for containers for an AKS cluster, you delete the Log Analytics workspace the
cluster was sending its data to, when attempting to upgrade the cluster it will fail. To work around this, you will
have to disable monitoring and then re-enable it referencing a different valid workspace in your subscription.
When you try to perform the cluster upgrade again, it should process and complete successfully.
Which ports and domains do I need to open/whitelist for the agent?
See the Network firewall requirements for the proxy and firewall configuration information required for the
containerized agent with Azure, Azure US Government, and Azure China 21Vianet clouds.
NOTE
We configure performance counters for the workspace that affects all VMs that report to the workspace, whether or not you
have chosen to onboard them to Azure Monitor for VMs. For more details on how performance counters are configured for
the workspace, please refer to our documentation. For information about the counters configured for Azure Monitor for VMs,
please refer to our enable Azure Monitor for VMs article.
Next steps
If your question isn't answered here, you can refer to the following forums to additional questions and answers.
Log Analytics
Application Insights
For general feedback on Azure Monitor please visit the feedback forum.
How to enable Azure Monitor for containers
12/12/2019 • 5 minutes to read • Edit Online
This article provides an overview of the options available to setup Azure Monitor for containers to monitor the
performance of workloads that are deployed to Kubernetes environments and hosted on:
Azure Kubernetes Service (AKS )
AKS Engine on Azure Stack or Kubernetes deployed on-premises.
Azure Red Hat OpenShift
Azure Monitor for containers can be enabled for new, or one or more existing deployments of Kubernetes using
the following supported methods:
From the Azure portal, Azure PowerShell, or with Azure CLI
Using Terraform and AKS
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Prerequisites
Before you start, make sure that you have the following:
A Log Analytics workspace.
Azure Monitor for containers supports a Log Analytics workspace in the regions listed in Azure Products by
region.
You can create a workspace when you enable monitoring of your new AKS cluster or let the onboarding
experience create a default workspace in the default resource group of the AKS cluster subscription. If you
chose to create it yourself, you can create it through Azure Resource Manager, through PowerShell, or in
the Azure portal. For a list of the supported mapping pairs used for the default workspace, see Region
mapping for Azure Monitor for containers.
You are a member of the Log Analytics contributor role to enable container monitoring. For more
information about how to control access to a Log Analytics workspace, see Manage workspaces.
You are a member of the Owner role on the AKS cluster resource.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
Prometheus metrics are not collected by default. Before configuring the agent to collect them, it is important
you review the Prometheus documentation to understand what can be scraped and methods supported.
Supported configurations
The following is officially supported with Azure Monitor for containers.
Environments: Azure Red Hat OpenShift, Kubernetes on-premises, and AKS Engine on Azure and Azure Stack.
For more information, see AKS Engine on Azure Stack.
Versions of Kubernetes and support policy are the same as versions of AKS supported.
*.ods.opinsights.azure.com 443
*.oms.opinsights.azure.com 443
*.blob.core.windows.net 443
dc.services.visualstudio.com 443
*.microsoftonline.com 443
*.monitoring.azure.com 443
login.microsoftonline.com 443
The information in the following table lists the proxy and firewall configuration information for Azure China.
The information in the following table lists the proxy and firewall configuration information for Azure US
Government.
AGENT RESOURCE PORTS DESCRIPTION
Components
Your ability to monitor performance relies on a containerized Log Analytics agent for Linux specifically developed
for Azure Monitor for containers. This specialized agent collects performance and event data from all nodes in the
cluster, and the agent is automatically deployed and registered with the specified Log Analytics workspace during
deployment. The agent version is microsoft/oms:ciprod04202018 or later, and is represented by a date in the
following format: mmddyyyy.
NOTE
With the preview release of Windows Server support for AKS, an AKS cluster with Windows Server nodes do not have an
agent installed to collect data and forward to Azure Monitor. Instead, a Linux node automatically deployed in the cluster as
part of the standard deployment collects and forwards the data to Azure Monitor on behalf all Windows nodes in the cluster.
When a new version of the agent is released, it is automatically upgraded on your managed Kubernetes clusters
hosted on Azure Kubernetes Service (AKS ). To follow the versions released, see agent release announcements.
NOTE
If you have already deployed an AKS cluster, you enable monitoring by using either Azure CLI or a provided Azure Resource
Manager template, as demonstrated later in this article. You cannot use kubectl to upgrade, delete, re-deploy, or deploy
the agent. The template needs to be deployed in the same resource group as the cluster.
You enable Azure Monitor for containers by using one of the following methods described in the following table.
New Kubernetes cluster Create AKS cluster using Azure CLI You can enable monitoring of a new
AKS cluster that you create with Azure
CLI.
Create AKS cluster using Terraform You can enable monitoring of a new
AKS cluster that you create using the
open-source tool Terraform.
DEPLOYMENT STATE METHOD DESCRIPTION
Create OpenShift cluster using an Azure You can enable monitoring of a new
Resource Manager template OpenShift cluster that you create with a
pre-configured Azure Resource
Manager template.
Existing Kubernetes cluster Enable for AKS cluster using Azure CLI You can enable monitoring of an AKS
cluster already deployed using Azure
CLI.
Enable for AKS cluster using Terraform You can enable monitoring of an AKS
cluster already deployed using the
open-source tool Terraform.
Enable for AKS cluster from Azure You can enable monitoring of one or
Monitor more AKS clusters already deployed
from the multi-cluster page in Azure
Monitor.
Enable from AKS cluster You can enable monitoring directly from
an AKS cluster in the Azure portal.
Enable for AKS cluster using an Azure You can enable monitoring of an AKS
Resource Manager template cluster with a pre-configured Azure
Resource Manager template.
Enable for hybrid Kubernetes cluster You can enable monitoring of an AKS
Engine hosted in Azure Stack or for
Kubernetes hosted on-premises.
Enable for OpenShift cluster from Azure You can enable monitoring of one or
Monitor more OpenShift clusters already
deployed from the multi-cluster page in
Azure Monitor.
Next steps
With monitoring enabled, you can begin analyzing the performance of your Kubernetes clusters hosted on
Azure Kubernetes Service (AKS ), Azure Stack, or other environment. To learn how to use Azure Monitor for
containers, see View Kubernetes cluster performance.
Enable monitoring of a new Azure Kubernetes
Service (AKS) cluster
12/12/2019 • 3 minutes to read • Edit Online
This article describes how to set up Azure Monitor for containers to monitor managed Kubernetes cluster hosted
on Azure Kubernetes Service that you are preparing to deploy in your subscription.
You can enable monitoring of an AKS cluster using one of the supported methods:
Azure CLI
Terraform
NOTE
If you choose to use the Azure CLI, you first need to install and use the CLI locally. You must be running the Azure CLI
version 2.0.74 or later. To identify your version, run az --version . If you need to install or upgrade the Azure CLI, see
Install the Azure CLI. If you have installed the aks-preview CLI extension version 0.4.12 or higher, remove any changes you
have made to enable a preview extension as it can override the default Azure CLI behavior since AKS Preview features aren't
available in Azure US Governmnet cloud.
NOTE
If you choose to use Terraform, you must be running the Terraform Azure RM Provider version 1.17.0 or above.
To add Azure Monitor for containers to the workspace, see azurerm_log_analytics_solution and complete the
profile by including the addon_profile and specify oms_agent.
After you've enabled monitoring and all configuration tasks are completed successfully, you can monitor the
performance of your cluster in either of two ways:
Directly in the AKS cluster by selecting Health in the left pane.
By selecting the Monitor Container insights tile in the AKS cluster page for the selected cluster. In Azure
Monitor, in the left pane, select Health.
After you've enabled monitoring, it might take about 15 minutes before you can view health metrics for the cluster.
The output should resemble the following, which indicates that it was deployed properly:
The output should resemble the following, which indicates that it was deployed properly:
The output should resemble the following, which indicates that it was deployed properly:
After a few minutes, the command completes and returns JSON -formatted information about solution. The results
of the command should show the monitoring add-on profile and resembles the following example output:
"addonProfiles": {
"omsagent": {
"config": {
"logAnalyticsWorkspaceResourceID":
"/subscriptions/<WorkspaceSubscription>/resourceGroups/<DefaultWorkspaceRG>/providers/Microsoft.OperationalInsi
ghts/workspaces/<defaultWorkspaceName>"
},
"enabled": true
}
}
Next steps
If you experience issues while attempting to onboard the solution, review the troubleshooting guide
With monitoring enabled to collect health and resource utilization of your AKS cluster and workloads
running on them, learn how to use Azure Monitor for containers.
Enable monitoring of Azure Kubernetes Service (AKS)
cluster already deployed
1/14/2020 • 8 minutes to read • Edit Online
This article describes how to set up Azure Monitor for containers to monitor managed Kubernetes cluster hosted
on Azure Kubernetes Service that have already been deployed in your subscription.
You can enable monitoring of an AKS cluster that's already deployed using one of the supported methods:
Azure CLI
Terraform
From Azure Monitor or directly from the AKS cluster in the Azure portal
With the provided Azure Resource Manager template by using the Azure PowerShell cmdlet
New-AzResourceGroupDeployment or with Azure CLI.
provisioningState : Succeeded
3. The following example displays the list of workspaces in your subscriptions in the default JSON format.
In the output, find the workspace name, and then copy the full resource ID of that Log Analytics workspace
under the field id.
4. Run the following command to enable the monitoring add-on, replacing the value for the
--workspace-resource-id parameter. The string value must be within the double quotes:
provisioningState : Succeeded
addon_profile {
oms_agent {
enabled = true
log_analytics_workspace_id = "${azurerm_log_analytics_workspace.test.id}"
}
}
NOTE
If you want to create a new Log Analytics workspace for storing the monitoring data from the cluster, follow the
instructions in Create a Log Analytics workspace. Be sure to create the workspace in the same subscription that the
AKS container is deployed to.
After you've enabled monitoring, it might take about 15 minutes before you can view health metrics for the cluster.
NOTE
If you want to create a new Log Analytics workspace for storing the monitoring data from the cluster, follow the
instructions in Create a Log Analytics workspace. Be sure to create the workspace in the same subscription that the
AKS container is deployed to.
After you've enabled monitoring, it might take about 15 minutes before you can view operational data for the
cluster.
NOTE
The template needs to be deployed in the same resource group as the cluster.
The Log Analytics workspace has to be created before you enable monitoring using Azure PowerShell or CLI. To
create the workspace, you can set it up through Azure Resource Manager, through PowerShell, or in the Azure
portal.
If you are unfamiliar with the concept of deploying resources by using a template, see:
Deploy resources with Resource Manager templates and Azure PowerShell
Deploy resources with Resource Manager templates and the Azure CLI
If you choose to use the Azure CLI, you first need to install and use the CLI locally. You must be running the Azure
CLI version 2.0.59 or later. To identify your version, run az --version . If you need to install or upgrade the Azure
CLI, see Install the Azure CLI.
Create and execute a template
1. Copy and paste the following JSON syntax into your file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"aksResourceId": {
"type": "string",
"metadata": {
"description": "AKS Cluster Resource ID"
}
},
"aksResourceLocation": {
"type": "string",
"metadata": {
"description": "Location of the AKS resource e.g. \"East US\""
}
},
"aksResourceTagValues": {
"type": "object",
"metadata": {
"description": "Existing all tags on AKS Cluster Resource"
}
},
"workspaceResourceId": {
"type": "string",
"metadata": {
"description": "Azure Monitor Log Analytics Resource ID"
}
}
},
"resources": [
{
"name": "[split(parameters('aksResourceId'),'/')[8]]",
"type": "Microsoft.ContainerService/managedClusters",
"location": "[parameters('aksResourceLocation')]",
"tags": "[parameters('aksResourceTagValues')]",
"apiVersion": "2018-03-31",
"properties": {
"mode": "Incremental",
"id": "[parameters('aksResourceId')]",
"addonProfiles": {
"omsagent": {
"enabled": true,
"config": {
"logAnalyticsWorkspaceResourceID": "[parameters('workspaceResourceId')]"
}
}
}
}
}
]
}
4. Edit the values for aksResourceId and aksResourceLocation using the values on the AKS Overview
page for the AKS cluster. The value for workspaceResourceId is the full resource ID of your Log Analytics
workspace, which includes the workspace name.
Edit the values for aksResourceTagValues to match the existing tag values specified for the AKS cluster.
5. Save this file as existingClusterParam.json to a local folder.
6. You are ready to deploy this template.
To deploy with Azure PowerShell, use the following commands in the folder that contains the
template:
The configuration change can take a few minutes to complete. When it's completed, a message is
displayed that's similar to the following and includes the result:
provisioningState : Succeeded
az login
az account set --subscription "Subscription Name"
az group deployment create --resource-group <ResourceGroupName> --template-file
./existingClusterOnboarding.json --parameters @./existingClusterParam.json
The configuration change can take a few minutes to complete. When it's completed, a message is
displayed that's similar to the following and includes the result:
provisioningState : Succeeded
After you've enabled monitoring, it might take about 15 minutes before you can view health metrics
for the cluster.
The output should resemble the following, which indicates that it was deployed properly:
The output should resemble the following, which indicates that it was deployed properly:
The output should resemble the following, which indicates that it was deployed properly:
"addonProfiles": {
"omsagent": {
"config": {
"logAnalyticsWorkspaceResourceID":
"/subscriptions/<WorkspaceSubscription>/resourceGroups/<DefaultWorkspaceRG>/providers/Microsoft.OperationalInsi
ghts/workspaces/<defaultWorkspaceName>"
},
"enabled": true
}
}
Next steps
If you experience issues while attempting to onboard the solution, review the troubleshooting guide
With monitoring enabled to collect health and resource utilization of your AKS cluster and workloads
running on them, learn how to use Azure Monitor for containers.
Configure hybrid Kubernetes clusters with Azure
Monitor for containers
1/24/2020 • 7 minutes to read • Edit Online
Azure Monitor for containers provides rich monitoring experience for the Azure Kubernetes Service (AKS ) and
AKS Engine clusters hosted in Azure. This article describes how to enable monitoring of Kubernetes clusters
hosted outside of Azure and achieve a similar monitoring experience.
Prerequisites
Before you start, make sure that you have the following:
A Log Analytics workspace.
Azure Monitor for containers supports a Log Analytics workspace in the regions listed in Azure Products by
region. To create your own workspace, it can be created through Azure Resource Manager, through
PowerShell, or in the Azure portal.
NOTE
Enable monitoring of multiple clusters with the same cluster name to same Log Analytics workspace is not supported.
Cluster names must be unique.
You are a member of the Log Analytics contributor role to enable container monitoring. For more
information about how to control access to a Log Analytics workspace, see Manage access to workspace
and log data
HELM client to onboard the Azure Monitor for containers chart for the specified Kubernetes cluster.
The following proxy and firewall configuration information is required for the containerized version of the
Log Analytics agent for Linux to communicate with Azure Monitor:
The containerized agent requires Kubelet’s cAdvisor secure port: 10250 or unsecure port :10255 to be
opened on all nodes in the cluster to collect performance metrics. We recommend you configure
secure port: 10250 on the Kubelet’s cAdvisor if it’s not configured already.
The containerized agent requires the following environmental variables to be specified on the container in
order to communicate with the Kubernetes API service within the cluster to collect inventory data -
KUBERNETES_SERVICE_HOST and KUBERNETES_PORT_443_TCP_PORT .
IMPORTANT
The minimum agent version supported for monitoring hybrid Kubernetes clusters is ciprod10182019 or later.
Supported configurations
The following is officially supported with Azure Monitor for containers.
Environments: Kubernetes on-premises, AKS Engine on Azure and Azure Stack. For more information, see AKS
Engine on Azure Stack.
Versions of Kubernetes and support policy are the same as versions of AKS supported.
Container Runtime: Docker and Moby
Linux OS release for master and worked nodes: Ubuntu (18.04 LTS and 16.04 LTS )
Access control supported: Kubernetes RBAC and non-RBAC
Enable monitoring
Enabling Azure Monitor for containers for the hybrid Kubernetes cluster consists of performing the following steps
in order.
1. Configure your Log Analytics workspace with Container Insights solution.
2. Enable the Azure Monitor for containers HELM chart with Log Analytics workspace.
How to add the Azure Monitor Containers solution
You can deploy the solution with the provided Azure Resource Manager template by using the Azure PowerShell
cmdlet New-AzResourceGroupDeployment or with Azure CLI.
If you are unfamiliar with the concept of deploying resources by using a template, see:
Deploy resources with Resource Manager templates and Azure PowerShell
Deploy resources with Resource Manager templates and the Azure CLI
If you choose to use the Azure CLI, you first need to install and use the CLI locally. You must be running the Azure
CLI version 2.0.59 or later. To identify your version, run az --version . If you need to install or upgrade the Azure
CLI, see Install the Azure CLI.
This method includes two JSON templates. One template specifies the configuration to enable monitoring, and the
other contains parameter values that you configure to specify the following:
workspaceResourceId - the full resource ID of your Log Analytics workspace.
workspaceRegion - the region the workspace is created in, which is also referred to as Location in the
workspace properties when viewing from the Azure portal.
To first identify the full resource ID of your Log Analytics workspace required for the workspaceResourceId
parameter value in the containerSolutionParams.json file, perform the following steps and then run the
PowerShell cmdlet or Azure CLI command to add the solution.
1. List all the subscriptions that you have access to using the following command:
3. The following example displays the list of workspaces in your subscriptions in the default JSON format.
In the output, find the workspace name, and then copy the full resource ID of that Log Analytics workspace
under the field ID.
4. Copy and paste the following JSON syntax into your file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceResourceId": {
"type": "string",
"metadata": {
"description": "Azure Monitor Log Analytics Workspace Resource ID"
}
},
"workspaceRegion": {
"type": "string",
"metadata": {
"description": "Azure Monitor Log Analytics Workspace region"
}
}
},
"resources": [
{
"type": "Microsoft.Resources/deployments",
"name": "[Concat('ContainerInsights', '-', uniqueString(parameters('workspaceResourceId')))]",
"apiVersion": "2017-05-10",
"subscriptionId": "[split(parameters('workspaceResourceId'),'/')[2]]",
"resourceGroup": "[split(parameters('workspaceResourceId'),'/')[4]]",
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-
01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2015-11-01-preview",
"type": "Microsoft.OperationsManagement/solutions",
"location": "[parameters('workspaceRegion')]",
"name": "[Concat('ContainerInsights', '(',
split(parameters('workspaceResourceId'),'/')[8], ')')]",
"properties": {
"workspaceResourceId": "[parameters('workspaceResourceId')]"
},
"plan": {
"name": "[Concat('ContainerInsights', '(',
split(parameters('workspaceResourceId'),'/')[8], ')')]",
"product": "[Concat('OMSGallery/', 'ContainerInsights')]",
"promotionCode": "",
"publisher": "Microsoft"
}
}
]
},
"parameters": {}
}
}
]
}
7. Edit the values for workspaceResourceId using the value you copied in step 3, and for workspaceRegion
copy the Region value after running the Azure CLI command az monitor log-analytics workspace show.
8. Save this file as containerSolutionParams.json to a local folder.
9. You are ready to deploy this template.
To deploy with Azure PowerShell, use the following commands in the folder that contains the
template:
# configure and login to the cloud of log analytics workspace.Specify the corresponding cloud
environment of your workspace to below command.
Connect-AzureRmAccount -Environment <AzureCloud | AzureChinaCloud | AzureUSGovernment>
# execute deployment command to add container insights solution to the specified Log Analytics
workspace
New-AzureRmResourceGroupDeployment -Name OnboardCluster -ResourceGroupName <resource group of log
analytics workspace> -TemplateFile .\containerSolution.json -TemplateParameterFile
.\containerSolutionParams.json
The configuration change can take a few minutes to complete. When it's completed, a message is
displayed that's similar to the following and includes the result:
provisioningState : Succeeded
az login
az account set --name <AzureCloud | AzureChinaCloud | AzureUSGovernment>
az login
az account set --subscription "Subscription Name"
# execute deployment command to add container insights solution to the specified Log Analytics
workspace
az group deployment create --resource-group <resource group of log analytics workspace> --
template-file ./containerSolution.json --parameters @./containerSolutionParams.json
The configuration change can take a few minutes to complete. When it's completed, a message is
displayed that's similar to the following and includes the result:
provisioningState : Succeeded
After you've enabled monitoring, it might take about 15 minutes before you can view health metrics
for the cluster.
If the Log Analytics workspace is in Azure China, run the following command:
If the Log Analytics workspace is in Azure US Government, run the following command:
NOTE
Ingestion latency is around five to ten minutes from agent to commit in the Azure Log Analytics workspace. Status of the
cluster show the value No data or Unknown until all the required monitoring data is available in Azure Monitor.
Troubleshooting
If you encounter an error while attempting to enable monitoring for your hybrid Kubernetes cluster, copy the
PowerShell script TroubleshootError_nonAzureK8s.ps1 and save it to a folder on your computer. This script is
provided to help detect and fix the issues encountered. The issues it is designed to detect and attempt correction of
are the following:
The specified Log Analytics workspace is valid
The Log Analytics workspace is configured with the Azure Monitor for Containers solution. If not, configure the
workspace.
OmsAgent replicaset pod are running
OmsAgent daemonset pod are running
OmsAgent Health service is running
The Log Analytics workspace Id and key configured on the containerized agent match with the workspace the
Insight is configured with.
Validate all the Linux worker nodes have kubernetes.io/role=agent label to schedule rs pod. If it doesn't exist,
add it.
Validate cAdvisor secure port:10250 or unsecure port: 10255 is opened on all nodes in the cluster.
To execute with Azure PowerShell, use the following commands in the folder that contains the script:
.\TroubleshootError_nonAzureK8s.ps1 - azureLogAnalyticsWorkspaceResourceId
</subscriptions/<subscriptionId>/resourceGroups/<resourcegroupName>/providers/Microsoft.OperationalInsights/wo
rkspaces/<workspaceName> -kubeConfig <kubeConfigFile> -clusterContextInKubeconfig <clusterContext>
Next steps
With monitoring enabled to collect health and resource utilization of your hybrid Kubernetes cluster and
workloads running on them, learn how to use Azure Monitor for containers.
Configure Azure Red Hat OpenShift clusters with
Azure Monitor for containers
1/14/2020 • 6 minutes to read • Edit Online
Azure Monitor for containers provides rich monitoring experience for the Azure Kubernetes Service (AKS ) and
AKS Engine clusters. This article describes how to enable monitoring of Kubernetes clusters hosted on Azure Red
Hat OpenShift to achieve a similar monitoring experience.
NOTE
Support for Azure Red Hat OpenShift is a feature in public preview at this time.
Azure Monitor for containers can be enabled for new, or one or more existing deployments of Azure Red Hat
OpenShift using the following supported methods:
For an existing cluster from the Azure portal or using Azure Resource Manager template
For a new cluster using Azure Resource Manager template
Prerequisites
To enable and access the features in Azure Monitor for containers, at a minimum you need to be a member
of the Azure Contributor role in the Azure subscription, and a member of the Log Analytics Contributor role
of the Log Analytics workspace configured with Azure Monitor for containers.
To view the monitoring data, you are a member of the Log Analytics reader role permission with the Log
Analytics workspace configured with Azure Monitor for containers.
2. Sign in to Azure
az login
If you have access to multiple subscriptions, run az account set -s {subscription ID} replacing
{subscription ID} with the subscription you want to use.
3. Create a resource group for your cluster if you don't already have one. For a list of Azure regions that
supports OpenShift on Azure, see Supported Regions.
4. Edit the JSON parameter file newClusterWithMonitoringParam.json and update the following values:
location
clusterName
aadTenantId
aadClientId
aadClientSecret
aadCustomerAdminGroupId
workspaceResourceId
masterNodeCount
computeNodeCount
infraNodeCount
5. The following step deploys the cluster with monitoring enabled by using the Azure CLI.
provisioningState : Succeeded
After you've enabled monitoring, it might take about 15 minutes before you can view health metrics for the cluster.
Enable using an Azure Resource Manager template
This method includes two JSON templates. One template specifies the configuration to enable monitoring, and the
other contains parameter values that you configure to specify the following:
The Azure RedHat OpenShift cluster resource ID.
The resource group the cluster is deployed in.
A Log Analytics workspace.
If you are unfamiliar with the concept of deploying resources by using a template, see:
Deploy resources with Resource Manager templates and Azure PowerShell
Deploy resources with Resource Manager templates and the Azure CLI
If you choose to use the Azure CLI, you first need to install and use the CLI locally. You must be running the Azure
CLI version 2.0.65 or later. To identify your version, run az --version . If you need to install or upgrade the Azure
CLI, see Install the Azure CLI.
The Log Analytics workspace has to be created before you enable monitoring using Azure PowerShell or CLI. To
create the workspace, you can set it up through Azure Resource Manager, through PowerShell, or in the Azure
portal.
1. Download the template and parameter file to update your cluster with the monitoring add-on using the
following commands:
curl -LO https://raw.githubusercontent.com/microsoft/OMS-
docker/ci_feature/docs/aro/enable_monitoring_to_existing_cluster/existingClusterOnboarding.json
2. Sign in to Azure
az login
If you have access to multiple subscriptions, run az account set -s {subscription ID} replacing
{subscription ID} with the subscription you want to use.
4. Run the following command to identify the cluster location and resource ID:
5. Edit the JSON parameter file existingClusterParam.json and update the values araResourceId and
araResoruceLocation. The value for workspaceResourceId is the full resource ID of your Log Analytics
workspace, which includes the workspace name.
6. To deploy with Azure CLI, run the following commands:
provisioningState : Succeeded
Next steps
With monitoring enabled to collect health and resource utilization of your RedHat OpenShift cluster and
workloads running on them, learn how to use Azure Monitor for containers.
To learn how to stop monitoring your cluster with Azure Monitor for containers, see How to Stop
Monitoring Your Azure Red Hat OpenShift cluster.
Region mappings supported by Azure Monitor for
containers
12/12/2019 • 2 minutes to read • Edit Online
When enabling Azure Monitor for containers, only certain regions are supported for linking a Log Analytics
workspace and an AKS cluster, and collecting custom metrics submitted to Azure Monitor.
Africa
SouthAfricaNorth WestEurope
SouthAfricaWest WestEurope
Australia
AustraliaEast AustraliaEast
AustraliaCentral AustraliaCentral
AustraliaCentral2 AustraliaCentral
AustraliaEast AustraliaEast
Asia Pacific
EastAsia EastAsia
SoutheastAsia SoutheastAsia
Brazil
BrazilSouth SouthCentralUS
Canada
CanadaCentral CanadaCentral
CanadaEast CanadaCentral
Europe
AKS CLUSTER REGION LOG ANALYTICS WORKSPACE REGION
FranceCentral FranceCentral
FranceSouth FranceCentral
NorthEurope NorthEurope
UKSouth UKSouth
UKWest UKSouth
WestEurope WestEurope
India
CentralIndia CentralIndia
SouthIndia CentralIndia
WestIndia CentralIndia
Japan
JapanEast JapanEast
JapanWest JapanEast
Korea
KoreaCentral KoreaCentral
KoreaSouth KoreaCentral
US
CentralUS CentralUS
EastUS EastUS
EastUS2 EastUS2
WestUS WestUS
WestUS2 WestUS2
WestCentralUS1 EastUS1
1 Due to capacity restraints, the region isn't available when creating new
resources. This includes a Log Analytics
workspace. However, preexisting linked resources in the region should continue to work.
Custom metrics supported regions
Collecting metrics from Azure Kubernetes Services (AKS ) clusters nodes and pods are supported for publishing as
custom metrics only in the following Azure regions.
Next steps
To begin monitoring your AKS cluster, review How to enable the Azure Monitor for containers to understand the
requirements and available methods to enable monitoring.
Monitor your Kubernetes cluster performance with
Azure Monitor for containers
1/8/2020 • 16 minutes to read • Edit Online
With Azure Monitor for containers, you can use the performance charts and health status to monitor the
workload of Kubernetes clusters hosted on Azure Kubernetes Service (AKS ), Azure Stack, or other environment
from two perspectives. You can monitor directly from the cluster, or you can view all clusters in a subscription
from Azure Monitor. Viewing Azure Container Instances is also possible when monitoring a specific AKS cluster.
This article helps you understand the two perspectives, and how Azure Monitor helps you quickly assess,
investigate, and resolve detected issues.
For information about how to enable Azure Monitor for containers, see Onboard Azure Monitor for containers.
Azure Monitor provides a multi-cluster view that shows the health status of all monitored Kubernetes clusters
running Linux and Windows Server 2019 deployed across resource groups in your subscriptions. It shows
clusters discovered across all environments that aren't monitored by the solution. You can immediately
understand cluster health, and from here, you can drill down to the node and controller performance page or
navigate to see performance charts for the cluster. For AKS clusters that were discovered and identified as
unmonitored, you can enable monitoring for them at any time.
The main differences in monitoring a Windows Server cluster with Azure Monitor for containers compared to a
Linux cluster are described here in the overview article.
You can scope the results presented in the grid to show clusters that are:
Azure - AKS and AKS -Engine clusters hosted in Azure Kubernetes Service
Azure Stack (Preview) - AKS -Engine clusters hosted on Azure Stack
Non-Azure (Preview) - Kubernetes clusters hosted on-premises
All - View all the Kubernetes clusters hosted in Azure, Azure Stack, and on-premises environments that are
onboarded to Azure Monitor for containers
To view clusters from a specific environment, select it from the Environments pill on the top-left corner of the
page.
STATUS AVAILABILITY
User pod
Healthy 100%
STATUS AVAILABILITY
Warning 90 - 99%
Critical <90%
System pod
Healthy 100%
Warning N/A
Critical <100%
Node
Healthy >85%
Warning 60 - 84%
Critical <60%
From the list of clusters, you can drill down to the Cluster page by selecting the name of the cluster. Then go to
the Nodes performance page by selecting the rollup of nodes in the Nodes column for that specific cluster. Or,
you can drill down to the Controllers performance page by selecting the rollup of the User pods or System
pods column.
NOTE
The experience described in the remainder of this article are also applicable for viewing performance and health status of
your Kubernetes clusters hosted on Azure Stack or other environment when selected from the multi-cluster view.
The default page opens and displays four line performance charts that show key performance metrics of your
cluster.
The performance charts display four performance metrics:
Node CPU utilization %: An aggregated perspective of CPU utilization for the entire cluster. To filter the
results for the time range, select Avg, Min, 50th, 90th, 95th, or Max in the percentiles selector above the
chart. The filters can be used either individually or combined.
Node memory utilization %: An aggregated perspective of memory utilization for the entire cluster. To filter
the results for the time range, select Avg, Min, 50th, 90th, 95th, or Max in the percentiles selector above the
chart. The filters can be used either individually or combined.
Node count: A node count and status from Kubernetes. Statuses of the cluster nodes represented are Total,
Ready, and Not Ready. They can be filtered individually or combined in the selector above the chart.
Active pod count: A pod count and status from Kubernetes. Statuses of the pods represented are Total,
Pending, Running, Unknown, Succeeded, or Failed. They can be filtered individually or combined in the
selector above the chart.
Use the Left and Right arrow keys to cycle through each data point on the chart. Use the Up and Down arrow
keys to cycle through the percentile lines. Select the pin icon in the upper-right corner of any one of the charts to
pin the selected chart to the last Azure dashboard you viewed. From the dashboard, you can resize and reposition
the chart. Selecting the chart from the dashboard redirects you to Azure Monitor for containers and loads the
correct scope and view.
Azure Monitor for containers also supports Azure Monitor metrics explorer, where you can create your own plot
charts, correlate and investigate trends, and pin to dashboards. From metrics explorer, you also can use the
criteria that you set to visualize your metrics as the basis of a metric-based alert rule.
insights.container/nodes
NAMESPACE METRIC DESCRIPTION
insights.container/pods
You can split a metric to view it by dimension and visualize how different segments of it compare to each other.
For a node, you can segment the chart by the host dimension. From a pod, you can segment it by the following
dimensions:
Controller
Kubernetes namespace
Node
Phase
Windows Server containers that run the Windows Server 2019 OS are shown after all of the Linux-based nodes
in the list. When you expand a Windows Server node, you can view one or more pods and containers that run on
the node. After a node is selected, the properties pane shows version information. Agent information is excluded
because Windows Server nodes don't have an agent installed.
Azure Container Instances virtual nodes that run the Linux OS are shown after the last AKS cluster node in the
list. When you expand a Container Instances virtual node, you can view one or more Container Instances pods
and containers that run on the node. Metrics aren't collected and reported for nodes, only for pods.
From an expanded node, you can drill down from the pod or container that runs on the node to the controller to
view performance data filtered for that controller. Select the value under the Controller column for the specific
node.
Select controllers or containers at the top of the page to review the status and resource utilization for those
objects. To review memory utilization, in the Metric drop-down list, select Memory RSS or Memory working
set. Memory RSS is supported only for Kubernetes version 1.8 and later. Otherwise, you view values for Min %
as NaN %, which is a numeric data type value that represents an undefined or unrepresentable value.
Memory working set shows both the resident memory and virtual memory (cache) included and is a total of
what the application is using. Memory RSS shows only main memory (which is nothing but the resident
memory in other words). This metric shows the actual capacity of available memory. What is the difference
between resident memory and virtual memory?
Resident memory or main memory, is the actual amount of machine memory available to the nodes of the
cluster.
Virtual memory is reserved hard disk space (cache) used by the operating system to swap data from
memory to disk when under memory pressure, and then fetch it back to memory when needed.
By default, performance data is based on the last six hours, but you can change the window by using the
TimeRange option at the upper left. You also can filter the results within the time range by selecting Min, Avg,
50th, 90th, 95th, and Max in the percentile selector.
When you hover over the bar graph under the Trend column, each bar shows either CPU or memory usage,
depending on which metric is selected, within a sample period of 15 minutes. After you select the trend chart
through a keyboard, use the Alt+Page up key or Alt+Page down key to cycle through each bar individually. You
get the same details that you would if you hovered over the bar.
In the next example, for the first node in the list, aks-nodepool1 -, the value for Containers is 9. This value is a
rollup of the total number of containers deployed.
This information can help you quickly identify whether you have a proper balance of containers between nodes in
your cluster.
The information that's presented when you view the Nodes tab is described in the following table.
COLUMN DESCRIPTION
Min %, Avg %, 50th %, 90th %, 95th %, Max % Average node percentage based on percentile during the
selected duration.
Min, Avg, 50th, 90th, 95th, Max Average nodes' actual value based on percentile during the
time duration selected. The average value is measured from
the CPU/Memory limit set for a node. For pods and
containers, it's the average value reported by the host.
Trend Min %, Avg %, 50th %, 90th %, 95th %, Max % Bar graph trend represents the average percentile metric
percentage of the controller.
Here you can view the performance health of your controllers and Container Instances virtual node controllers or
virtual node pods not connected to a controller.
The row hierarchy starts with a controller. When you expand a controller, you view one or more pods. Expand a
pod, and the last row displays the container grouped to the pod. From an expanded controller, you can drill down
to the node it's running on to view performance data filtered for that node. Container Instances pods not
connected to a controller are listed last in the list.
Select the value under the Node column for the specific controller.
The information that's displayed when you view controllers is described in the following table.
COLUMN DESCRIPTION
Status The rollup status of the containers after it's finished running
with status such as OK, Terminated, Failed, Stopped, or
Paused. If the container is running but the status either
wasn't properly displayed or wasn't picked up by the agent
and hasn't responded for more than 30 minutes, the status is
Unknown. Additional details of the status icon are provided in
the following table.
Min %, Avg %, 50th %, 90th %, 95th %, Max % Rollup average of the average percentage of each entity for
the selected metric and percentile.
Min, Avg, 50th, 90th, 95th, Max Rollup of the average CPU millicore or memory performance
of the container for the selected percentile. The average value
is measured from the CPU/Memory limit set for a pod.
Trend Min %, Avg %, 50th %, 90th %, 95th %, Max % Bar graph trend represents the average percentile metric of
the controller.
The icons in the status field indicate the online status of the containers.
ICON STATUS
Running (Ready)
Waiting or Paused
The status icon displays a count based on what the pod provides. It shows the worst two states, and when you
hover over the status, it displays a rollup status from all pods in the container. If there isn't a ready state, the
status value displays (0).
In the selector, select Containers.
Here you can view the performance health of your Azure Kubernetes and Azure Container Instances containers.
From a container, you can drill down to a pod or node to view performance data filtered for that object. Select the
value under the Pod or Node column for the specific container.
The information that's displayed when you view containers is described in the following table.
COLUMN DESCRIPTION
Min %, Avg %, 50th %, 90th %, 95th %, Max % The rollup of the average percentage of each entity for the
selected metric and percentile.
Min, Avg, 50th, 90th, 95th, Max The rollup of the average CPU millicore or memory
performance of the container for the selected percentile. The
average value is measured from the CPU/Memory limit set
for a pod.
Trend Min %, Avg %, 50th %, 90th %, 95th %, Max % Bar graph trend represents the average percentile metric
percentage of the container.
The icons in the status field indicate the online statuses of pods, as described in the following table.
ICON STATUS
Running (Ready)
Waiting or Paused
Failed state
Workbooks
Workbooks combine text,log queries, metrics, and parameters into rich interactive reports. Workbooks are
editable by any other team members who have access to the same Azure resources.
Azure Monitor for containers includes four workbooks to get you started:
Disk capacity: Presents interactive disk usage charts for each disk presented to the node within a
container by the following perspectives:
Disk percent usage for all disks.
Free disk space for all disks.
A grid that shows each node's disk, its percentage of used space, trend of percentage of used space,
free disk space (GiB ), and trend of free disk space (GiB ). When a row is selected in the table, the
percentage of used space and free disk space (GiB ) is shown underneath the row.
Disk IO: Presents interactive disk utilization charts for each disk presented to the node within a container
by the following perspectives:
Disk I/O summarized across all disks by read bytes/sec, writes bytes/sec, and read and write bytes/sec
trends.
Eight performance charts show key performance indicators to help measure and identify disk I/O
bottlenecks.
Kubelet: Includes two grids that show key node operating statistics:
Overview by node grid summarizes total operation, total errors, and successful operations by percent
and trend for each node.
Overview by operation type summarizes for each operation the total operation, total errors, and
successful operations by percent and trend.
Network: Presents interactive network utilization charts for each node's network adapter, and a grid
presents the key performance indicators to help measure the performance of your network adapters.
You access these workbooks by selecting each one from the View Workbooks drop-down list.
Next steps
Review Create performance alerts with Azure Monitor for containers to learn how to create alerts for high
CPU and memory utilization to support your DevOps or operational processes and procedures.
View log query examples to see predefined queries and examples to evaluate or customize to alert,
visualize, or analyze your clusters.
View monitor cluster health to learn about viewing the health status your Kubernetes cluster.
Understand Kubernetes cluster health with Azure
Monitor for containers
12/13/2019 • 5 minutes to read • Edit Online
With Azure Monitor for containers, it monitors and reports health status of the managed infrastructure
components and all nodes running on any Kubernetes cluster supported by Azure Monitor for containers. This
experience extends beyond the cluster health status calculated and reported on the multi-cluster view, where now
you can understand if one or more nodes in the cluster are resource constrained, or a node or pod is unavailable
that could impact a running application in the cluster based on curated metrics.
NOTE
The Health feature is in public preview at this time.
For information about how to enable Azure Monitor for containers, see Onboard Azure Monitor for containers.
NOTE
To support AKS Engine clusters, verify it meets the following:
It is using the latest version of the HELM client.
The containerized agent version is microsoft/oms:ciprod11012019. To upgrade the agent, see upgrading agent on
Kubernetes cluster.
Overview
In Azure Monitor for containers, the Health (preview ) feature provides proactive health monitoring of your
Kubernetes cluster to help you identify and diagnose issues. It gives you the ability to view significant issues
detected. Monitors evaluating the health of your cluster run on the containerized agent in your cluster, and the
health data is written to the KubeHealth table in your Log Analytics workspace.
Kubernetes cluster health is based on a number of monitoring scenarios organized by the following Kubernetes
objects and abstractions:
Kubernetes infrastructure - provides a rollup of the Kubernetes API server, ReplicaSets, and DaemonSets
running on nodes deployed in your cluster by evaluating CPU and memory utilization, and a Pods
availability
Nodes - provides a rollup of the Node pools and state of individual Nodes in each pool, by evaluating CPU
and memory utilization, and a Node's status as reported by Kubernetes.
Currently, only the status of a virtual kubelet is supported. The health state for CPU and memory utilization of
virtual kublet nodes is reported as Unknown, since a signal is not received from them.
All monitors are shown in a hierarchical layout in the Health Hierarchy pane, where an aggregate monitor
representing the Kubernetes object or abstraction (that is, Kubernetes infrastructure or Nodes) are the top-most
monitor reflecting the combined health of all dependent child monitors. The key monitoring scenarios used to
derive health are:
Evaluate CPU utilization from the node and container.
Evaluate memory utilization from the node and container.
Status of Pods and Nodes based on calculation of their ready state reported by Kubernetes.
The icons used to indicate state are as follows:
ICON MEANING
Warning (yellow)
Critical (red)
Unknown (gray)
Monitor configuration
To understand the behavior and configuration of each monitor supporting Azure Monitor for containers Health
feature, see Health monitor configuration guide.
If you select a unit monitor in the Health Hierarchy pane, it also shows under Last state change the
previous samples calculated and reported by the containerized agent within the last four hours. This is
based on the unit monitors calculation for comparing several consecutive values to determine its state. For
example, if you selected the Pod ready state unit monitor, it shows the last two samples controlled by the
parameter ConsecutiveSamplesForStateTransition. For more information, see the detailed description of
unit monitors.
If the time reported by Last state change is a day or older, it is the result of no changes in state for the
monitor. However, if the last sample received for a unit monitor is more than four hours old, this likely
indicates the containerized agent has not been sending data. If the agent knows that a particular resource
exists, for example a Node, but it hasn't received data from the Node's CPU or memory utilization monitors
(as an example), then the health state of the monitor is set to Unknown.
On theConfig tab, it shows the default configuration parameter settings (only for unit monitors, not
aggregate monitors) and their values.
On the Knowledge tab, it contains information explaining the behavior of the monitor and how it evaluates
for the unhealthy condition.
Monitoring data on this page does not refresh automatically and you need to select Refresh at the top of the page
to see the most recent health state received from the cluster.
Next steps
View log query examples to see predefined queries and examples to evaluate or customize to alert, visualize, or
analyze your clusters.
How to view Kubernetes logs, events, and pod
metrics in real-time
12/13/2019 • 6 minutes to read • Edit Online
Azure Monitor for containers includes the Live Data (preview ) feature, which is an advanced diagnostic feature
allowing you direct access to your Azure Kubernetes Service (AKS ) container logs (stdout/stderror), events, and
pod metrics. It exposes direct access to kubectl logs -c , kubectl get events, and kubectl top pods . A console
pane shows the logs, events, and metrics generated by the container engine to further assist in troubleshooting
issues in real-time.
This article provides a detailed overview and helps you understand how to use this feature.
NOTE
AKS clusters enabled as private clusters are not supported with this feature. This feature relies on directly accessing the
Kubernetes API through a proxy server from your browser. Enabling networking security to block the Kubernetes API from
this proxy will block this traffic.
NOTE
This feature is available in all Azure regions, including Azure China. It is currently not available in Azure US Government.
For help setting up or troubleshooting the Live Data (preview ) feature, review our setup guide. This feature
directly access the Kubernetes API, and additional information about the authentication model can be found here.
The Live Data (preview ) feature includes search functionality. In the Search field, you can filter results by typing a
key word or term and any matching results are highlighted to allow quick review. While viewing events, you can
additionally limit the results using the Filter pill found to the right of the search bar. Depending on what resource
you have selected, the pill lists a Pod, Namespace, or cluster to chose from.
Scroll Lock and Pause
To suspend autoscroll and control the behavior of the pane, allowing you to manually scroll through the new data
read, you can use the Scroll option. To re-enable autoscroll, simply select the Scroll option again. You can also
pause retrieval of log or event data by selecting the the Pause option, and when you are ready to resume, simply
select Play.
IMPORTANT
We recommend only suspending or pausing autoscroll for a short period of time while troubleshooting an issue. These
requests may impact the availability and throttling of the Kubernetes API on your cluster.
IMPORTANT
No data is stored permanently during operation of this feature. All information captured during the session is deleted when
you close your browser or navigate away from it. Data only remains present for visualization inside the five minute window
of the metrics feature; any metrics older than five minutes are also deleted. The Live Data (preview) buffer queries within
reasonable memory usage limits (need to be more specific here, what is reasonable?).
View logs
You can view real-time log data as they are generated by the container engine from the Nodes, Controllers, and
Containers view. To view log data, perform the following steps.
1. In the Azure portal, browse to the AKS cluster resource group and select your AKS resource.
2. On the AKS cluster dashboard, under Monitoring on the left-hand side, choose Insights.
3. Select either the Nodes, Controllers, or Containers tab.
4. Select an object from the performance grid, and on the properties pane found on the right side, select View
live data (preview) option. If the AKS cluster is configured with single sign-on using Azure AD, you are
prompted to authenticate on first use during that browser session. Select your account and complete
authentication with Azure.
NOTE
When viewing the data from your Log Analytics workspace by selecting the View in analytics option from the
properties pane, the log search results will potentially show Nodes, Daemon Sets, Replica Sets, Jobs, Cron Jobs,
Pods, and Containers which may no longer exist. Attempting to search logs for a container which isn’t available in
kubectl will also fail here. Review the View in analytics feature to learn more about viewing historical logs, events
and metrics.
After successfully authenticating, the Live Data (preview ) console pane will appear below the performance data
grid where you can view log data in a continuous stream. If the fetch status indicator shows a green check mark,
which is on the far right of the pane, it means data can be retrieved and it begins streaming to your console.
The pane title shows the name of the pod the container is grouped with.
View events
You can view real-time event data as they are generated by the container engine from the Nodes, Controllers,
Containers, and Deployments (preview) view when a container, pod, node, ReplicaSet, DaemonSet, job,
CronJob or Deployment is selected. To view events, perform the following steps.
1. In the Azure portal, browse to the AKS cluster resource group and select your AKS resource.
2. On the AKS cluster dashboard, under Monitoring on the left-hand side, choose Insights.
3. Select either the Nodes, Controllers, Containers, or Deployments (preview) tab.
4. Select an object from the performance grid, and on the properties pane found on the right side, select View
live data (preview) option. If the AKS cluster is configured with single sign-on using Azure AD, you are
prompted to authenticate on first use during that browser session. Select your account and complete
authentication with Azure.
NOTE
When viewing the data from your Log Analytics workspace by selecting the View in analytics option from the
properties pane, the log search results will potentially show Nodes, Daemon Sets, Replica Sets, Jobs, Cron Jobs,
Pods, and Containers which may no longer exist. Attempting to search logs for a container which isn’t available in
kubectl will also fail here. Review the View in analytics feature to learn more about viewing historical logs, events
and metrics.
After successfully authenticating, the Live Data (preview ) console pane will appear below the performance data
grid. If the fetch status indicator shows a green check mark, which is on the far right of the pane, it means data can
be retrieved and it begins streaming to your console.
If the object you selected was a container, select the Events option in the pane. If you selected a Node, Pod, or
controller, viewing events is automatically selected.
The pane title shows the name of the Pod the container is grouped with.
Filter events
While viewing events, you can additionally limit the results using the Filter pill found to the right of the search bar.
Depending on what resource you have selected, the pill lists a Pod, Namespace, or cluster to chose from.
View metrics
You can view real-time metric data as they are generated by the container engine from the Nodes or Controllers
view only when a Pod is selected. To view metrics, perform the following steps.
1. In the Azure portal, browse to the AKS cluster resource group and select your AKS resource.
2. On the AKS cluster dashboard, under Monitoring on the left-hand side, choose Insights.
3. Select either the Nodes or Controllers tab.
4. Select a Pod object from the performance grid, and on the properties pane found on the right side, select
View live data (preview) option. If the AKS cluster is configured with single sign-on using Azure AD, you
are prompted to authenticate on first use during that browser session. Select your account and complete
authentication with Azure.
NOTE
When viewing the data from your Log Analytics workspace by selecting the View in analytics option from the
properties pane, the log search results will potentially show Nodes, Daemon Sets, Replica Sets, Jobs, Cron Jobs,
Pods, and Containers which may no longer exist. Attempting to search logs for a container which isn’t available in
kubectl will also fail here. Review the View in analytics feature to learn more about viewing historical logs, events
and metrics.
After successfully authenticating, the Live Data (preview ) console pane will appear below the performance data
grid. Metric data is retrieved and begins streaming to your console for presentation in the two charts. The pane
title shows the name of the pod the container is grouped with.
Next steps
To continue learning how to use Azure Monitor and monitor other aspects of your AKS cluster, see View
Azure Kubernetes Service health.
View log query examples to see predefined queries and examples to create alerts, visualizations, or perform
further analysis of your clusters.
How to setup the Live Data (preview) feature
12/13/2019 • 5 minutes to read • Edit Online
To view Live Data (preview ) with Azure Monitor for containers from Azure Kubernetes Service (AKS ) clusters, you
need to configure authentication to grant permission to access to your Kubernetes data. This security configuration
allows real time access to your data through the Kubernetes API directly in the Azure portal.
This feature supports three different methods to control access to the logs, events, and metrics:
AKS without Kubernetes RBAC authorization enabled
AKS enabled with Kubernetes RBAC authorization
AKS enabled with Azure Active Directory (AD ) SAML -based single-sign on
These instructions require both administrative access to your Kubernetes cluster, and if configuring to use Azure
Active Directory (AD ) for user authentication, administrative access to Azure AD.
This article explains how to configure authentication to control access to the Live Data (preview ) feature from the
cluster:
Role based access control (RBAC ) enabled AKS cluster
Azure Active Directory integrated AKS cluster.
NOTE
AKS clusters enabled as private clusters are not supported with this feature. This feature relies on directly accessing the
Kubernetes API through a proxy server from your browser. Enabling networking security to block the Kubernetes API from
this proxy will block this traffic.
NOTE
This feature is available in all Azure regions, including Azure China. It is currently not available in Azure US Government.
Authentication model
The Live Data (preview ) features utilizes the Kubernetes API, identical to the kubectl command-line tool. The
Kubernetes API endpoints utilize a self-signed certificate, which your browser will be unable to validate. This
feature utilizes a internal proxy to validate the certificate with the AKS service, ensuring the traffic is trusted.
The Azure portal prompts you to validate your login credentials for an Azure Active Directory cluster, and redirect
you to the client registration setup during cluster creation (and re-configured in this article). This behavior is
similar to the authentication process required by kubectl .
NOTE
Authorization to your cluster is managed by Kubernetes and the security model it is configured with. Users accessing this
feature require permission to download the Kubernetes configuration (kubeconfig), similar to running
az aks get-credentials -n {your cluster name} -g {your resource group} . This configuration file contains the
authorization and authentication token for Azure Kubernetes Service Cluster User Role, in the case of Azure RBAC-
enabled and AKS clusters without RBAC authorization enabled. It contains information about Azure AD and client registration
details when AKS is enabled with Azure Active Directory (AD) SAML-based single-sign on.
IMPORTANT
Users of this features requires [Azure Kubernetes Cluster User Role](../../azure/role-based-access-control/built-in-
roles.md#azure-kubernetes-service-cluster-user-role permissions) to the cluster in order to download the kubeconfig and
use this feature. Users do not require contributor access to the cluster to utilize this feature.
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: containerHealth-log-reader
rules:
- apiGroups: ["", "metrics.k8s.io", "extensions", "apps"]
resources:
- "pods/log"
- "events"
- "nodes"
- "pods"
- "deployments"
- "replicasets"
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: containerHealth-read-logs-global
roleRef:
kind: ClusterRole
name: containerHealth-log-reader
apiGroup: rbac.authorization.k8s.io
subjects:
- kind: User
name: clusterUser
apiGroup: rbac.authorization.k8s.io
2. To update your configuration, run the following command: kubectl apply -f LogReaderRBAC.yaml .
NOTE
If you have applied a previous version of the LogReaderRBAC.yaml file to your cluster, update it by copying and pasting the
new code shown in step 1 above, and then run the command shown in step 2 to apply it to your cluster.
Configure AD integrated authentication
An AKS cluster configured to use Azure Active Directory (AD ) for user authentication utilizes the login credentials
of the person accessing this feature. In this configuration, you can sign in to an AKS cluster by using your Azure
AD authentication token.
Azure AD client registration must be reconfigured to allow the Azure portal to redirect authorization pages as a
trusted redirect URL. Users from Azure AD are then granted access directly to the same Kubernetes API endpoints
through ClusterRoles and ClusterRoleBindings.
For more information on advanced security setup in Kubernetes, review the Kubernetes documentation.
NOTE
If you are creating a new RBAC-enabled cluster, see Integrate Azure Active Directory with Azure Kubernetes Service and
follow the steps to configure Azure AD authentication. During the steps to create the client application, a note in that section
highlights the two redirect URLs you need to create for Azure Monitor for containers.
NOTE
If you're using this feature in Azure China, the first base URL value should be
https://afd.hosting.azureportal.chinaloudapi.cn/monitoring/Content/iframe/infrainsights.app/web/base-
libs/auth/auth.html
and the second base URL value should be
https://monitoring.hosting.azureportal.chinaloudapi.cn/monitoring/Content/iframe/infrainsights.app/web/base-
libs/auth/auth.html
.
4. After registering the redirect URLs, under Advanced settings, select the options Access tokens and ID
tokens and then save your changes.
NOTE
Configuring authentication with Azure Active Directory for single-sign on can only be accomplished during initial deployment
of a new AKS cluster. You cannot configure single-sign on for an AKS cluster already deployed.
IMPORTANT
If you reconfigured Azure AD for user authentication using the updated URI, clear your browser's cache to ensure the
updated authentication token is downloaded and applied.
Grant permission
Each Azure AD account must be granted permission to the appropriate APIs in Kubernetes in order to access the
Live Data (preview ) feature. The steps to grant the Azure Active Directory account are similar to the steps
described in the Kubernetes RBAC authentication section. Before applying the yaml configuration template to your
cluster, replace clusterUser under ClusterRoleBinding with the desired user.
IMPORTANT
If the user you grant the RBAC binding for is in the same Azure AD tenant, assign permissions based on the
userPrincipalName. If the user is in a different Azure AD tenant, query for and use the objectId property.
For additional help configuring your AKS cluster ClusterRoleBinding, see Create RBAC binding.
Next steps
Now that you have setup authentication, you can view metrics, Deployments, and events and logs in real-time
from your cluster.
How to view metrics in real-time
12/13/2019 • 3 minutes to read • Edit Online
Azure Monitor for containers Live Data (preview ) feature allows you to visualize metrics about node and pod state
in a cluster in real-time. It emulates direct access to the kubectl top nodes , kubectl get pods –all-namespaces , and
kubectl get nodes commands to call, parse, and visualize the data in performance charts that are included with
this Insight.
This article provides a detailed overview and helps you understand how to use this feature.
NOTE
AKS clusters enabled as private clusters are not supported with this feature. This feature relies on directly accessing the
Kubernetes API through a proxy server from your browser. Enabling networking security to block the Kubernetes API from
this proxy will block this traffic.
NOTE
This feature is available in all Azure regions, including Azure China. It is currently not available in Azure US Government.
For help with setting up or troubleshooting the Live Data (preview ) feature, review our setup guide.
How it Works
The Live Data (preview ) feature directly access the Kubernetes API, and additional information about the
authentication model can be found here.
This feature performs a polling operation against the metrics endpoints (including /api/v1/nodes ,
/apis/metrics.k8s.io/v1beta1/nodes , and /api/v1/pods ), which is every five seconds by default. This data is cached
in your browser and charted in the four performance charts included in Azure Monitor for containers on the
Cluster tab by selecting Go Live (preview). Each subsequent poll is charted into a rolling five-minute
visualization window.
The polling interval is configured from the Set interval drop-down allowing you to set polling for new data every
1, 5, 15 and 30 seconds.
IMPORTANT
We recommend setting the polling interval to one second while troubleshooting an issue for a short period of time. These
requests may impact the availability and throttling of the Kubernetes API on your cluster. Afterwards, reconfigure to a longer
polling interval.
IMPORTANT
No data is stored permanently during operation of this feature. All information captured during this session is immediately
deleted when you close your browser or navigate away from the feature. Data only remains present for visualization inside
the five minute window; any metrics older than five minutes are also permanently deleted.
These charts cannot be pinned to the last Azure dashboard you viewed in live mode.
Metrics captured
Node CPU utilization % / Node Memory utilization %
These two performance charts map to an equivalent of invoking kubectl top nodes and capturing the results of
the CPU% and MEMORY% columns to the respective chart.
The percentile calculations will function in larger clusters to help identify outlier nodes in your cluster. For example,
to understand if nodes are under-utilized for scale down purposes. Utilizing the Min aggregation you can see
which nodes have low utilization in the cluster. To further investigate, you select the Nodes tab and sort the grid by
CPU or memory utilization.
This also helps you understand which nodes are being pushed to their limits and if scale-out may be required.
Utilizing both the Max and P95 aggregations can help you see if there are nodes in the cluster with high resource
utilization. For further investigation, you would again switch to the Nodes tab.
Node count
This performance chart maps to an equivalent of invoking kubectl get nodes and mapping the STATUS column
to a chart grouped by status types.
Nodes are reported either in a Ready or Not Ready state. They are counted (and a total count is created), and the
results of these two aggregations are charted. For example, to understand if your nodes are falling into failed
states. Utilizing the Not Ready aggregation you can quickly see the number of nodes in your cluster currently in
the Not Ready state.
Active pod count
This performance chart maps to an equivalent of invoking kubectl get pods –all-namespaces and maps the
STATUS column the chart grouped by status types.
NOTE
Names of status as interpreted by kubectl may not exactly match in the chart.
Next steps
View log query examples to see predefined queries and examples to create alerts, visualizations, or perform
further analysis of your clusters.
How to view Deployments (preview) in real-time
12/13/2019 • 2 minutes to read • Edit Online
With Azure Monitor for containers, the view Deployments (preview ) feature emulates direct access to Kubernetes
Deployment objects in real time by exposing the kubeclt get deployments and
kubectl describe deployment {your deployment} commands.
NOTE
AKS clusters enabled as private clusters are not supported with this feature. This feature relies on directly accessing the
Kubernetes API through a proxy server from your browser. Enabling networking security to block the Kubernetes API from
this proxy will block this traffic.
NOTE
This feature is available in all Azure regions, including Azure China. It is currently not available in Azure US Government.
How it works
The Live Data (preview ) feature directly access the Kubernetes API, and additional information about the
authentication model can be found here.
The Deployments (preview ) feature performs a one time (refreshable) load against the deployments endpoint
/apis/apps/v1/deployments . It allows you to select a given deployment and load the describe details for that specific
deployment against the deployment endpoint
/apis/apps/v1/namespaces/${nameSpace}/deployments/${deploymentName} .
Selecting Refresh in the top left of the page refreshes the deployment list. This simulates re-running the kubectl
command.
IMPORTANT
No data is stored permanently during operation of this feature. All information captured during the session is deleted when
you close your browser or navigate away from it.
NOTE
You cannot pin Live Data (Preview) data from the console to an Azure dashboard.
Deployments describe
To view Describe details for a deployment, which is the equivalent to kubectl describe deployment , perform the
following steps.
1. In the Azure portal, browse to the AKS cluster resource group and select your AKS resource.
2. On the AKS cluster dashboard, under Monitoring on the left-hand side, choose Insights.
3. Select the Deployments (preview) tab.
The view shows a list of all the running deployments along with the namespace and other detailed information,
emulating execution of the command kubectl get deployments –all-namespaces . You can sort the results by
selecting any one of the columns.
When you select a deployment from the list, a property pane automatically displays on the right side of the page. It
shows information related to the selected deployment that you would view if you ran the command
kubectl describe deployment {deploymentName} . You may have noticed that the describe information is missing
some details. Most notably the Template is missing. Selecting the Raw tab allows you to navigate to the un-
parsed Describe details.
While you review deployment details, you can see container logs and events in real time. Select the Show live
console and the Live Data (preview ) console pane will appear below the deployments data grid where you can
view live log data in a continuous stream. If the fetch status indicator shows a green check mark, which is on the far
right of the pane, it means data can be retrieved and it begins streaming to your console.
You can also filter by namespace or cluster level events. To learn more about the viewing data real-time in the
console, see View Live Data (preview ) with Azure Monitor for containers.
Next steps
To continue learning how to use Azure Monitor and monitor other aspects of your AKS cluster, see View
Azure Kubernetes Service health.
View log query examples to see predefined queries and examples to create alerts, visualizations, or perform
further analysis of your clusters.
How to query logs from Azure Monitor for containers
12/12/2019 • 4 minutes to read • Edit Online
Azure Monitor for containers collects performance metrics, inventory data, and health state information from
container hosts and containers, and forwards it to the Log Analytics workspace in Azure Monitor. Data is collected
every three minutes. This data is available for query in Azure Monitor. You can apply this data to scenarios that
include migration planning, capacity analysis, discovery, and on-demand performance troubleshooting.
Container records
Examples of records that are collected by Azure Monitor for containers and the data types that appear in log search
results are displayed in the following table:
Performance metrics for nodes part of Perf | where ObjectName == “K8SNode” Computer, ObjectName, CounterName
the Kubernetes cluster (cpuAllocatableBytes,
memoryAllocatableBytes,
cpuCapacityNanoCores,
memoryCapacityBytes,
memoryRssBytes, cpuUsageNanoCores,
memoryWorkingsetBytes,
restartTimeEpoch), CounterValue,
TimeGenerated, CounterPath,
SourceSystem
Performance metrics for containers part Perf | where ObjectName == CounterName ( cpuRequestNanoCores,
of the Kubernetes cluster “K8SContainer” memoryRequestBytes,
cpuLimitNanoCores,
memoryWorkingSetBytes,
restartTimeEpoch, cpuUsageNanoCores,
memoryRssBytes), CounterValue,
TimeGenerated, CounterPath,
SourceSystem
1 The Tags property represents multiple dimensions for the corresponding metric. For additional information about
the metrics collected and stored in the InsightsMetrics table and a description of the record properties, see
InsightsMetrics overview.
NOTE
Support for Prometheus is a feature in public preview at this time.
The container logs output that's forwarded to your workspace are STDOUT and STDERR. Because Azure Monitor
is monitoring Azure-managed Kubernetes (AKS ), Kube-system is not collected today because of the large volume
of generated data.
Example log search queries
It's often useful to build queries that start with an example or two and then modify them to fit your requirements.
To help build more advanced queries, you can experiment with the following sample queries:
QUERY DESCRIPTION
InsightsMetrics
| where Namespace == 'container.azm.ms/diskio'
| where TimeGenerated > ago(1h)
| where Name == 'reads'
| extend Tags = todynamic(Tags)
| extend HostName = tostring(Tags.hostName), Device = Tags.name
| extend NodeDisk = strcat(Device, "/", HostName)
| order by NodeDisk asc, TimeGenerated asc
| serialize
| extend PrevVal = iif(prev(NodeDisk) != NodeDisk, 0.0, prev(Val)), PrevTimeGenerated = iif(prev(NodeDisk) !=
NodeDisk, datetime(null), prev(TimeGenerated))
| where isnotnull(PrevTimeGenerated) and PrevTimeGenerated != TimeGenerated
| extend Rate = iif(PrevVal > Val, Val / (datetime_diff('Second', TimeGenerated, PrevTimeGenerated) * 1),
iif(PrevVal == Val, 0.0, (Val - PrevVal) / (datetime_diff('Second', TimeGenerated, PrevTimeGenerated) * 1)))
| where isnotnull(Rate)
| project TimeGenerated, NodeDisk, Rate
| render timechart
To view Prometheus metrics scraped by Azure Monitor filtered by Namespace, specify "prometheus". Here is a
sample query to view Prometheus metrics from the default kubernetes namespace.
InsightsMetrics
| where Namespace == "prometheus"
| extend tags=parse_json(Tags)
| summarize count() by Name
InsightsMetrics
| where Namespace == "prometheus"
| where Name contains "some_prometheus_metric"
Azure Monitor for containers monitors the performance of container workloads that are deployed to Azure
Container Instances or to managed Kubernetes clusters that are hosted on Azure Kubernetes Service (AKS ).
This article describes how to enable alerts for the following situations:
When CPU or memory utilization on cluster nodes exceeds a threshold
When CPU or memory utilization on any container within a controller exceeds a threshold as compared to a
limit that's set on the corresponding resource
NotReady status node counts
Failed, Pending, Unknown, Running, or Succeeded pod-phase counts
When free disk space on cluster nodes exceeds a threshold
To alert for high CPU or memory utilization, or low free disk space on cluster nodes, use the queries that are
provided to create a metric alert or a metric measurement alert. Metric alerts have lower latency than log alerts.
But log alerts provide advanced querying and greater sophistication. Log alerts queries compare a datetime to the
present by using the now operator and going back one hour. (Azure Monitor for containers stores all dates in
Coordinated Universal Time (UTC ) format.)
If you're not familiar with Azure Monitor alerts, see Overview of alerts in Microsoft Azure before you start. To
learn more about alerts that use log queries, see Log alerts in Azure Monitor. For more about metric alerts, see
Metric alerts in Azure Monitor.
The following query calculates average memory utilization as an average of member nodes' memory utilization
every minute.
let endDateTime = now();
let startDateTime = ago(1h);
let trendBinSize = 1m;
let capacityCounterName = 'memoryCapacityBytes';
let usageCounterName = 'memoryRssBytes';
KubeNodeInventory
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
// cluster filter would go here if multiple clusters are reporting to the same Log Analytics workspace
| distinct ClusterName, Computer
| join hint.strategy=shuffle (
Perf
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where ObjectName == 'K8SNode'
| where CounterName == capacityCounterName
| summarize LimitValue = max(CounterValue) by Computer, CounterName, bin(TimeGenerated, trendBinSize)
| project Computer, CapacityStartTime = TimeGenerated, CapacityEndTime = TimeGenerated + trendBinSize,
LimitValue
) on Computer
| join kind=inner hint.strategy=shuffle (
Perf
| where TimeGenerated < endDateTime + trendBinSize
| where TimeGenerated >= startDateTime - trendBinSize
| where ObjectName == 'K8SNode'
| where CounterName == usageCounterName
| project Computer, UsageValue = CounterValue, TimeGenerated
) on Computer
| where TimeGenerated >= CapacityStartTime and TimeGenerated < CapacityEndTime
| project ClusterName, Computer, TimeGenerated, UsagePercent = UsageValue * 100.0 / LimitValue
| summarize AggregatedValue = avg(UsagePercent) by bin(TimeGenerated, trendBinSize), ClusterName
IMPORTANT
The following queries use the placeholder values <your-cluster-name> and <your-controller-name> to represent your
cluster and controller. Replace them with values specific to your environment when you set up alerts.
The following query calculates the average CPU utilization of all containers in a controller as an average of CPU
utilization of every container instance in a controller every minute. The measurement is a percentage of the limit
set up for a container.
let endDateTime = now();
let startDateTime = ago(1h);
let trendBinSize = 1m;
let capacityCounterName = 'cpuLimitNanoCores';
let usageCounterName = 'cpuUsageNanoCores';
let clusterName = '<your-cluster-name>';
let controllerName = '<your-controller-name>';
KubePodInventory
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where ClusterName == clusterName
| where ControllerName == controllerName
| extend InstanceName = strcat(ClusterId, '/', ContainerName),
ContainerName = strcat(controllerName, '/', tostring(split(ContainerName, '/')[1]))
| distinct Computer, InstanceName, ContainerName
| join hint.strategy=shuffle (
Perf
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where ObjectName == 'K8SContainer'
| where CounterName == capacityCounterName
| summarize LimitValue = max(CounterValue) by Computer, InstanceName, bin(TimeGenerated, trendBinSize)
| project Computer, InstanceName, LimitStartTime = TimeGenerated, LimitEndTime = TimeGenerated +
trendBinSize, LimitValue
) on Computer, InstanceName
| join kind=inner hint.strategy=shuffle (
Perf
| where TimeGenerated < endDateTime + trendBinSize
| where TimeGenerated >= startDateTime - trendBinSize
| where ObjectName == 'K8SContainer'
| where CounterName == usageCounterName
| project Computer, InstanceName, UsageValue = CounterValue, TimeGenerated
) on Computer, InstanceName
| where TimeGenerated >= LimitStartTime and TimeGenerated < LimitEndTime
| project Computer, ContainerName, TimeGenerated, UsagePercent = UsageValue * 100.0 / LimitValue
| summarize AggregatedValue = avg(UsagePercent) by bin(TimeGenerated, trendBinSize) , ContainerName
The following query calculates the average memory utilization of all containers in a controller as an average of
memory utilization of every container instance in a controller every minute. The measurement is a percentage of
the limit set up for a container.
let endDateTime = now();
let startDateTime = ago(1h);
let trendBinSize = 1m;
let capacityCounterName = 'memoryLimitBytes';
let usageCounterName = 'memoryRssBytes';
let clusterName = '<your-cluster-name>';
let controllerName = '<your-controller-name>';
KubePodInventory
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where ClusterName == clusterName
| where ControllerName == controllerName
| extend InstanceName = strcat(ClusterId, '/', ContainerName),
ContainerName = strcat(controllerName, '/', tostring(split(ContainerName, '/')[1]))
| distinct Computer, InstanceName, ContainerName
| join hint.strategy=shuffle (
Perf
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where ObjectName == 'K8SContainer'
| where CounterName == capacityCounterName
| summarize LimitValue = max(CounterValue) by Computer, InstanceName, bin(TimeGenerated, trendBinSize)
| project Computer, InstanceName, LimitStartTime = TimeGenerated, LimitEndTime = TimeGenerated +
trendBinSize, LimitValue
) on Computer, InstanceName
| join kind=inner hint.strategy=shuffle (
Perf
| where TimeGenerated < endDateTime + trendBinSize
| where TimeGenerated >= startDateTime - trendBinSize
| where ObjectName == 'K8SContainer'
| where CounterName == usageCounterName
| project Computer, InstanceName, UsageValue = CounterValue, TimeGenerated
) on Computer, InstanceName
| where TimeGenerated >= LimitStartTime and TimeGenerated < LimitEndTime
| project Computer, ContainerName, TimeGenerated, UsagePercent = UsageValue * 100.0 / LimitValue
| summarize AggregatedValue = avg(UsagePercent) by bin(TimeGenerated, trendBinSize) , ContainerName
The following query returns all nodes and counts that have a status of Ready and NotReady.
The following query returns pod phase counts based on all phases: Failed, Pending, Unknown, Running, or
Succeeded.
NOTE
To alert on certain pod phases, such as Pending, Failed, or Unknown, modify the last line of the query. For example, to alert
on FailedCount use:
| summarize AggregatedValue = avg(FailedCount) by bin(TimeGenerated, trendBinSize)
The following query returns cluster nodes disks which exceed 90% free space used. To get the cluster ID, first run
the following query and copy the value from the ClusterId property:
InsightsMetrics
| extend Tags = todynamic(Tags)
| project ClusterId = Tags['container.azm.ms/clusterId']
| distinct tostring(ClusterId)
let clusterId = '<cluster-id>';
let endDateTime = now();
let startDateTime = ago(1h);
let trendBinSize = 1m;
InsightsMetrics
| where TimeGenerated < endDateTime
| where TimeGenerated >= startDateTime
| where Origin == 'container.azm.ms/telegraf'
| where Namespace == 'container.azm.ms/disk'
| extend Tags = todynamic(Tags)
| project TimeGenerated, ClusterId = Tags['container.azm.ms/clusterId'], Computer = tostring(Tags.hostName),
Device = tostring(Tags.device), Path = tostring(Tags.path), DiskMetricName = Name, DiskMetricValue = Val
| where ClusterId =~ clusterId
| where DiskMetricName == 'used_percent'
| summarize AggregatedValue = max(DiskMetricValue) by bin(TimeGenerated, trendBinSize)
| where AggregatedValue >= 90
NOTE
The following procedure to create an alert rule for container resource utilization requires you to switch to a new log alerts
API as described in Switch API preference for log alerts.
Next steps
View log query examples to see pre-defined queries and examples to evaluate or customize for alerting,
visualizing, or analyzing your clusters.
To learn more about Azure Monitor and how to monitor other aspects of your Kubernetes cluster, see View
Kubernetes cluster performance and View Kubernetes cluster health.
How to manage the Azure Monitor for containers
agent
1/13/2020 • 3 minutes to read • Edit Online
Azure Monitor for containers uses a containerized version of the Log Analytics agent for Linux. After initial
deployment, there are routine or optional tasks you may need to perform during its lifecycle. This article details on
how to manually upgrade the agent and disable collection of environmental variables from a particular container.
NOTE
While you are performing this maintenance activity, the nodes in the cluster are not forwarding collected data, and
performance views will not show data between the time you remove the agent and install the new version.
To install the new version of the agent, follow the steps described in the enable monitoring using Azure CLI, to
complete this process.
After you've re-enabled monitoring, it might take about 15 minutes before you can view updated health metrics
for the cluster. To verify the agent upgraded successfully, run the command:
kubectl logs omsagent-484hw --namespace=kube-system
The status should resemble the following example, where the value for omi and omsagent should match the latest
version specified in the agent release history.
User@aksuser:~$ kubectl logs omsagent-484hw --namespace=kube-system
:
:
instance of Container_HostInventory
{
[Key] InstanceID=3a4407a5-d840-4c59-b2f0-8d42e07298c2
Computer=aks-nodepool1-39773055-0
DockerVersion=1.13.1
OperatingSystem=Ubuntu 16.04.3 LTS
Volume=local
Network=bridge host macvlan null overlay
NodeRole=Not Orchestrated
OrchestratorType=Kubernetes
}
Primary Workspace: b438b4f6-912a-46d5-9cb1-b44069212abc Status: Onboarded(OMSAgent Running)
omi 1.4.2.5
omsagent 1.6.0-163
docker-cimprov 1.0.0.31
- name: AZMON_COLLECT_ENV
value: "False"
Run the following command to apply the change to Kubernetes clusters other than Azure Red Hat OpenShift):
kubectl apply -f <path to yaml file> . To edit ConfigMap and apply this change for Azure Red Hat OpenShift
clusters, run the command:
This opens your default text editor. After setting the variable, save the file in the editor.
To verify the configuration change took effect, select a container in the Containers view in Azure Monitor for
containers, and in the property panel, expand Environment Variables. The section should show only the variable
created earlier - AZMON_COLLECT_ENV=FALSE. For all other containers, the Environment Variables section
should list all the environment variables discovered.
To re-enable discovery of the environmental variables, apply the same process earlier and change the value from
False to True, and then rerun the kubectl command to update the container.
- name: AZMON_COLLECT_ENV
value: "True"
Next steps
If you experience issues while upgrading the agent, review the troubleshooting guide for support.
Configure scraping of Prometheus metrics with Azure Monitor
for containers
1/13/2020 • 10 minutes to read • Edit Online
Prometheus is a popular open source metric monitoring solution and is a part of the Cloud Native Compute Foundation. Azure
Monitor for containers provides a seamless onboarding experience to collect Prometheus metrics. Typically, to use Prometheus, you
need to set up and manage a Prometheus server with a store. By integrating with Azure Monitor, a Prometheus server is not
required. You just need to expose the Prometheus metrics endpoint through your exporters or pods (application), and the
containerized agent for Azure Monitor for containers can scrape the metrics for you.
NOTE
The minimum agent version supported for scraping Prometheus metrics is ciprod07092019 or later, and the agent version supported for writing
configuration and agent errors in the KubeMonAgentEvents table is ciprod10112019. For more information about the agent versions and what's
included in each release, see agent release notes. To verify your agent version, from the Node tab select a node, and in the properties pane note
value of the Agent Image Tag property.
NOTE
For Azure Red Hat OpenShift, a template ConfigMap file is created in the openshift-azure-logging namespace. It is not configured to actively
scrape metrics or data collection from the agent.
oc get groups
If you are member of osa-customer-admins group, you should be able to list the container-azm-ms-agentconfig ConfigMap using the
following command:
When a URL is specified, Azure Monitor for containers only scrapes the endpoint. When Kubernetes service is specified, the service
name is resolved with the cluster DNS server to get the IP address and then the resolved service is scraped.
String
monitor_kubernetes_pods_namespaces Comma-separated array An allow list of
namespaces to scrape
metrics from Kubernetes
pods.
For example,
monitor_kubernetes_pods_namespaces
= ["default1", "default2",
"default3"]
Node-wide or Cluster- fieldpass String Comma-separated array You can specify certain
wide fielddrop metrics to be collected or
not from the endpoint by
setting the allow (
fieldpass ) and disallow
( fielddrop ) listing. You
must set the allow list
first.
ConfigMaps is a global list and there can be only one ConfigMap applied to the agent. You cannot have another ConfigMaps
overruling the collections.
NOTE
This step is not required when working with Azure Red Hat OpenShift since the ConfigMap template already exists on the cluster.
2. Edit the ConfigMap yaml file with your customizations to scrape Prometheus metrics. If you are editing the ConfigMap yaml
file for Azure Red Hat OpenShift, first run the command
oc edit configmaps container-azm-ms-agentconfig -n openshift-azure-logging to open the file in a text editor.
NOTE
The following annotation openshift.io/reconcile-protect: "true" must be added under the metadata of container-azm-ms-
agentconfig ConfigMap to prevent reconciliation.
metadata:
annotations:
openshift.io/reconcile-protect: "true"
To collect of Kubernetes services cluster-wide, configure the ConfigMap file using the following example.
prometheus-data-collection-settings: |-
# Custom Prometheus metrics data collection settings
[prometheus_data_collection_settings.cluster]
interval = "1m" ## Valid time units are s, m, h.
fieldpass = ["metric_to_pass1", "metric_to_pass12"] ## specify metrics to pass through
fielddrop = ["metric_to_drop"] ## specify metrics to drop from collecting
kubernetes_services = ["http://my-service-dns.my-namespace:9102/metrics"]
To configure scraping of Prometheus metrics from a specific URL across the cluster, configure the ConfigMap file using
the following example.
prometheus-data-collection-settings: |-
# Custom Prometheus metrics data collection settings
[prometheus_data_collection_settings.cluster]
interval = "1m" ## Valid time units are s, m, h.
fieldpass = ["metric_to_pass1", "metric_to_pass12"] ## specify metrics to pass through
fielddrop = ["metric_to_drop"] ## specify metrics to drop from collecting
urls = ["http://myurl:9101/metrics"] ## An array of urls to scrape metrics from
To configure scraping of Prometheus metrics from an agent's DaemonSet for every individual node in the cluster,
configure the following in the ConfigMap:
prometheus-data-collection-settings: |-
# Custom Prometheus metrics data collection settings
[prometheus_data_collection_settings.node]
interval = "1m" ## Valid time units are s, m, h.
urls = ["http://$NODE_IP:9103/metrics"]
fieldpass = ["metric_to_pass1", "metric_to_pass2"]
fielddrop = ["metric_to_drop"]
NOTE
$NODE_IP is a specific Azure Monitor for containers parameter and can be used instead of node IP address. It must be all
uppercase.
To configure scraping of Prometheus metrics by specifying a pod annotation, perform the following steps:
a. In the ConfigMap, specify the following:
prometheus-data-collection-settings: |-
# Custom Prometheus metrics data collection settings
[prometheus_data_collection_settings.cluster]
interval = "1m" ## Valid time units are s, m, h
monitor_kubernetes_pods = true
If you want to restrict monitoring to specific namespaces for pods that have annotations, for example only
include pods dedicated for production workloads, set the monitor_kubernetes_pod to true in ConfigMap, and
add the namespace filter monitor_kubernetes_pods_namespaces specifying the namespaces to scrape from. For
example, monitor_kubernetes_pods_namespaces = ["default1", "default2", "default3"]
3. For clusters other than Azure Red Hat OpenShift, run the following kubectl command:
kubectl apply -f <configmap_yaml_file.yaml> .
Verify configuration
To verify the configuration was successfully applied to a cluster, use the following command to review the logs from an agent pod:
kubectl logs omsagent-fdf58 -n=kube-system .
NOTE
This command is not applicable to Azure Red Hat OpenShift cluster.
If there are configuration errors from the omsagent pods, the output will show errors similar to the following:
Errors related to applying configuration changes are also available for review. The following options are available to perform
additional troubleshooting of configuration changes and scraping of Prometheus metrics:
From an agent pod logs using the same kubectl logs command
NOTE
This command is not applicable to Azure Red Hat OpenShift cluster.
From Live Data (preview ). Live Data (preview ) logs show errors similar to the following:
From the KubeMonAgentEvents table in your Log Analytics workspace. Data is sent every hour with Warning severity for
scrape errors and Error severity for configuration errors. If there are no errors, the entry in the table will have data with
severity Info, which reports no errors. The Tags property contains more information about the pod and container ID on which
the error occurred and also the first occurrence, last occurrence, and count in the last hour.
For Azure Red Hat OpenShift, check the omsagent logs by searching the ContainerLog table to verify if log collection of
openshift-azure-logging is enabled.
Errors prevent omsagent from parsing the file, causing it to restart and use the default configuration. After you correct the error(s) in
ConfigMap on clusters other than Azure Red Hat OpenShift, save the yaml file and apply the updated ConfigMaps by running the
command: kubectl apply -f <configmap_yaml_file.yaml .
For Azure Red Hat OpenShift, edit and save the updated ConfigMaps by running the command:
oc edit configmaps container-azm-ms-agentconfig -n openshift-azure-logging .
InsightsMetrics
| where Namespace == "prometheus"
| where TimeGenerated > ago(24h)
| summarize VolumeInGB = (sum(_BilledSize) / (1024 * 1024 * 1024)) by Name
| order by VolumeInGB desc
| render barchart
To estimate what each metrics size in GB is for a month to understand if the volume of data ingested received in the workspace is
high, the following query is provided.
InsightsMetrics
| where Namespace contains "prometheus"
| where TimeGenerated > ago(24h)
| summarize EstimatedGBPer30dayMonth = (sum(_BilledSize) / (1024 * 1024 * 1024)) * 30 by Name
| order by EstimatedGBPer30dayMonth desc
| render barchart
Next steps
Learn more about configuring the agent collection settings for stdout, stderr, and environmental variables from container workloads
here.
Configure agent data collection for Azure Monitor for
containers
1/13/2020 • 7 minutes to read • Edit Online
Azure Monitor for containers collects stdout, stderr, and environmental variables from container workloads deployed to managed
Kubernetes clusters from the containerized agent. You can configure agent data collection settings by creating a custom
Kubernetes ConfigMaps to control this experience.
This article demonstrates how to create ConfigMap and configure data collection based on your requirements.
NOTE
For Azure Red Hat OpenShift, a template ConfigMap file is created in the openshift-azure-logging namespace.
IMPORTANT
The minimum agent version supported to collect stdout, stderr, and environmental variables from container workloads is ciprod06142019 or
later. To verify your agent version, from the Node tab select a node, and in the properties pane note value of the Agent Image Tag property.
For additional information about the agent versions and what's included in each release, see agent release notes.
Boolean
[log_collection_settings.enrich_container_logs] true or false This setting controls container
enabled = log enrichment to populate the
Name and Image property values
for every log record written to
the ContainerLog table for all
container logs in the cluster. It
defaults to enabled = false
when not specified in ConfigMap.
ConfigMaps is a global list and there can be only one ConfigMap applied to the agent. You cannot have another ConfigMaps
overruling the collections.
NOTE
This step is not required when working with Azure Red Hat OpenShift since the ConfigMap template already exists on the cluster.
2. Edit the ConfigMap yaml file with your customizations to collect stdout, stderr, and/or environmental variables. If you are
editing the ConfigMap yaml file for Azure Red Hat OpenShift, first run the command
oc edit configmaps container-azm-ms-agentconfig -n openshift-azure-logging to open the file in a text editor.
To exclude specific namespaces for stdout log collection, you configure the key/value using the following example:
[log_collection_settings.stdout] enabled = true exclude_namespaces = ["my-namespace-1", "my-namespace-2"] .
To disable environment variable collection for a specific container, set the key/value
[log_collection_settings.env_var] enabled = true to enable variable collection globally, and then follow the steps
here to complete configuration for the specific container.
To disable stderr log collection cluster-wide, you configure the key/value using the following example:
[log_collection_settings.stderr] enabled = false .
3. For clusters other than Azure Red Hat OpenShift, create ConfigMap by running the following kubectl command:
kubectl apply -f <configmap_yaml_file.yaml> on clusters other than Azure Red Hat OpenShift.
Verify configuration
To verify the configuration was successfully applied to a cluster other than Azure Red Hat OpenShift, use the following command
to review the logs from an agent pod: kubectl logs omsagent-fdf58 -n=kube-system . If there are configuration errors from the
omsagent pods, the output will show errors similar to the following:
Errors related to applying configuration changes are also available for review. The following options are available to perform
additional troubleshooting of configuration changes:
From an agent pod logs using the same kubectl logs command.
NOTE
This command is not applicable to Azure Red Hat OpenShift cluster.
From Live logs. Live logs show errors similar to the following:
config::error::Exception while parsing config map for log collection/env variable settings: \nparse error on value \"$\"
($end), using defaults, please check config map for errors
From the KubeMonAgentEvents table in your Log Analytics workspace. Data is sent every hour with Error severity for
configuration errors. If there are no errors, the entry in the table will have data with severity Info, which reports no errors.
The Tags property contains more information about the pod and container ID on which the error occurred and also the first
occurrence, last occurrence and count in the last hour.
With Azure Red Hat OpenShift, check the omsagent logs by searching the ContainerLog table to verify if log collection of
openshift-azure-logging is enabled.
After you correct the error(s) in ConfigMap on clusters other than Azure Red Hat OpenShift, save the yaml file and apply the
updated ConfigMaps by running the command: kubectl apply -f <configmap_yaml_file.yaml . For Azure Red Hat OpenShift, edit
and save the updated ConfigMaps by running the command:
The configuration change can take a few minutes to finish before taking effect, and all omsagent pods in the cluster will restart. The
restart is a rolling restart for all omsagent pods, not all restart at the same time. When the restarts are finished, a message is
displayed that's similar to the following and includes the result: configmap "container-azm-ms-agentconfig" updated .
Name: omsagent-fdf58
Namespace: kube-system
Node: aks-agentpool-95673144-0/10.240.0.4
Start Time: Mon, 10 Jun 2019 15:01:03 -0700
Labels: controller-revision-hash=589cc7785d
dsName=omsagent-ds
pod-template-generation=1
Annotations: agentVersion=1.10.0.1
dockerProviderVersion=5.0.0-0
schema-versions=v1
Next steps
Azure Monitor for containers does not include a predefined set of alerts. Review the Create performance alerts with Azure
Monitor for containers to learn how to create recommended alerts for high CPU and memory utilization to support your
DevOps or operational processes and procedures.
With monitoring enabled to collect health and resource utilization of your AKS or hybrid cluster and workloads running on
them, learn how to use Azure Monitor for containers.
View log query examples to see pre-defined queries and examples to evaluate or customize for alerting, visualizing, or
analyzing your clusters.
How to update Azure Monitor for containers to
enable metrics
1/23/2020 • 13 minutes to read • Edit Online
Azure Monitor for containers is introducing support for collecting metrics from Azure Kubernetes Services (AKS )
clusters nodes and pods and writing them to the Azure Monitor metrics store. This change is intended to deliver
improved timeliness when presenting aggregate calculations (Avg, Count, Max, Min, Sum) in performance charts,
support pinning performance charts in Azure portal dashboards, and support metric alerts.
NOTE
This feature does not currently support Azure Red Hat OpenShift clusters.
Updating the cluster to support these new capabilities can be performed from the Azure portal, Azure PowerShell,
or with Azure CLI. With Azure PowerShell and CLI, you can enable this per-cluster or for all clusters in your
subscription. New deployments of AKS will automatically include this configuration change and capabilities.
Either process assigns the Monitoring Metrics Publisher role to the cluster’s service principal so that the data
collected by the agent can be published to your clusters resource. Monitoring Metrics Publisher has permission
only to push metrics to the resource, it cannot alter any state, update the resource, or read any data. For further
information about the role, see Monitoring Metrics Publisher role.
Prerequisites
Before you start, confirm the following:
Custom metrics are only available in a subset of Azure regions. A list of supported regions is documented here.
You are a member of the Owner role on the AKS cluster resource to enable collection of node and pod custom
performance metrics.
If you choose to use the Azure CLI, you first need to install and use the CLI locally. You must be running the Azure
CLI version 2.0.59 or later. To identify your version, run az --version . If you need to install or upgrade the Azure
CLI, see Install the Azure CLI.
Upgrade a cluster from the Azure portal
For existing AKS clusters monitored by Azure Monitor for containers, after selecting the cluster to view its health
from the multi-cluster view in Azure Monitor or directly from the cluster by selecting Insights from the left-hand
pane, you should see a banner at the top of the portal.
Clicking Enable will initiate the process to upgrade the cluster. This process can take several seconds to finish, and
you can track its progress under Notifications from the menu.
az login
az account set --subscription "Subscription Name"
curl -sL https://aka.ms/ci-md-onboard-atscale | bash -s subscriptionId
The configuration change can take a few seconds to complete. When it's completed, a message is displayed
that's similar to the following and includes the result:
<#
.DESCRIPTION
Adds the Monitoring Metrics Publisher role assignment to the all AKS clusters in specified
subscription
.PARAMETER SubscriptionId
Subscription Id that the AKS cluster is in
#>
param(
[Parameter(mandatory = $true)]
[string]$SubscriptionId
)
# checks the required Powershell modules exist and if not exists, request the user permission to install
$azAccountModule = Get-Module -ListAvailable -Name Az.Accounts
$azAksModule = Get-Module -ListAvailable -Name Az.Aks
$azResourcesModule = Get-Module -ListAvailable -Name Az.Resources
if (($null -eq $azAccountModule) -or ( $null -eq $azAksModule ) -or ($null -eq $azResourcesModule)) {
$currentPrincipal = New-Object
Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
if ($currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) {
Write-Host("Running script as an admin...")
Write-Host("")
}
else {
Write-Host("Please run the script as an administrator") -ForegroundColor Red
Stop-Transcript
exit
}
$message = "This script will try to install the latest versions of the following Modules : `
Az.Resources, Az.Accounts and Az.Aks using the command if not installed already`
`'Install-Module {Insert Module Name} -Repository PSGallery -Force -AllowClobber -ErrorAction
Stop -WarningAction Stop'
`If you do not have the latest version of these Modules, this troubleshooting script may not
run."
$question = "Do you want to Install the modules and run the script or just run the script?"
switch ($decision) {
0 {
}
1 {
}
2 {
Write-Host("")
Stop-Transcript
exit
}
}
}
try {
Write-Host("")
Write-Host("Trying to get the current Az login context...")
$account = Get-AzContext -ErrorAction Stop
Write-Host("Successfully fetched current AzContext context...") -ForegroundColor Green
Write-Host("")
}
catch {
Write-Host("")
Write-Host("Could not fetch AzContext..." ) -ForegroundColor Red
Write-Host("")
}
#
# get all the AKS clusters in specified subscription
#
Write-Host("getting all aks clusters in specified subscription ...")
$allClusters = Get-AzAks -ErrorVariable notPresent -ErrorAction SilentlyContinue
if ($notPresent) {
Write-Host("")
Write-Host("Failed to get Aks clusters in specified subscription. Please make sure that you have access
to the existing clusters") -ForegroundColor Red
Write-Host("")
Stop-Transcript
exit
}
Write-Host("Successfully got all aks clusters ...") -ForegroundColor Green
$clustersCount = $allClusters.Id.Length
#
# Add Monitoring Metrics Publisher role assignment to the AKS cluster resource
#
$servicePrincipalClientId = $allClusters.ServicePrincipalProfile[$index].ClientId
$clusterResourceId = $allClusters.Id[$index]
$clusterName = $allClusters.Name[$index]
if ($assignmentError) {
}
else {
Write-Host("Completed adding role assignment for the aks clusters in subscriptionId :$SubscriptionId")
.\onboard_metrics_atscale.ps1 subscriptionId
The configuration change can take a few seconds to complete. When it's completed, a message is displayed
that's similar to the following and includes the result:
Completed adding role assignment for the aks clusters in subscriptionId :<subscriptionId>
<#
.DESCRIPTION
Adds the Monitoring Metrics Publisher role assignment to the specified AKS cluster
.PARAMETER SubscriptionId
Subscription Id that the AKS cluster is in
.PARAMETER resourceGroupName
Resource Group name that the AKS cluster is in
.PARAMETER clusterName
Name of the AKS cluster.
#>
param(
[Parameter(mandatory = $true)]
[string]$SubscriptionId,
[Parameter(mandatory = $true)]
[string]$resourceGroupName,
[Parameter(mandatory = $true)]
[string] $clusterName
)
# checks the required Powershell modules exist and if not exists, request the user permission to install
$azAccountModule = Get-Module -ListAvailable -Name Az.Accounts
$azAksModule = Get-Module -ListAvailable -Name Az.Aks
$azResourcesModule = Get-Module -ListAvailable -Name Az.Resources
if (($null -eq $azAccountModule) -or ($null -eq $azAksModule) -or ($null -eq $azResourcesModule)) {
$currentPrincipal = New-Object
Security.Principal.WindowsPrincipal([Security.Principal.WindowsIdentity]::GetCurrent())
if ($currentPrincipal.IsInRole([Security.Principal.WindowsBuiltInRole]::Administrator)) {
Write-Host("Running script as an admin...")
Write-Host("")
}
}
else {
Write-Host("Please run the script as an administrator") -ForegroundColor Red
Stop-Transcript
exit
}
$message = "This script will try to install the latest versions of the following Modules : `
Az.Resources, Az.Accounts and Az.Aks using the command`
`'Install-Module {Insert Module Name} -Repository PSGallery -Force -AllowClobber -ErrorAction
Stop -WarningAction Stop'
`If you do not have the latest version of these Modules, this troubleshooting script may not
run."
$question = "Do you want to Install the modules and run the script or just run the script?"
switch ($decision) {
0 {
}
1 {
if ($null -eq $azResourcesModule) {
try {
Import-Module Az.Resources -ErrorAction Stop
}
catch {
Write-Host("Could not import Az.Resources...") -ForegroundColor Red
Write-Host("Close other powershell logins and try installing the latest modules for
Az.Resources in a new powershell window: eg. 'Install-Module Az.Resources -Repository PSGallery -
Force'") -ForegroundColor Red
Stop-Transcript
exit
}
}
if ($null -eq $azAccountModule) {
try {
Import-Module Az.Accounts -ErrorAction Stop
}
catch {
Write-Host("Could not import Az.Accounts...") -ForegroundColor Red
Write-Host("Close other powershell logins and try installing the latest modules for
Az.Accounts in a new powershell window: eg. 'Install-Module Az.Accounts -Repository PSGallery -Force'")
-ForegroundColor Red
Stop-Transcript
exit
}
}
if ($null -eq $azAksModule) {
try {
Import-Module Az.Aks -ErrorAction Stop
}
catch {
Write-Host("Could not import Az.Aks... Please reinstall this Module") -ForegroundColor
Red
Stop-Transcript
exit
}
}
}
2 {
Write-Host("")
Stop-Transcript
exit
}
}
}
try {
Write-Host("")
Write-Host("Trying to get the current Az login context...")
$account = Get-AzContext -ErrorAction Stop
Write-Host("Successfully fetched current AzContext context...") -ForegroundColor Green
Write-Host("")
}
catch {
Write-Host("")
Write-Host("Could not fetch AzContext..." ) -ForegroundColor Red
Write-Host("")
}
#
# Check AKS cluster existence and access check
#
Write-Host("Checking aks cluster exists...")
$cluster = Get-AzAks -ResourceGroupName $resourceGroupName -Name $clusterName -ErrorVariable notPresent
-ErrorAction SilentlyContinue
if ($notPresent) {
Write-Host("")
Write-Host("Could not find Aks cluster. Please make sure that specified cluster exists: '" +
$clusterName + "'is correct and you have access to the cluster") -ForegroundColor Red
Write-Host("")
Stop-Transcript
exit
}
Write-Host("Successfully checked specified cluster exists details...") -ForegroundColor Green
$servicePrincipalClientId = $cluster.ServicePrincipalProfile.clientId
$clusterResourceId = $cluster.Id
#
# Add Monitoring Metrics Publisher role assignment to the AKS cluster resource
#
if ($assignmentError) {
}
else {
The configuration change can take a few seconds to complete. When it's completed, a message is displayed
that's similar to the following and includes the result:
Verify update
After initiating the update using one of the methods described earlier, you can use Azure Monitor metrics explorer
and verify from the Metric namespace that insights is listed. If it is, this indicates you can go ahead and start
setting up metric alerts or pinning your charts to dashboards.
Azure Monitor for containers health monitor
configuration guide
12/12/2019 • 6 minutes to read • Edit Online
Monitors are the primary element for measuring health and detecting errors in Azure Monitor for containers. This
article helps you understand the concepts of how health is measured and the elements that comprise the health
model to monitor and report on the health of your Kubernetes cluster with the Health (preview ) feature.
NOTE
The Health feature is in public preview at this time.
Monitors
A monitor measures the health of some aspect of a managed object. Monitors each have either two or three health
states. A monitor will be in one and only one of its potential states at any given time. When a monitor loaded by
the containerized agent, it is initialized to a healthy state. The state changes only if the specified conditions for
another state are detected.
The overall health of a particular object is determined from the health of each of its monitors. This hierarchy is
illustrated in the Health Hierarchy pane in Azure Monitor for containers. The policy for how health is rolled up is
part of the configuration of the aggregate monitors.
Types of monitors
MONITOR DESCRIPTION
Aggregate monitors
MONITOR NAME DESCRIPTION ALGORITHM
Nodes (parent of Node pool) This is an aggregate monitor of all the Worst of
node pools. Its state is based on the
worst state of its child monitors (that is,
the node pools present in the cluster).
Cluster (parent of nodes/ This is the parent monitor that matches Worst of
Kubernetes infrastructure) the state of the child monitor with the
worst health state, that is kubernetes
infrastructure and nodes.
MONITOR NAME DESCRIPTION ALGORITHM
Next steps
View monitor cluster health to learn about viewing the health status your Kubernetes cluster.
How to stop monitoring your Azure Kubernetes
Service (AKS) with Azure Monitor for containers
1/23/2020 • 4 minutes to read • Edit Online
After you enable monitoring of your AKS cluster, you can stop monitoring the cluster if you decide you no longer
want to monitor it. This article shows how to accomplish this using the Azure CLI or with the provided Azure
Resource Manager templates.
Azure CLI
Use the az aks disable-addons command to disable Azure Monitor for containers. The command removes the
agent from the cluster nodes, it does not remove the solution or the data already collected and stored in your
Azure Monitor resource.
To re-enable monitoring for your cluster, see Enable monitoring using Azure CLI.
NOTE
The template needs to be deployed in the same resource group of the cluster. If you omit any other properties or add-ons
when using this template, it can result in their removal from the cluster. For example, enableRBAC for RBAC policies
implemented in your cluster, or aksResourceTagValues if tags are specified for the AKS cluster.
If you choose to use the Azure CLI, you first need to install and use the CLI locally. You must be running the Azure
CLI version 2.0.27 or later. To identify your version, run az --version . If you need to install or upgrade the Azure
CLI, see Install the Azure CLI.
Create template
1. Copy and paste the following JSON syntax into your file:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"aksResourceId": {
"type": "string",
"metadata": {
"description": "AKS Cluster Resource ID"
}
},
"aksResourceLocation": {
"type": "string",
"metadata": {
"description": "Location of the AKS resource e.g. \"East US\""
}
},
"aksResourceTagValues": {
"type": "object",
"metadata": {
"description": "Existing all tags on AKS Cluster Resource"
}
}
},
"resources": [
{
"name": "[split(parameters('aksResourceId'),'/')[8]]",
"type": "Microsoft.ContainerService/managedClusters",
"location": "[parameters('aksResourceLocation')]",
"tags": "[parameters('aksResourceTagValues')]",
"apiVersion": "2018-03-31",
"properties": {
"mode": "Incremental",
"id": "[parameters('aksResourceId')]",
"addonProfiles": {
"omsagent": {
"enabled": false,
"config": null
}
}
}
}
]
}
4. Edit the values for aksResourceId and aksResourceLocation by using the values of the AKS cluster, which
you can find on the Properties page for the selected cluster.
While you are on the Properties page, also copy the Workspace Resource ID. This value is required if you
decide you want to delete the Log Analytics workspace later. Deleting the Log Analytics workspace is not
performed as part of this process.
Edit the values for aksResourceTagValues to match the existing tag values specified for the AKS cluster.
5. Save this file as OptOutParam.json to a local folder.
6. You are ready to deploy this template.
Remove the solution using Azure CLI
Execute the following command with Azure CLI on Linux to remove the solution and clean up the configuration on
your AKS cluster.
az login
az account set --subscription "Subscription Name"
az group deployment create --resource-group <ResourceGroupName> --template-file ./OptOutTemplate.json --
parameters @./OptOutParam.json
The configuration change can take a few minutes to complete. When it's completed, a message similar to the
following that includes the result is returned:
ProvisioningState : Succeeded
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Execute the following PowerShell commands in the folder containing the template to remove the solution and
clean up the configuration from your AKS cluster.
Connect-AzAccount
Select-AzSubscription -SubscriptionName <yourSubscriptionName>
New-AzResourceGroupDeployment -Name opt-out -ResourceGroupName <ResourceGroupName> -TemplateFile
.\OptOutTemplate.json -TemplateParameterFile .\OptOutParam.json
The configuration change can take a few minutes to complete. When it's completed, a message similar to the
following that includes the result is returned:
ProvisioningState : Succeeded
Next steps
If the workspace was created only to support monitoring the cluster and it's no longer needed, you have to
manually delete it. If you are not familiar with how to delete a workspace, see Delete an Azure Log Analytics
workspace with the Azure portal. Don't forget about the Workspace Resource ID copied earlier in step 4, you're
going to need that.
How to stop monitoring your Azure Red Hat
OpenShift cluster with Azure Monitor for containers
1/14/2020 • 2 minutes to read • Edit Online
After you enable monitoring of your Azure Red Hat OpenShift cluster, you can stop monitoring the cluster if you
decide you no longer want to monitor it. This article shows how to accomplish this using the provided Azure
Resource Manager templates.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"aroResourceId": {
"value":
"/subscriptions/<subscriptionId>/resourceGroups/<ResourceGroupName>/providers/Microsoft.ContainerService
/openShiftManagedClusters/<clusterName>"
},
"aroResourceLocation": {
"value": "<azure region of the cluster e.g. westcentralus>"
}
}
}
4. Edit the values for aroResourceId and aroResourceLocation by using the values of the OpenShift cluster,
which you can find on the Properties page for the selected cluster.
5. Save this file as OptOutParam.json to a local folder.
6. You are ready to deploy this template.
Remove the solution using Azure CLI
Execute the following command with Azure CLI on Linux to remove the solution and clean up the configuration on
your cluster.
az login
az account set --subscription "Subscription Name"
az group deployment create --resource-group <ResourceGroupName> --template-file ./OptOutTemplate.json --
parameters @./OptOutParam.json
The configuration change can take a few minutes to complete. When it's completed, a message similar to the
following that includes the result is returned:
ProvisioningState : Succeeded
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Execute the following PowerShell commands in the folder containing the template to remove the solution and
clean up the configuration from your cluster.
Connect-AzAccount
Select-AzSubscription -SubscriptionName <yourSubscriptionName>
New-AzResourceGroupDeployment -Name opt-out -ResourceGroupName <ResourceGroupName> -TemplateFile
.\OptOutTemplate.json -TemplateParameterFile .\OptOutParam.json
The configuration change can take a few minutes to complete. When it's completed, a message similar to the
following that includes the result is returned:
ProvisioningState : Succeeded
Next steps
If the workspace was created only to support monitoring the cluster and it's no longer needed, you have to
manually delete it. If you are not familiar with how to delete a workspace, see Delete an Azure Log Analytics
workspace.
Troubleshooting Azure Monitor for containers
12/13/2019 • 5 minutes to read • Edit Online
When you configure monitoring of your Azure Kubernetes Service (AKS ) cluster with Azure Monitor for
containers, you may encounter an issue preventing data collection or reporting status. This article details some
common issues and troubleshooting steps.
The output should resemble the following, which indicates that it was deployed properly:
2. Check the deployment status with agent version 06072018 or later using the command:
kubectl get deployment omsagent-rs -n=kube-system
The output should resemble the following example, which indicates that it was deployed properly:
3. Check the status of the pod to verify that it is running using the command:
kubectl get pods --namespace=kube-system
The output should resemble the following example with a status of Running for the omsagent:
4. Check the agent logs. When the containerized agent gets deployed, it runs a quick check by running OMI
commands and displays the version of the agent and provider.
5. To verify that the agent has been deployed successfully, run the command:
kubectl logs omsagent-484hw --namespace=kube-system
Error messages
The table below summarizes known errors you may encounter while using Azure Monitor for containers.
Error Message No data for selected filters It may take some time to establish monitoring data flow for
newly created clusters. Allow at least 10 to 15 minutes for
data to appear for your cluster.
ERROR MESSAGES ACTION
Error Message Error retrieving data While Azure Kubernetes Service cluster is setting up for health
and performance monitoring, a connection is established
between the cluster and Azure Log Analytics workspace. A
Log Analytics workspace is used to store all monitoring data
for your cluster. This error may occur when your Log Analytics
workspace has been deleted. Check if the workspace was
deleted and if it was, you will need to re-enable monitoring of
your cluster with Azure Monitor for containers and specify an
existing or create a new workspace. To re-enable, you will
need to disable monitoring for the cluster and enable Azure
Monitor for containers again.
Error retrieving data after adding Azure Monitor for When enable monitoring using az aks cli , Azure Monitor
containers through az aks cli for containers may not be properly deployed. Check whether
the solution is deployed. To do this, go to your Log Analytics
workspace and see if the solution is available by selecting
Solutions from the pane on the left-hand side. To resolve this
issue, you will need to redeploy the solution by following the
instructions on how to deploy Azure Monitor for containers
To help diagnose the problem, we have provided a troubleshooting script available here.
Azure Monitor for containers agent ReplicaSet Pods are not scheduled
on non-Azure Kubernetes cluster
Azure Monitor for containers agent ReplicaSet Pods has a dependency on the following node selectors on the
worker (or agent) nodes for the scheduling:
nodeSelector:
beta.kubernetes.io/os: Linux
kubernetes.io/role: agent
If your worker nodes don’t have node labels attached, then agent ReplicaSet Pods will not get scheduled. Refer to
Kubernetes assign label selectors for instructions on how to attach the label.
Next steps
With monitoring enabled to capture health metrics for both the AKS cluster nodes and pods, these health metrics
are available in the Azure portal. To learn how to use Azure Monitor for containers, see View Azure Kubernetes
Service health.
Container Monitoring solution in Azure Monitor
10/24/2019 • 21 minutes to read • Edit Online
This article describes how to set up and use the Container Monitoring solution in Azure Monitor, which helps you
view and manage your Docker and Windows container hosts in a single location. Docker is a software
virtualization system used to create containers that automate software deployment to their IT infrastructure.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log
Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology
to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
The solution shows which containers are running, what container image they’re running, and where containers are
running. You can view detailed audit information showing commands used with containers. And, you can
troubleshoot containers by viewing and searching centralized logs without having to remotely view Docker or
Windows hosts. You can find containers that may be noisy and consuming excess resources on a host. And, you
can view centralized CPU, memory, storage, and network usage and performance information for containers. On
computers running Windows, you can centralize and compare logs from Windows Server, Hyper-V, and Docker
containers. The solution supports the following container orchestrators:
Docker Swarm
DC/OS
Kubernetes
Service Fabric
Red Hat OpenShift
If you have containers deployed in Azure Service Fabric, we recommend enabling both the Service Fabric solution
and this solution to include monitoring of cluster events. Before enabling the Service Fabric solution, review Using
the Service Fabric solution to understand what it provides and how to use it.
If you are interested in monitoring the performance of your workloads deployed to Kubernetes environments
hosted on Azure Kubernetes Service (AKS ), see Monitor Azure Kubernetes Service. The Container Monitoring
solution does not support monitoring that platform.
The following diagram shows the relationships between various container hosts and agents with Azure Monitor.
System requirements and supported platforms
Before starting, review the following details to verify you meet the prerequisites.
Container monitoring solution support for Docker Orchestrator and OS platform
The following table outlines the Docker orchestration and operating system monitoring support of container
inventory, performance, and logs with Azure Monitor.
CONTAI CONTAI
NER IMAGE NODE NER CONTAI CONTAI
WINDO INVENT INVENT INVENT PERFOR NER EVENT NER
ACS LINUX WS ORY ORY ORY MANCE EVENT LOG LOG
Kubern • • • • • • • • • •
etes
Mesos • • • • • • • • •
phere
DC/OS
Docker • • • • • • • • •
Swarm
Service • • • • • • • • •
Fabric
Red • • • • • • •
Hat
Open
Shift
Windo • • • • • • •
ws
Server
(standa
lone)
CONTAI CONTAI
NER IMAGE NODE NER CONTAI CONTAI
WINDO INVENT INVENT INVENT PERFOR NER EVENT NER
ACS LINUX WS ORY ORY ORY MANCE EVENT LOG LOG
Linux • • • • • • •
Server
(standa
lone)
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite to Azure Monitor, the Operations
Management Suite Agent for Windows or Linux will be referred to as the Log Analytics agent for Windows and Log Analytics
agent for Linux.
IMPORTANT
Docker must be running before you install the Log Analytics agent for Linux on your container hosts. If you've already
installed the agent before installing Docker, you need to reinstall the Log Analytics agent for Linux. For more information
about Docker, see the Docker website.
Se c u r e se c r e t s fo r D o c k e r Sw a r m
For Docker Swarm, once the secret for Workspace ID and Primary Key is created, use the following information to
create your secret information.
1. Run the following on the master node.
3. Run the following command to mount the secrets to the containerized Log Analytics agent.
If you want to use secrets to secure your Log Analytics Workspace ID and Primary Key when using the Log
Analytics agent daemon-set yaml file, perform the following steps.
1. Sign on to the OpenShift master node and copy the yaml file ocp-ds-omsagent.yaml and secret generating
script ocp-secretgen.sh from GitHub. This script will generate the secrets yaml file for Log Analytics
Workspace ID and Primary Key to secure your secrete information.
2. Run the following commands to create a project for Azure Monitor and set the user account. The secret
generating script asks for your Log Analytics Workspace ID <WSID> and Primary Key <KEY> and upon
completion, it creates the ocp-secret.yaml file.
5. Deploy the Log Analytics agent daemon-set yaml file by running the following:
oc create -f ocp-ds-omsagent.yaml
Type: Opaque
Data
====
KEY: 89 bytes
WSID: 37 bytes
Name: omsagent-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: Opaque
Data
====
WSID: 36 bytes
KEY: 88 bytes
2. Verify that the Log Analytics agent DaemonSet is running, similar to the following:
For Kubernetes, use a script to generate the secrets yaml file for Workspace ID and Primary Key for the Log
Analytics agent for Linux. Use the following example information with the omsagent yaml file to secure your secret
information.
Type: Opaque
Data
====
WSID: 36 bytes
KEY: 88 bytes
Type: Opaque
Data
====
WSID: 36 bytes
KEY: 88 bytes
2. Verify that the Log Analytics agent DaemonSet is running, similar to the following:
3. To install the agent on the Worker Node, which are running Windows, follow the steps in the section install
and configure Windows container hosts.
Use Helm to deploy Log Analytics agent on Linux Kubernetes
To use helm to deploy Log Analytics agent on your Linux Kubernetes environment, perform the following steps.
1. Create your omsagent daemon-set by running
helm install --name omsagent --set omsagent.secret.wsid=<WSID>,omsagent.secret.key=<KEY> stable/msoms
NAME: omsagent
LAST DEPLOYED: Tue Sep 19 20:37:46 2017
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Secret
NAME TYPE DATA AGE
omsagent-msoms Opaque 3 3s
==> v1beta1/DaemonSet
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE-SELECTOR AGE
omsagent-msoms 3 3 3 3 3 <none> 3s
3. You can check the status of the omsagent by running: helm status "omsagent" and the output will look
similar to the following:
keiko@k8s-master-3814F33-0:~$ helm status omsagent
LAST DEPLOYED: Tue Sep 19 20:37:46 2017
NAMESPACE: default
STATUS: DEPLOYED
RESOURCES:
==> v1/Secret
NAME TYPE DATA AGE
omsagent-msoms Opaque 3 17m
==> v1beta1/DaemonSet
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE-SELECTOR AGE
omsagent-msoms 3 3 3 3 3 <none> 17m
Perform the following PowerShell commands to enable TCP pipe and named pipe for Windows Server:
Stop-Service docker
dockerd --unregister-service
dockerd --register-service -H npipe:// -H 0.0.0.0:2375
Start-Service docker
For more information about the Docker daemon configuration used with Windows Containers, see Docker Engine
on Windows.
Install Windows agents
To enable Windows and Hyper-V container monitoring, install the Microsoft Monitoring Agent (MMA) on
Windows computers that are container hosts. For computers running Windows in your on-premises environment,
see Connect Windows computers to Azure Monitor. For virtual machines running in Azure, connect them to Azure
Monitor using the virtual machine extension.
You can monitor Windows containers running on Service Fabric. However, only virtual machines running in Azure
and computers running Windows in your on-premises environment are currently supported for Service Fabric.
You can verify that the Container Monitoring solution is set correctly for Windows. To check whether the
management pack was download properly, look for ContainerManagement.xxx. The files should be in the
C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Management Packs folder.
Solution components
From the Azure portal, navigate to the Solutions Gallery and add the Container Monitoring Solution. If you are
using Windows agents, then the following management pack is installed on each computer with an agent when
you add this solution. No configuration or maintenance is required for the management pack.
ContainerManagement.xxx installed in C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service
State\Management Packs
Container data collection details
The Container Monitoring solution collects various performance metrics and log data from container hosts and
containers using agents that you enable.
Data is collected every three minutes by the following agent types.
Log Analytics agent for Linux
Windows agent
Log Analytics VM extension
Container records
The following table shows examples of records collected by the Container Monitoring solution and the data types
that appear in log search results.
Labels appended to PodLabel data types are your own custom labels. The appended PodLabel labels shown in the
table are examples. So, PodLabel_deployment_s , PodLabel_deploymentconfig_s , PodLabel_docker_registry_s will
differ in your environment's data set and generically resemble PodLabel_yourlabel_s .
Monitor containers
After you have the solution enabled in the Azure portal, the Containers tile shows summary information about
your container hosts and the containers running in hosts.
The tile shows an overview of how many containers you have in the environment and whether they're failed,
running, or stopped.
Using the Containers dashboard
Click the Containers tile. From there you'll see views organized by:
Container Events - Shows container status and computers with failed containers.
Container Logs - Shows a chart of container log files generated over time and a list of computers with the
highest number of log files.
Kubernetes Events - Shows a chart of Kubernetes events generated over time and a list of the reasons why
pods generated the events. This data set is used only in Linux environments.
Kubernetes Namespace Inventory - Shows the number of namespaces and pods and shows their hierarchy.
This data set is used only in Linux environments.
Container Node Inventory - Shows the number of orchestration types used on container nodes/hosts. The
computer nodes/hosts are also listed by the number of containers. This data set is used only in Linux
environments.
Container Images Inventory - Shows the total number of container images used and number of image
types. The number of images are also listed by the image tag.
Containers Status - Shows the total number of container nodes/host computers that have running containers.
Computers are also listed by the number of running hosts.
Container Process - Shows a line chart of container processes running over time. Containers are also listed by
running command/process within containers. This data set is used only in Linux environments.
Container CPU Performance - Shows a line chart of the average CPU utilization over time for computer
nodes/hosts. Also lists the computer nodes/hosts based on average CPU utilization.
Container Memory Performance - Shows a line chart of memory usage over time. Also lists computer
memory utilization based on instance name.
Computer Performance - Shows line charts of the percent of CPU performance over time, percent of
memory usage over time, and megabytes of free disk space over time. You can hover over any line in a chart to
view more details.
Each area of the dashboard is a visual representation of a search that is run on collected data.
In the Container Status area, click the top area, as shown below.
Log Analytics opens, displaying information about the state of your containers.
From here, you can edit the search query to modify it to find the specific information you're interested in. For more
information about log queries, see Log queries in Azure Monitor.
3. Expand the Failed line and click + to add its criteria to the query. Then comment out the Summarize line in the
query.
4. Run the query and then expand a line in the results to view the image ID.
5. Type the following in the log query. ContainerImageInventory | where ImageID == <ImageID> to see details about
the image such as image size and number of stopped and failed images.
Query logs for container data
When you're troubleshooting a specific error, it can help to see where it is occurring in your environment. The
following log types will help you create queries to return the information you want.
ContainerImageInventory – Use this type when you're trying to find information organized by image and to
view image information such as image IDs or sizes.
ContainerInventory – Use this type when you want information about container location, what their names
are, and what images they're running.
ContainerLog – Use this type when you want to find specific error log information and entries.
ContainerNodeInventory_CL Use this type when you want the information about host/node where
containers are residing. It provides you Docker version, orchestration type, storage, and network information.
ContainerProcess_CL Use this type to quickly see the process running within the container.
ContainerServiceLog – Use this type when you're trying to find audit trail information for the Docker
daemon, such as start, stop, delete, or pull commands.
KubeEvents_CL Use this type to see the Kubernetes events.
KubePodInventory_CL Use this type when you want to understand the cluster hierarchy information.
To query logs for container data
Choose an image that you know has failed recently and find the error logs for it. Start by finding a container
name that is running that image with a ContainerInventory search. For example, search for
ContainerInventory | where Image == "ubuntu" and ContainerState == "Failed"
Expand any row in the results to view details for that container.
Next steps
Query logs to view detailed container data records.
Monitoring your storage service with Azure Monitor
for Storage (preview)
12/13/2019 • 14 minutes to read • Edit Online
Azure Monitor for Storage (preview ) provides comprehensive monitoring of your Azure Storage accounts by
delivering a unified view of your Azure Storage services performance, capacity, and availability. You can observe
storage capacity, and performance in two ways, view directly from a storage account or view from Azure Monitor
to see across groups of storage accounts.
This article will help you understand the experience Azure Monitor for Storage (preview ) delivers to derive
actionable knowledge on the health and performance of Storage accounts at scale, with a capability to focus on
hotspots and diagnose latency, throttling, and availability issues.
NOTE
There is no charge to access this feature and you will only be charged for the Azure Monitor essential features you configure
or enable, as described on the Azure Monitor pricing details page.
NOTE
Azure Monitor for Storage does not support general-purpose v1 accounts.
Overview workbook
On the Overview workbook for the selected subscription, the table displays interactive storage metrics and
service availability state for up to 10 storage accounts grouped within the subscription. You can filter the results
based on the options you select from the following drop-down lists:
Subscriptions - only subscriptions that have storage accounts are listed.
Storage Accounts - by default, 10 storage accounts are pre-selected. If you select all or multiple storage
accounts in the scope selector, up to 200 storage accounts will be returned. For example, if you had a total
of 573 storage accounts across three subscriptions that you've selected, only 200 accounts would be
displayed.
Time Range - by default, displays the last 4 hours of information based on the corresponding selections
made.
The counter tile under the drop-down lists rolls-up the total number of storage accounts in the subscription and
reflects how many of the total are selected. There is conditional color-coding or heatmaps for columns in the
workbook that report transaction metrics or errors. The deepest color has the highest value and a lighter color is
based on the lowest values. For the error-based columns, the value is in red and for the metric-based columns, the
value is in blue.
Select a value in the columns Availability, E2E Latency, Server Latency, and transaction error type/Errors
directs you to a report tailored to the specific type of storage metrics that match the column selected for that
storage account. For more information about the workbooks for each category, see the Detailed storage
workbooks section below.
NOTE
For details on which errors can be shown in the report, see Response Type schema and look for response types such as
ServerOtherError, ClientOtherError, ClientThrottlingError. Depending on the storage accounts selected, if there are
more than three types of errors reported, all other errors are represented under the category of Other.
There is conditional color-coding or heatmaps for columns in the workbook that report capacity metrics with a
blue value. The deepest color has the highest value and a lighter color is based on the lowest values.
When you select a value under any one of the columns in the workbook, you drill down to the Capacity
workbook for the storage account. Further details about the drill-down report are described in the Detailed
storage workbooks section below.
Selecting any of the error categories listed in the grid open the Failure workbook. The report shows metric
tiles of all other client-side errors except described ones and successful requests, client-throttling errors, a
performance chart for the transaction Response Type dimension metric specific to ClientOtherError
attribute, and two tables - Transactions by API name and Transactions by Response type.
Capacity opens the Capacity workbook. It shows the total amount of storage used for each storage data
object in the account in the tiles and the chart, and how many data objects are stored in the account.
Pin and export
You can pin any one of the metric sections to an Azure Dashboard by selecting the pushpin icon at the top right of
the section.
The multi-subscription and storage account Overview or Capacity workbooks support exporting the results in
Excel format by selecting the down arrow icon to the right of the pushpin icon.
Customize Azure Monitor for Storage (preview)
This section highlights common scenarios for editing the workbook to customize in support of your data analytics
needs:
Scope the workbook to always select a particular subscription or storage account(s)
Change metrics in the grid
Change the availability threshold
Change the color rendering
The customizations are saved to a custom workbook to prevent overwriting the default configuration in our
published workbook. Workbooks are saved within a resource group, either in the My Reports section that's
private to you or in the Shared Reports section that's accessible to everyone with access to the resource group.
After you save the custom workbook, you need to go to the workbook gallery to launch it.
4. We are going to remove the Account used capacity timeline column, so select Column Settings in the
metrics grid.
5. In the Edit column settings pane, select under the Columns section
microsoft.storage/storageaccounts-Capacity-UsedCapacity Timeline$|Account used capacity
Timeline$, and under the drop-down list Column renderer select Hidden.
6. Select Save and close to commit your change.
Now let's change the color theme for the capacity metrics in the report to use green instead of blue.
1. Select Column Settings in the metrics grid.
2. In the Edit column settings pane, select under the Columns section
microsoft.storage/storageaccounts-Capacity-
UsedCapacity$|microsoft.storage/storageaccounts/blobservices-Capacity-
BlobCapacity$|microsoft.storage/storageaccounts/fileservices-Capacity-
FileCapacity$|microsoft.storage/storageaccounts/queueservices-Capacity-
QueueCapacity$|microsoft.storage/storageaccounts/tableservices-Capacity-TableCapacity$.
Under the drop-down list Color palette, select Green.
3. Select Save and close to commit your change.
4. Select Save as from the command bar to save a copy of the workbook with your customizations, and then
click Done editing to return to reading mode.
Modify the availability threshold
In this example, we are working with the storage account capacity workbook and demonstrating how to modify
the availability threshold. By default, the tile and grid reporting percent availability are configured with a minimum
threshold of 90 and maximum threshold of 99. We are going to change the minimum threshold value of the
Availability % in the Availability by API name grid to 85%, which means the health state changes to critical if
the threshold is less than 85 percent.
1. Select Storage accounts from the portal and then select a storage account from the list.
2. Select Insights (preview) from the left-hand pane.
3. In the workbook, select Availability to switch to the availability workbook, and then select Edit from the
command bar.
4. Scroll down to the bottom of the page and on the left-hand side next to the Availability by API grid, select
Edit.
5. Select Column settings and then in the Edit column settings pane, under the Columns section select
Availability (%) (Thresholds + Formatted).
6. Change the value for the Critical health state from 90 to 85 and then click Save and Close.
7. Select Save as from the command bar to save a copy of the workbook with your customizations, and then
click Done editing to return to reading mode.
Troubleshooting
This section will help you with the diagnosis and troubleshooting of some of the common issues you may
encounter when using Azure Monitor for Storage (preview ). Use the list below to locate the information relevant
to your specific issue.
Resolving performance, capacity, or availability issues
To help troubleshoot any storage-related issues you identify with Azure Monitor for Storage (preview ), see the
Azure Storage troubleshooting guidance.
Why can I only see 200 storage accounts?
The number of selected storage accounts has a limit of 200, regardless of the number of subscriptions that are
selected.
What happens when I click on a recently pinned tile in the dashboard?
If you click anywhere on the tile, it will take you to the tab where the tile was pinned from. For example, if you
pin a graph in the "Storage Account Overview" tab then when you click that tile in the dashboard it will open up
that default view, however if you pin a graph from your own saved copy then it will open up your saved copy's
view.
The filter icon in the top left of the title opens the "Configure tile settings" tab.
The ellipse icon in the top right will give you the options to "Customize title data", "customize", "refresh" and
"remove from dashboard".
What happens when I save a workbook?
When you save a workbook, it lets you create a new copy of the workbook with your edits and change the title.
Saving does not overwrite the workbook, the current workbook will always be the default view.
An unsaved workbook is just the default view.
Why don’t I see all my subscriptions in the portal?
The portal will show data only for selected subscriptions on portal launch. To change what subscriptions are
selected, go to the top right and click on the notebook with a filter icon. This will show the Directory +
subscriptions tab.
If you want to see n different types of error than specify splitByLimit as n+1, 1 extra for rest of the errors.
I saved my workbook while on some Storage Account. Why can’t I find it now?
Each workbook is saved in the storage account that you saved it in. Try to find the specific Storage Account in
which the user saved the workbook. Otherwise, there is no way to find a specific workbook without knowing the
resource (storage account).
What is time range?
Time range shows you data from a certain time frame. For example, if the time range is 24 hours, then it's showing
data from the past 24 hours.
What is time granularity (time grain)?
Time granularity is the time difference between two data points. For example, if the time grain is set to 1 second
that means metrics are collected each second.
What is the time granularity once we pin any part of the workbooks to a dashboard?
The default time granularity is set to automatic, it currently can't be changed at this time.
How do I change the timespan/ time range of the workbook step on my dashboard?
By default the timespan/time range on your dashboard tile is set to 24 hours, to change this click on the ellipses in
the top right, select Customize tile data, check "override the dashboard time settings at the title level" box and
then pick a timespan using the dropdown menu.
How do I change the title of the workbook or a workbook step I pinned to a dashboard?
The title of the workbook or workbook step that is pinned to a dashboard retains the same name it had in the
workbook. To change the title, you must save your own copy of the workbook. Then you will be able to name the
workbook before you press save.
To change the name of a step in your saved workbook select edit under the step and then select the gear at the
very bottom of settings.
Next steps
Configure metric alerts and service health notifications to set up automated alerting to aid in detecting
issues.
Learn the scenarios workbooks are designed to support, how to author new and customize existing reports,
and more by reviewing Create interactive reports with Azure Monitor workbooks.
For an in-depth guide on using Storage Analytics and other tools to identify, diagnose, and troubleshoot
Azure Storage-related issues, see Monitor, diagnose, and troubleshoot Microsoft Azure Storage.
Monitor Azure SQL Database using Azure SQL
Analytics (Preview)
12/4/2019 • 11 minutes to read • Edit Online
Azure SQL Analytics is an advanced cloud monitoring solution for monitoring performance of Azure SQL
databases, elastic pools, and Managed Instances at scale and across multiple subscriptions through a single pane
of glass. It collects and visualizes important Azure SQL Database performance metrics with built-in intelligence for
performance troubleshooting.
By using metrics that you collect with the solution, you can create custom monitoring rules and alerts. The solution
helps you to identify issues at each layer of your application stack. It uses Azure Diagnostic metrics along with
Azure Monitor views to present data about all your Azure SQL databases, elastic pools, and databases in Managed
Instances in a single Log Analytics workspace. Azure Monitor helps you to collect, correlate, and visualize
structured and unstructured data.
For a hands-on overview on using Azure SQL Analytics solution and for typical usage scenarios, see the
embedded video:
Connected sources
Azure SQL Analytics is a cloud only monitoring solution supporting streaming of diagnostics telemetry for Azure
SQL databases: single, pooled, and Managed Instance databases. As the solution does not use agents to connect to
Azure Monitor, the solution does not support monitoring of SQL Server hosted on-premises or in VMs, see the
compatibility table below.
Diagnostics settings Yes Azure metric and log data are sent to
Azure Monitor Logs directly by Azure.
Configuration
Use the process described in Add Azure Monitor solutions from the Solutions Gallery to add the Azure SQL
Analytics (Preview ) solution to your Log Analytics workspace.
Configure Azure SQL Databases, elastic pools and Managed Instances to stream diagnostics telemetry
Once you have created Azure SQL Analytics solution in your workspace, you need to configure each resources
that you want to monitor to stream its diagnostics telemetry to the solution. Follow detailed instructions on this
page:
Enable Azure Diagnostics for your Azure SQL database to stream diagnostics telemetry to Azure SQL
Analytics.
The above page also provides instructions on enabling support for monitoring multiple Azure subscriptions from
a single Azure SQL Analytics workspace as a single pane of glass.
Once loaded, the tile shows the number of Azure SQL databases, elastic pools, Managed Instances, and databases
in Managed instances that the solution is receiving diagnostics telemetry from.
The solution provides two separate views -- one for monitoring Azure SQL Databases and elastic pools, and the
other view for monitoring Managed Instance, and databases in Managed Instances.
To view Azure SQL Analytics monitoring dashboard for Azure SQL Databases and elastic pools, click on the upper
part of the tile. To view Azure SQL Analytics monitoring dashboard for Managed Instance, and databases in
Managed Instance, click on the lower part of the tile.
Viewing Azure SQL Analytics data
The dashboard includes the overview of all databases that are monitored through different perspectives. For
different perspectives to work, you must enable proper metrics or logs on your SQL resources to be streamed to
Log Analytics workspace.
Note that if some metrics or logs are not streamed into Azure Monitor, the tiles in the solution are not populated
with monitoring information.
Azure SQL Database and elastic pool view
Once the Azure SQL Analytics tile for the database is selected, the monitoring dashboard is shown.
Selecting any of the tiles, opens a drill-down report into the specific perspective. Once the perspective is selected,
the drill-down report is opened.
Each perspective in this view provides summaries on subscription, server, elastic pool, and database level. In
addition, each perspective shows a perspective specific to the report on the right. Selecting subscription, server,
pool, or database from the list continues the drill-down.
Managed Instance and databases in Managed Instance view
Once the Azure SQL Analytics tile for the databases is selected, the monitoring dashboard is shown.
Selecting any of the tiles, opens a drill-down report into the specific perspective. Once the perspective is selected,
the drill-down report is opened.
Selecting Managed Instance view, shows details on the Managed Instance utilization, databases it contains, and
telemetry on the queries executed across the instance.
Query reports
Through the Query duration and query waits perspectives, you can correlate the performance of any query
through the query report. This report compares the query performance across different databases and makes it
easy to pinpoint databases that perform the selected query well versus ones that are slow.
Permissions
To use Azure SQL Analytics, users need to be granted a minimum permission of the Reader role in Azure. This
role, however, does not allow users to see the query text, or perform any Automatic tuning actions. More
permissive roles in Azure that allow using the solution to the fullest extent are Owner, Contributor, SQL DB
Contributor, or SQL Server Contributor. You also might want to consider creating a custom role in the portal with
specific permissions required only to use Azure SQL Analytics, and with no access to managing other resources.
Creating a custom role in portal
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Recognizing that some organizations enforce strict permission controls in Azure, find the following PowerShell
script enabling creation of a custom role “SQL Analytics Monitoring Operator” in Azure portal with the minimum
read and write permissions required to use Azure SQL Analytics to its fullest extent.
Replace the “{SubscriptionId}" in the below script with your Azure subscription ID, and execute the script logged in
as an Owner or Contributor role in Azure.
Connect-AzAccount
Select-AzSubscription {SubscriptionId}
$role = Get-AzRoleDefinition -Name Reader
$role.Name = "SQL Analytics Monitoring Operator"
$role.Description = "Lets you monitor database performance with Azure SQL Analytics as a reader. Does not
allow change of resources."
$role.IsCustom = $true
$role.Actions.Add("Microsoft.SQL/servers/databases/read");
$role.Actions.Add("Microsoft.SQL/servers/databases/topQueries/queryText/*");
$role.Actions.Add("Microsoft.Sql/servers/databases/advisors/read");
$role.Actions.Add("Microsoft.Sql/servers/databases/advisors/write");
$role.Actions.Add("Microsoft.Sql/servers/databases/advisors/recommendedActions/read");
$role.Actions.Add("Microsoft.Sql/servers/databases/advisors/recommendedActions/write");
$role.Actions.Add("Microsoft.Sql/servers/databases/automaticTuning/read");
$role.Actions.Add("Microsoft.Sql/servers/databases/automaticTuning/write");
$role.Actions.Add("Microsoft.Sql/servers/advisors/read");
$role.Actions.Add("Microsoft.Sql/servers/advisors/write");
$role.Actions.Add("Microsoft.Sql/servers/advisors/recommendedActions/read");
$role.Actions.Add("Microsoft.Sql/servers/advisors/recommendedActions/write");
$role.Actions.Add("Microsoft.Resources/deployments/write");
$role.AssignableScopes = "/subscriptions/{SubscriptionId}"
New-AzRoleDefinition $role
Once the new role is created, assign this role to each user that you need to grant custom permissions to use Azure
SQL Analytics.
AzureMetrics
| where ResourceProvider=="MICROSOFT.SQL"
| where ResourceId contains "/DATABASES/"
| where MetricName=="cpu_percent"
| summarize AggregatedValue = max(Maximum) by bin(TimeGenerated, 5m)
| render timechart
NOTE
Pre-requirement of setting up this alert is that monitored databases stream Basic metrics to the solution.
Replace the MetricName value cpu_percent with dtu_consumption_percent to obtain high DTU results instead.
AzureMetrics
| where ResourceProvider=="MICROSOFT.SQL"
| where ResourceId contains "/ELASTICPOOLS/"
| where MetricName=="cpu_percent"
| summarize AggregatedValue = max(Maximum) by bin(TimeGenerated, 5m)
| render timechart
NOTE
Pre-requirement of setting up this alert is that monitored databases stream Basic metrics to the solution.
Replace the MetricName value cpu_percent with dtu_consumption_percent to obtain high DTU results instead.
NOTE
Pre-requirement of setting up this alert is that monitored databases stream SQLInsights diagnostics log to the solution.
This query requires an alert rule to be set up to run with the same frequency as alert_run_interval in order to avoid
duplicate results. The rule should be set up to fire off the alert when there exist results (> 0 results) from the query.
Customize the alert_run_interval to specify the time range to check if the condition has occurred on databases configured
to stream SQLInsights log to the solution.
Customize the insights_string to capture the output of the Insights root cause analysis text. This is the same text
displayed in the UI of the solution that you can use from the existing insights. Alternatively, you can use the query below
to see the text of all Insights generated on your subscription. Use the output of the query to harvest the distinct strings
for setting up alerts on Insights.
AzureDiagnostics
| where Category == "SQLInsights" and status_s == "Active"
| distinct rootCauseAnalysis_s
NOTE
Pre-requirement of setting up this alert is that monitored Managed Instance has the streaming of ResourceUsageStats
log enabled to the solution.
This query requires an alert rule to be set up to fire off an alert when there exist results (> 0 results) from the query,
denoting that the condition exists on the Managed Instance. The output is storage percentage consumption on the
Managed Instance.
Managed Instance CPU average consumption is above 95% in the last 1 hr
NOTE
Pre-requirement of setting up this alert is that monitored Managed Instance has the streaming of ResourceUsageStats
log enabled to the solution.
This query requires an alert rule to be set up to fire off an alert when there exist results (> 0 results) from the query,
denoting that the condition exists on the Managed Instance. The output is average CPU utilization percentage
consumption in defined period on the Managed Instance.
Pricing
While the solution is free to use, consumption of diagnostics telemetry above the free units of data ingestion
allocated each month applies, see Log Analytics pricing. The free units of data ingestion provided enable free
monitoring of several databases each month. Note that more active databases with heavier workloads ingest more
data versus idle databases. You can easily monitor your data ingestion consumption in the solution by selecting
OMS Workspace on the navigation menu of Azure SQL Analytics, and then selecting Usage and Estimated Costs.
Next steps
Use log queries in Azure Monitor to view detailed Azure SQL data.
Create your own dashboards showing Azure SQL data.
Create alerts when specific Azure SQL events occur.
Optimize your SQL environment with the SQL Server
Health Check solution in Azure Monitor
1/17/2020 • 12 minutes to read • Edit Online
You can use the SQL Health Check solution to assess the risk and health of your server environments on a regular
interval. This article will help you install the solution so that you can take corrective actions for potential problems.
This solution provides a prioritized list of recommendations specific to your deployed server infrastructure. The
recommendations are categorized across six focus areas which help you quickly understand the risk and take
corrective action.
The recommendations made are based on the knowledge and experience gained by Microsoft engineers from
thousands of customer visits. Each recommendation provides guidance about why an issue might matter to you
and how to implement the suggested changes.
You can choose focus areas that are most important to your organization and track your progress toward running
a risk free and healthy environment.
After you've added the solution and an assessment is completed, summary information for focus areas is shown
on the SQL Health Check dashboard for the infrastructure in your environment. The following sections describe
how to use the information on the SQL Health Check dashboard, where you can view and then take
recommended actions for your SQL Server infrastructure.
Prerequisites
The SQL Health Check solution requires a supported version of .NET Framework 4.6.2 installed on each
computer that has the Microsoft Monitoring Agent (MMA) installed. The MMA agent is used by System
Center 2016 - Operations Manager and Operations Manager 2012 R2, and Azure Monitor.
The solution supports SQL Server version 2012, 2014, and 2016.
A Log Analytics workspace to add the SQL Health Check solution from the Azure marketplace in the Azure
portal. In order to install the solution, you must be an administrator or contributor in the Azure
subscription.
NOTE
After you've added the solution, the AdvisorAssessment.exe file is added to servers with agents. Configuration data is
read and then sent to Azure Monitor in the cloud for processing. Logic is applied to the received data and the cloud
service records the data.
To perform the health check against your SQL Server servers, they require an agent and connectivity to Azure
Monitor using one of the following supported methods:
1. Install the Microsoft Monitoring Agent (MMA) if the server is not already monitored by System Center 2016 -
Operations Manager or Operations Manager 2012 R2.
2. If it is monitored with System Center 2016 - Operations Manager or Operations Manager 2012 R2 and the
management group is not integrated with Azure Monitor, the server can be multi-homed with Log Analytics to
collect data and forward to the service and still be monitored by Operations Manager.
3. Otherwise, if your Operations Manager management group is integrated with the service, you need to add the
domain controllers for data collection by the service following the steps under add agent-managed computers
after you enable the solution in your workspace.
The agent on your SQL Server which reports to an Operations Manager management group, collects data,
forwards to its assigned management server, and then is sent directly from a management server to Azure
Monitor. The data is not written to the Operations Manager databases.
If the SQL Server is monitored by Operations Manager, you need to configure an Operations Manager Run As
account. See Operations Manager run-as accounts for Azure Monitor below for more information.
NOTE
By default workflows in the management pack runs in the security context of the Local System account. If you are using the
Microsoft Monitoring Agent connected directly to the service rather than reporting directly to an Operations Manager
management group, skip steps 1-5 below and run either the T-SQL or PowerShell sample, specifying NT
AUTHORITY\SYSTEM as the user name.
1. In Operations Manager, open the Operations console, and then click Administration.
2. Under Run As Configuration, click Profiles, and open SQL Assessment Run As Profile.
3. On the Run As Accounts page, click Add.
4. Select a Windows Run As account that contains the credentials needed for SQL Server, or click New to
create one.
NOTE
The Run As account type must be Windows. The Run As account must also be part of Local Administrators group on
all Windows Servers hosting SQL Server Instances.
5. Click Save.
6. Modify and then execute the following T-SQL sample on each SQL Server instance to grant minimum
permissions required for the Run As Account to perform the health check. However, you don’t need to do
this if a Run As Account is already part of the sysadmin server role on SQL Server instances.
---
-- Replace <UserName> with the actual user name being used as Run As Account.
USE master
-- Create login for the user, comment this line if login is already created.
CREATE LOGIN [<UserName>] FROM WINDOWS
-- Add database user for all the databases on SQL Server Instance, this is required for connecting to
individual databases.
-- NOTE: This command must be run anytime new databases are added to SQL Server instances.
EXEC sp_msforeachdb N'USE [?]; CREATE USER [<UserName>] FOR LOGIN [<UserName>];'
import-module OperationsManager
New-SCOMManagementGroupConnection "<your management group name>"
Ignore recommendations
If you have recommendations that you want to ignore, you can create a text file that Azure Monitor will use to
prevent recommendations from appearing in your assessment results.
To identify recommendations that you will ignore
1. In the Azure Monitor menu, click Logs.
2. Use the following query to list recommendations that have failed for computers in your environment.
3. Choose recommendations that you want to ignore. You’ll use the values for RecommendationId in the next
procedure.
To create and use an IgnoreRecommendations.txt text file
1. Create a file named IgnoreRecommendations.txt.
2. Paste or type each RecommendationId for each recommendation that you want Azure Monitor to ignore on a
separate line and then save and close the file.
3. Put the file in the following folder on each computer where you want Azure Monitor to ignore
recommendations.
On computers with the Microsoft Monitoring Agent (connected directly or through Operations
Manager) - SystemDrive:\Program Files\Microsoft Monitoring Agent\Agent
On the Operations Manager management server - SystemDrive:\Program Files\Microsoft System
Center 2012 R2\Operations Manager\Server
On the Operations Manager 2016 management server - SystemDrive:\Program Files\Microsoft System
Center 2016\Operations Manager\Server
To verify that recommendations are ignored
1. After the next scheduled assessment runs, by default every 7 days, the specified recommendations are
marked Ignored and will not appear on the assessment dashboard.
2. You can use the following Log Search queries to list all the ignored recommendations.
3. If you decide later that you want to see ignored recommendations, remove any IgnoreRecommendations.txt
files, or you can remove RecommendationIDs from them.
SQLAssessmentRecommendation
| distinct RecommendationId, FocusArea, ActionArea, Recommendation, Description
| sort by FocusArea,ActionArea, Recommendation
Next steps
Log queries to learn how to analyze detailed SQL Health Check data and recommendations.
Explore Azure Monitor for Azure Cosmos DB
(preview)
10/30/2019 • 4 minutes to read • Edit Online
Azure Monitor for Azure Cosmos DB (preview ) provides a view of the overall performance, failures, capacity, and
operational health of all your Azure Cosmos DB resources in a unified interactive experience. This article will help
you understand the benefits of this new monitoring experience, and how you can modify and adapt the experience
to fit the unique needs of your organization.
Introduction
Before diving into the experience, you should understand how it presents and visualizes information.
It delivers:
At scale perspective of your Azure Cosmos DB resources across all your subscriptions in a single location,
with the ability to selectively scope to only those subscriptions and resources you are interested in
evaluating.
Drill down analysis of a particular Azure CosmosDB resource to help diagnose issues or perform detailed
analysis by category - utilization, failures, capacity, and operations. Selecting any one of those options
provides an in-depth view of the relevant Azure Cosmos DB metrics.
Customizable - This experience is built on top of Azure Monitor workbook templates allowing you to
change what metrics are displayed, modify or set thresholds that align with your limits, and then save into a
custom workbook. Charts in the workbooks can then be pinned to Azure dashboards.
This feature does not require you to enable or configure anything, these Azure Cosmos DB metrics are collected
by default.
NOTE
There is no charge to access this feature and you will only be charged for the Azure Monitor essential features you configure
or enable, as described on the Azure Monitor pricing details page.
Selecting the Azure Cosmos DB resource name highlighted in blue will take you to the default Overview for the
associated Azure Cosmos DB account.
Failures
Select Failures at the top of the page and the Failures portion of the workbook template opens. It shows you total
requests with the distribution of responses that make up those requests:
CODE DESCRIPTION
For a full list of status codes, consult the Azure Cosmos DB HTTP status code article.
Capacity
Select Capacity at the top of the page and the Capacity portion of the workbook template opens. It shows you
how many documents you have, your document growth over time, data usage, and the total amount of available
storage that you have left. This can be used to help identify potential storage and data utilization issues.
As with the overview workbook, selecting the drop-down next to an Azure Cosmos DB resource in the
Subscription column will reveal a breakdown by the individual containers that make up the database.
Operations
Select Operations at the top of the page and the Operations portion of the workbook template opens. It gives
you the ability to see your requests broken down by the type of requests made.
So in the example below you see that eastus-billingint is predominantly receiving read requests, but with a
small number of upsert and create requests. Whereas westeurope-billingint is read-only from a request
perspective, at least over the past four hours that the workbook is currently scoped to via its time range parameter.
Pin, export, and expand
You can pin any one of the metric sections to an Azure Dashboard by selecting the pushpin icon at the top right of
the section.
To export your data into the Excel format, select the down arrow icon to the left of the pushpin icon.
To expand or collapse all drop-down views in the workbook, select the expand icon to the left of the export icon:
Workbooks are saved within a resource group, either in the My Reports section that's private to you or in the
Shared Reports section that's accessible to everyone with access to the resource group. After you save the
custom workbook, you need to go to the workbook gallery to launch it.
Next steps
Configure metric alerts and service health notifications to set up automated alerting to aid in detecting
issues.
Learn the scenarios workbooks are designed to support, how to author new and customize existing reports,
and more by reviewing Create interactive reports with Azure Monitor workbooks.
Azure Monitor for Networks (Preview)
11/8/2019 • 2 minutes to read • Edit Online
Azure Monitor for Network provides a comprehensive view of health and metrics for all deployed network
resource without any configuration. The advanced search capability helps identify resource dependencies,
enabling scenarios such as identifying resources that are hosting your website by simply searching for hosted
website name.
The Azure Monitor for Networks Overview page provides an effortless way to visualize the inventory of your
networking resources along with resource health and alerts. It is divided into four key functional areas:
Search and filtering
Resource Health and Metrics
Alerts
Dependency view
Clicking on the two Unavailable ER and VPN connections, launches a metric view.
You can click on each element in the grid view. Click on the Health icon to redirect to resource health for that
connection. Click on Alerts to redirect to alerts and metrics page respectively for that connection.
Alerts
The Alerts grid on the right provides a view of all the alerts generated for the selected resources across all
subscriptions. Click on the alert counts to navigate to detailed alerts page.
Dependency view
The Dependency view helps visualize how the resource is configured. Currently dependency view is only
supported for Application Gateway. Dependency view can be accessed by clicking on the Application Gateway
resource name from the metrics grid view.
The Dependency view for Application Gateway provides a simplified view of how the front-end IPs are
connected to the listeners, rules and backend pool. The connecting edges are color coded and provide additional
details based on the backend pool health. The view also provides a detailed view of Application Gateway metrics
and metrics for all related backend pools such as VMSS and VM instances.
The dependency graph enables easy navigation to configuration settings. Right click on a backend pool to access
to other functionality. For example, if the backend pool is a VM then you can directly access VM Insights and
Network Watcher connection troubleshoot to identify connectivity issues.
The search and filter bar on the dependency view provide an effortless way to search through the graph. For
example, searching for AppGWTestRule in the example below will narrow down the graphical view to all nodes
connected via AppGWTestRule.
Different filters provide help to narrow down on to a specific path and state. For example, select only Unhealthy
from the Health Status drop down to show all the edges where state is Unhealthy.
Click on Detailed Metric View to launch a pre-configured workbook with detailed metrics for the application
gateway, all backend pool resources and front end IPs.
Next steps
Learn more about network monitoring at What is Azure Network Watcher?.
Azure networking monitoring solutions in Azure
Monitor
12/6/2019 • 8 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Azure Monitor offers the following solutions for monitoring your networks:
Network Performance Monitor (NPM ) to
Monitor the health of your network
Azure Application Gateway analytics to review
Azure Application Gateway logs
Azure Application Gateway metrics
Solutions to monitor and audit network activity on your cloud network
Traffic Analytics
Azure Network Security Group Analytics
OPERATIONS
SYSTEMS MANAGER
CENTER AGENT DATA
OPERATIONS OPERATIONS SENT VIA
MANAGER MANAGER MANAGEMENT COLLECTION
PLATFORM DIRECT AGENT AGENT AZURE REQUIRED? GROUP FREQUENCY
$workspaceId = "/subscriptions/d2e37fee-1234-40b2-5678-0b2199de3b50/resourcegroups/oi-default-east-
us/providers/microsoft.operationalinsights/workspaces/rollingbaskets"
After you click the Azure Application Gateway analytics tile on the Overview, you can view summaries of your
logs and then drill in to details for the following categories:
Application Gateway Access logs
Client and server errors for Application Gateway access logs
Requests per hour for each Application Gateway
Failed requests per hour for each Application Gateway
Errors by user agent for Application Gateways
Application Gateway performance
Host health for Application Gateway
Maximum and 95th percentile for Application Gateway failed requests
On the Azure Application Gateway analytics dashboard, review the summary information in one of the blades,
and then click one to view detailed information on the log search page.
On any of the log search pages, you can view results by time, detailed results, and your log search history. You can
also filter by facets to narrow the results.
NOTE
The Network Security Group analytics solution is moving to community support since its functionality has been replaced by
Traffic Analytics.
The solution is now available in Azure Quickstart Templates and will soon no longer be available in the Azure Marketplace.
For existing customers who already added the solution to their workspace, it will continue to function with no changes.
Microsoft will continue to support sending NSG resource logs to your workspace using Diagnostics Settings.
For any field that has a suffix of _s, _d, or _g in the name, change the first character to lower case
For any field that has a suffix of _o in name, the data is split into individual fields based on the nested
field names.
5. Remove the Azure Networking Analytics (Deprecated ) solution.
If you are using PowerShell, use
Set-AzureOperationalInsightsIntelligencePack -ResourceGroupName <resource group that the workspace is
in> -WorkspaceName <name of the log analytics workspace> -IntelligencePackName "AzureNetwork" -
Enabled $false
Data collected before the change is not visible in the new solution. You can continue to query for this data using
the old Type and field names.
Troubleshooting
Troubleshoot Azure Diagnostics
If you receive the following error message, the Microsoft.insights resource provider is not registered:
Failed to update diagnostics for 'resource'. {"code":"Forbidden","message":"Please register the subscription
'subscription id' with Microsoft.Insights."}
To register the resource provider, perform the following steps in the Azure portal:
1. In the navigation pane on the left, click Subscriptions
2. Select the subscription identified in the error message
3. Click Resource Providers
4. Find the Microsoft.insights provider
5. Click the Register link
Once the Microsoft.insights resource provider is registered, retry configuring diagnostics.
In PowerShell, if you receive the following error message, you need to update your version of PowerShell:
Set-AzDiagnosticSetting : A parameter cannot be found that matches parameter name 'WorkspaceId'.
Update your version of Azure PowerShell, follow the instructions in the Install Azure PowerShell article.
Next steps
Use Log queries in Azure Monitor to view detailed Azure diagnostics data.
Gather insights about your DNS infrastructure with
the DNS Analytics Preview solution
1/14/2020 • 9 minutes to read • Edit Online
This article describes how to set up and use the Azure DNS Analytics solution in Azure Monitor to gather insights
into DNS infrastructure on security, performance, and operations.
DNS Analytics helps you to:
Identify clients that try to resolve malicious domain names.
Identify stale resource records.
Identify frequently queried domain names and talkative DNS clients.
View request load on DNS servers.
View dynamic DNS registration failures.
The solution collects, analyzes, and correlates Windows DNS analytic and audit logs and other related data from
your DNS servers.
Connected sources
The following table describes the connected sources that are supported by this solution:
System Center Operations Manager Yes The solution collects DNS information
management group from agents in a connected Operations
Manager management group. A direct
connection from the Operations
Manager agent to Azure Monitor is not
required. Data is forwarded from the
management group to the Log
Analytics workspace.
Solution dashboard
The solution dashboard shows summarized information for the various features of the solution. It also includes
links to the detailed view for forensic analysis and diagnosis. By default, the data is shown for the last seven days.
You can change the date and time range by using the date-time selection control, as shown in the following
image:
a. To view the log data for lookup queries, select LookUpQuery as the Subtype filter from the facet control
on the left. A table that lists all the lookup query events for the selected time period is displayed.
b. To view the log data for dynamic registrations, select DynamicRegistration as the Subtype filter from
the facet control on the left. A table that lists all the dynamic registration events for the selected time period
is displayed.
c. To view the log data for configuration changes, select ConfigurationChange as the Subtype filter from
the facet control on the left. A table that lists all the configuration change events for the selected time period
is displayed.
2. In the search query box, type DnsInventory to view all the DNS inventory-related data for the DNS
servers managed by the solution. The results list the log data for DNS servers, DNS zones, and resource
records.
Troubleshooting
Common troubleshooting steps:
1. Missing DNS Lookups Data - To troubleshoot this issue, try resetting the config or just loading the
configuration page once in portal. For resetting, just change a setting to another value, then change it back to to
the original value, and save the config.
Feedback
To provide feedback, visit the Log Analytics UserVoice page to post ideas for DNS Analytics features to work on.
Next steps
Query logs to view detailed DNS log records.
Network Performance Monitor solution in Azure
12/23/2019 • 16 minutes to read • Edit Online
Network Performance Monitor is a cloud-based hybrid network monitoring solution that helps you monitor
network performance between various points in your network infrastructure. It also helps you monitor network
connectivity to service and application endpoints and monitor the performance of Azure ExpressRoute.
Network Performance Monitor detects network issues like traffic blackholing, routing errors, and issues that
conventional network monitoring methods aren't able to detect. The solution generates alerts and notifies you
when a threshold is breached for a network link. It also ensures timely detection of network performance issues
and localizes the source of the problem to a particular network segment or device.
Network Performance Monitor offers three broad capabilities:
Performance Monitor: You can monitor network connectivity across cloud deployments and on-premises
locations, multiple data centers, and branch offices and mission-critical multitier applications or
microservices. With Performance Monitor, you can detect network issues before users complain.
Service Connectivity Monitor: You can monitor the connectivity from your users to the services you care
about, determine what infrastructure is in the path, and identify where network bottlenecks occur. You can
know about outages before your users, and see the exact location of the issues along your network path.
This capability helps you perform tests based on HTTP, HTTPS, TCP, and ICMP to monitor in near real
time or historically the availability and response time of your service. You also can monitor the contribution
of the network in packet loss and latency. With a network topology map, you can isolate network
slowdowns. You can identify problem spots that occur along the network path from the node to the service,
with latency data on each hop. With built-in tests, you can monitor network connectivity to Office 365 and
Dynamics CRM without any preconfiguration. With this capability, you can monitor network connectivity
to any TCP -capable endpoint, such as websites, SaaS applications, PaaS applications, and SQL databases.
ExpressRoute Monitor: Monitor end-to-end connectivity and performance between your branch offices
and Azure, over Azure ExpressRoute.
More information on the various capabilities supported by Network Performance Monitor is available online.
Supported Regions
NPM can monitor connectivity between networks and applications in any part of the world, from a workspace
that is hosted in one of the following regions:
North Europe
West Europe
France Central
West Central US
North Central US
South Central US
Central US
East US
East US 2
West US 2
East Japan
South East Asia
South East Australia
Australia Central
Australia East
South UK
East Asia
Korea Central
Central India
US Government Virginia
China East 2
The list of supported regions for ExpressRoute Monitor is available in the documentation.
NOTE
The script configures only Windows Firewall locally. If you have a network firewall, make sure that it allows traffic
destined for the TCP port used by Network Performance Monitor.
NOTE
You don't need to run the EnableRules.ps1 PowerShell script for Service Connectivity Monitor.
ICMP protocol: If you choose ICMP as the protocol for monitoring, enable the following firewall rules to
reliably utilize ICMP:
Service Connectivity Monitor: The capability provides built-in preconfigured tests to monitor network
connectivity to Office 365 and Dynamics 365 from your agents. Choose the Office 365 and Dynamics 365
services that you want to monitor by selecting the check boxes beside them. To choose the agents from
which you want to monitor, select Add Agents. If you don't want to use this capability or want to set it up
later, don't choose anything and select Save & Continue.
ExpressRoute Monitor: Select Discover Now to discover all the ExpressRoute private peerings that are
connected to the virtual networks in the Azure subscription linked with this Log Analytics workspace.
After the discovery is finished, the discovered circuits and peerings are listed in a table.
The monitoring for these circuits and peerings is initially in a disabled state. Select each resource that you want to
monitor, and configure monitoring for them from the details view on the right. Select Save to save the
configuration. To learn more, see the "Configure ExpressRoute monitoring" article.
After the setup is finished, it takes 30 minutes to an hour for the data to populate. While the solution aggregates
data from your network, you see the message Solution requires additional configuration on the Network
Performance Monitor Overview tile. After the data is collected and indexed, the Overview tile changes and
informs you of your network health in a summary. You then can edit the monitoring of the nodes on which Log
Analytics agents are installed, as well as the subnets discovered from your environment.
Edit monitoring settings for subnets and nodes
All subnets with at least one agent installed are listed on the Subnetworks tab on the configuration page.
To enable or disable monitoring of particular subnetworks:
1. Select or clear the check box next to the subnetwork ID. Then make sure that Use for Monitoring is selected
or cleared, as appropriate. You can select or clear multiple subnets. When disabled, subnetworks aren't
monitored, and the agents are updated to stop pinging other agents.
2. Choose the nodes that you want to monitor in a particular subnetwork. Select the subnetwork from the list,
and move the required nodes between the lists that contain unmonitored and monitored nodes. You can add a
custom description to the subnetwork.
3. Select Save to save the configuration.
Choose nodes to monitor
All the nodes that have an agent installed on them are listed on the Nodes tab.
1. Select or clear the nodes that you want to monitor or stop monitoring.
2. Select Use for Monitoring, or clear it, as appropriate.
3. Select Save.
Configure the capabilities you want:
Performance Monitor
Service Connectivity Monitor
ExpressRoute Monitor
OPERATIONS
SYSTEM MANAGER
CENTER AGENT DATA
OPERATIONS OPERATIONS SENT VIA
MANAGER AZURE MANAGER MANAGEMENT COLLECTION
PLATFORM DIRECT AGENT AGENT STORAGE REQUIRED? GROUP FREQUENCY
Windows • • TCP
handshakes/I
CMP ECHO
messages
every 5
seconds, data
sent every 3
minutes
The solution uses synthetic transactions to assess the health of the network. Log Analytics agents installed at
various points in the network exchange TCP packets or ICMP Echo with one another. Whether the agents use TCP
packets or ICMP Echo depends on the protocol you selected for monitoring. In the process, agents learn the
round-trip time and packet loss, if any. Periodically, each agent also performs a trace route to other agents to find
all the various routes in the network that must be tested. Using this data, the agents can deduce the network
latency and packet loss figures. The tests are repeated every five seconds. Data is aggregated for about three
minutes by the agents before it's uploaded to the Log Analytics workspace in Azure Monitor.
NOTE
Although agents communicate with each other frequently, they don't generate significant network traffic while conducting
the tests. Agents rely only on TCP SYN-SYNACK-ACK handshake packets to determine the loss and latency. No data
packets are exchanged. During this process, agents communicate with each other only when needed. The agent
communication topology is optimized to reduce network traffic.
Trend charts
At each level that you drill down, you can see the trend of the applicable metric. It can be loss, latency, response
time, or bandwidth utilization. To change the time interval for the trend, use the time control at the top of the
chart.
Trend charts show you a historical perspective of the performance of a performance metric. Some network issues
are transient in nature and are hard to catch by looking at only the current state of the network. Issues can surface
quickly and disappear before anyone notices, only to reappear at a later point in time. Such transient issues also
can be difficult for application administrators. The issues often show up as unexplained increases in application
response time, even when all application components appear to run smoothly.
You can easily detect these kinds of issues by looking at a trend chart. The issue appears as a sudden spike in
network latency or packet loss. To investigate the issue, use the Network State Recorder control to view the
network snapshot and topology for that point in time when the issue occurred.
Topology map
Network Performance Monitor shows you the hop-by-hop topology of routes between the source and
destination endpoint on an interactive topology map. To view the topology map, select the Topology tile on the
solution dashboard. You also can select the View topology link on the drill-down pages.
The topology map displays how many routes are between the source and destination and what paths the data
packets take. The latency contributed by each network hop is also visible. All the paths for which the total path
latency is above the threshold (set in the corresponding monitoring rule) are shown in red.
When you select a node or hover over it on the topology map, you see the node properties, such as FQDN and IP
address. Select a hop to see its IP address. You can identify the troublesome network hop by noticing the latency
it contributes. To filter particular routes, use the filters in the collapsible action pane. To simplify the network
topologies, hide the intermediate hops by using the slider in the action pane. You can zoom in or zoom out of the
topology map by using your mouse wheel.
The topology shown in the map is layer 3 topology and doesn't contain layer 2 devices and connections.
Alerts
Network Performance Monitor uses the alerting capabilities of Azure Monitor.
This means that all notifications are managed using action groups.
If you are an NPM user creating an alert via Log Analytics:
1. You will see a link that will redirect you to Azure portal. Click it to access the portal.
2. Click the Network Performance Monitor solution tile.
3. Navigate to Configure.
4. Select the test you want to create an alert on and follow the below mentioned steps.
If you are an NPM user creating an alert via Azure portal:
1. You can choose to enter your email directly or you can choose to create alerts via action groups.
2. If you choose to enter your email directly, an action group with the name NPM Email ActionGroup is
created and the email id is added to that action group.
3. If you choose to use action groups, you will have to select an previously created action group. You can learn
how to create an action group here.
4. Once the alert is successfully created, you can use Manage Alerts link to manage your alerts.
Each time you create an alert, NPM creates a query based log alert rule in Azure Monitor. This query is triggered
every 5 mins by default. Azure monitor does not charge for the first 250 log alert rules created, and any alert
rules above the 250 log alert rules limit will be billed as per Alerts pricing in Azure Monitor pricing page.
Notifications are charged separately as per Notifications pricing in Azure Monitor pricing page.
Pricing
Information on pricing is available online.
Provide feedback
UserVoice: You can post your ideas for Network Performance Monitor features that you want us to work
on. Visit the UserVoice page.
Join our cohort: We're always interested in having new customers join our cohort. As part of it, you get
early access to new features and an opportunity to help us improve Network Performance Monitor. If
you're interested in joining, fill out this quick survey.
Next steps
Learn more about Performance Monitor, Service Connectivity Monitor, and ExpressRoute Monitor.
Network Performance Monitor solution: Performance
monitoring
10/24/2019 • 10 minutes to read • Edit Online
The Performance Monitor capability in Network Performance Monitor helps you monitor network connectivity
across various points in your network. You can monitor cloud deployments and on-premises locations, multiple
data centers and branch offices, and mission-critical multitier applications or microservices. With Performance
Monitor, you can detect network issues before your users complain. Key advantages are that you can:
Monitor loss and latency across various subnets and set alerts.
Monitor all paths (including redundant paths) on the network.
Troubleshoot transient and point-in-time network issues, which are difficult to replicate.
Determine the specific segment on the network, which is responsible for degraded performance.
Monitor the health of the network, without the need for SNMP.
Configuration
To open the configuration for Network Performance Monitor, open the Network Performance Monitor solution,
and select Configure.
NOTE
We recommend that you disable the default rule and create custom monitoring rules, especially with large networks where
you use a large number of nodes for monitoring. Custom monitoring rules can reduce the traffic generated by the solution
and help you organize the monitoring of your network.
Create monitoring rules according to your business logic. An example is if you want to monitor the performance of
the network connectivity of two office sites to headquarters. Group all the subnets in office site1 in network O1.
Then group all the subnets in office site2 in network O2. Finally, group all the subnets in the headquarters in
network H. Create two monitoring rules--one between O1 and H and the other between O2 and H.
To create custom monitoring rules:
1. Select Add Rule on the Monitor tab, and enter the rule name and description.
2. Select the pair of network or subnetwork links to monitor from the lists.
3. Select the network that contains the subnetworks you want from the network drop-down list. Then select the
subnetworks from the corresponding subnetwork drop-down list. If you want to monitor all the subnetworks in
a network link, select All subnetworks. Similarly, select the other subnetworks you want. To exclude
monitoring for particular subnetwork links from the selections you made, select Add Exception.
4. Choose between ICMP and TCP protocolsto execute synthetic transactions.
5. If you don't want to create health events for the items you selected, clear Enable Health Monitoring on the
links covered by this rule.
6. Choose monitoring conditions. To set custom thresholds for health-event generation, enter threshold values.
Whenever the value of the condition exceeds its selected threshold for the selected network or subnetwork pair,
a health event is generated.
7. Select Save to save the configuration.
After you save a monitoring rule, you can integrate that rule with Alert Management by selecting Create Alert. An
alert rule is automatically created with the search query. Other required parameters are automatically filled in.
Using an alert rule, you can receive e-mail-based alerts, in addition to the existing alerts within Network
Performance Monitor. Alerts also can trigger remedial actions with runbooks, or they can integrate with existing
service management solutions by using webhooks. Select Manage Alert to edit the alert settings.
You can now create more Performance Monitor rules or move to the solution dashboard to use the capability.
Choose the protocol
Network Performance Monitor uses synthetic transactions to calculate network performance metrics like packet
loss and link latency. To understand this concept better, consider a Network Performance Monitor agent connected
to one end of a network link. This Network Performance Monitor agent sends probe packets to a second Network
Performance Monitor agent connected to another end of the network. The second agent replies with response
packets. This process repeats a few times. By measuring the number of replies and the time taken to receive each
reply, the first Network Performance Monitor agent assesses link latency and packet drops.
The format, size, and sequence of these packets is determined by the protocol that you choose when you create
monitoring rules. Based on the protocol of the packets, the intermediate network devices, such as routers and
switches, might process these packets differently. Consequently, your protocol choice affects the accuracy of the
results. Your protocol choice also determines whether you must take any manual steps after you deploy the
Network Performance Monitor solution.
Network Performance Monitor offers you the choice between ICMP and TCP protocols for executing synthetic
transactions. If you choose ICMP when you create a synthetic transaction rule, the Network Performance Monitor
agents use ICMP ECHO messages to calculate the network latency and packet loss. ICMP ECHO uses the same
message that's sent by the conventional ping utility. When you use TCP as the protocol, Network Performance
Monitor agents send TCP SYN packets over the network. This step is followed by a TCP handshake completion,
and the connection is removed by using RST packets.
Consider the following information before you choose a protocol:
Discovery of multiple network routes. TCP is more accurate when discovering multiple routes, and it
needs fewer agents in each subnet. For example, one or two agents that use TCP can discover all redundant
paths between subnets. You need several agents that use ICMP to achieve similar results. Using ICMP, if
you havea number of routes between two subnets, you need more than 5Nagents in either a source or
destination subnet.
Accuracy of results. Routers and switches tend to assign lower priority to ICMP ECHO packets compared
to TCP packets. In certain situations, when network devices are heavily loaded, the data obtained by TCP
more closely reflects the loss and latency experienced by applications. This occurs because most of the
application traffic flows over TCP. In such cases, ICMP provides less-accurate results compared to TCP.
Firewall configuration. TCP protocol requires that TCP packets are sent to a destination port. The default
port used by Network Performance Monitor agents is 8084. You can change the port when you configure
agents. Make sure that your network firewalls or network security group (NSG ) rules (in Azure) allow traffic
on the port. You also need to make sure that the local firewall on the computers where agents are installed is
configured to allow traffic on this port. You can use PowerShell scripts to configure firewall rules on your
computers running Windows, but you need to configure your network firewall manually. In contrast, ICMP
doesn't operate by using a port. In most enterprise scenarios, ICMP traffic is permitted through the firewalls
to allow you to use network diagnostics tools like the ping utility. If you can ping one machine from another,
you can use the ICMP protocol without having to configure firewalls manually.
NOTE
Some firewalls might block ICMP, which might lead to retransmission that results in a large number of events in your security
information and event management system. Make sure the protocol that you choose isn't blocked by a network firewall or
NSG. Otherwise, Network Performance Monitor can't monitor the network segment. We recommend that you use TCP for
monitoring. Use ICMP in scenarios where you can't use TCP, such as when:
You use Windows client-based nodes, because TCP raw sockets aren't allowed in Windows clients.
Your network firewall or NSG blocks TCP.
You don't know how to switch the protocol.
If you chose to use ICMP during deployment, you can switch to TCP at any time by editing the default monitoring
rule.
1. Go to Network Performance>Monitor>Configure>Monitor. Then selectDefault rule.
2. Scroll to the Protocol section, and select the protocol that you want to use.
3. Select Save to apply the setting.
Even if the default rule uses a specific protocol, you can create new rules with a different protocol. You can even
create a mix of rules where some rules use ICMP and others use TCP.
Walkthrough
Now look at a simple investigation into the root cause for a health event.
On the solution dashboard, a health event shows that a network link is unhealthy. To investigate the issue, select
the Network links being monitored tile.
The drill-down page shows that the DMZ2-DMZ1 network link is unhealthy. Select View subnet links for this
network link.
The drill-down page shows all the subnetwork links in the DMZ2-DMZ1 network link. For both subnetwork links,
the latency crossed the threshold, which makes the network link unhealthy. You also can see the latency trends of
both subnetwork links. Use the time selection control in the graph to focus on the required time range. You can see
the time of day when latency reached its peak. Search the logs later for this time period to investigate the issue.
Select View node links to drill down further.
Similar to the previous page, the drill-down page for the particular subnetwork link lists its constituent node links.
You can perform similar actions here as you did in the previous step. Select View topology to view the topology
between the two nodes.
All the paths between the two selected nodes are plotted in the topology map. You can visualize the hop-by-hop
topology of routes between two nodes on the topology map. It gives you a clear picture of how many routes exist
between the two nodes and what paths the data packets take. Network performance bottlenecks are shown in red.
To locate a faulty network connection or a faulty network device, look at the red elements on the topology map.
You can review the loss, latency, and the number of hops in each path in the Action pane. Use the scrollbar to view
the details of the unhealthy paths. Use the filters to select the paths with the unhealthy hop so that the topology for
only the selected paths is plotted. To zoom in or out of the topology map, use your mouse wheel.
In the following image, the root cause of the problem areas to the specific section of the network appear in the red
paths and hops. Select a node in the topology map to reveal the properties of the node, which includes the FQDN
and IP address. Selecting a hop shows the IP address of the hop.
Next steps
Search logs to view detailed network performance data records.
Service Connectivity Monitor
12/30/2019 • 6 minutes to read • Edit Online
You can use the Service Connectivity Monitor capability in Network Performance Monitor to monitor network
connectivity to any endpoint that has an open TCP port. Such endpoints include websites, SaaS applications, PaaS
applications, and SQL databases.
You can perform the following functions with Service Connectivity Monitor:
Monitor the network connectivity to your applications and network services from multiple branch offices or
locations. Applications and network services include Office 365, Dynamics CRM, internal line-of-business
applications, and SQL databases.
Use built-in tests to monitor network connectivity to Office 365 and Dynamics 365 endpoints.
Determine the response time, network latency, and packet loss experienced when connecting to the endpoint.
Determine whether poor application performance is because of the network or because of some issue on the
application provider's end.
Identify hot spots on the network that might be causing poor application performance by viewing the latency
contributed by each hop on a topology map.
Configuration
To open the configuration for Network Performance Monitor, open the Network Performance Monitor solution
and select Configure.
NOTE
For Windows server-based nodes, the capability uses TCP-based requests to perform the network measurements. For
Windows client-based nodes, the capability uses ICMP-based requests to perform the network measurements. In
some cases, the target application blocks incoming ICMP-based requests when the nodes are Windows client-based.
The solution is unable to perform network measurements. We recommend that you use Windows server-based
nodes in such cases.
9. If you don't want to create health events for the items you select, clear Enable Health Monitoring in the
targets covered by this test.
10. Choose monitoring conditions. You can set custom thresholds for health-event generation by entering
threshold values. Whenever the value of the condition goes above its selected threshold for the selected
network or subnetwork pair, a health event is generated.
11. Select Save to save the configuration.
Walkthrough
Go to the Network Performance Monitor dashboard view. To get a summary of the health of the different tests you
created, look at the Service Connectivity Monitor page.
Select the tile to view the details of the tests on the Tests page. In the table on the left, you can view the point-in-
time health and value of the service response time, network latency, and packet loss for all the tests. Use the
Network State Recorder control to view the network snapshot at another time in the past. Select the test in the
table that you want to investigate. In the charts in the pane on the right, you can view the historical trend of the
loss, latency, and response time values. Select the Test Details link to view the performance from each node.
In the Test Nodes view, you can observe the network connectivity from each node. Select the node that has
performance degradation. This is the node where the application is observed to be running slow.
Determine whether poor application performance is because of the network or an issue on the application
provider's end by observing the correlation between the response time of the application and the network latency.
Application issue: A spike in the response time but consistency in the network latency suggests that the
network is working fine and the problem might be due to an issue on the application end.
Network issue: A spike in response time accompanied with a corresponding spike in network latency
suggests that the increase in response time might be due to an increase in network latency.
After you determine that the problem is because of the network, select the Topology view link to identify the
troublesome hop on the topology map. An example is shown in the following image. Out of the 105-ms total
latency between the node and the application endpoint, 96 ms is because of the hop marked in red. After you
identify the troublesome hop, you can take corrective action.
Diagnostics
If you observe an abnormality, follow these steps:
If the service response time, network loss, and latency are shown as NA, one or more of the following
reasons might be the cause:
The application is down.
The node used for checking network connectivity to the service is down.
The target entered in the test configuration is incorrect.
The node doesn't have any network connectivity.
If a valid service response time is shown but network loss as well as latency are shown as NA, one or more
of the following reasons might be the cause:
If the node used for checking network connectivity to the service is a Windows client machine, either the
target service is blocking ICMP requests or a network firewall is blocking ICMP requests that originate
from the node.
The Perform network measurements check box is blank in the test configuration.
If the service response time is NA but network loss as well as latency are valid, the target service might not
be a web application. Edit the test configuration, and choose the test type as Network instead of Web.
If the application is running slow, determine whether poor application performance is because of the
network or an issue on the application provider's end.
FIELD GCC
MS Teams gov.teams.microsoft.us
Next steps
Search logs to view detailed network performance data records.
ExpressRoute Monitor
10/24/2019 • 5 minutes to read • Edit Online
You can use the Azure ExpressRoute Monitor capability in Network Performance Monitor to monitor end-to-end
connectivity and performance between your branch offices and Azure, over Azure ExpressRoute. Key advantages
are:
Autodetection of ExpressRoute circuits associated with your subscription.
Tracking of bandwidth utilization, loss and latency at the circuit, peering, and Azure Virtual Network level for
ExpressRoute.
Discovery of network topology of your ExpressRoute circuits.
Configuration
To open the configuration for Network Performance Monitor, open the Network Performance Monitor solution
and select Configure.
Configure network security group rules
For the servers in Azure that are used for monitoring via Network Performance Monitor, configure network
security group (NSG ) rules to allow TCP traffic on the port used by Network Performance Monitor for synthetic
transactions. The default port is 8084. This configuration allows the Log Analytics agent installed on Azure VMs to
communicate with an on-premises monitoring agent.
For more information about NSGs, seeNetwork security groups.
NOTE
Before you continue with this step, install the on-premises server agent and the Azure server agent, and run the
EnableRules.ps1 PowerShell script.
NOTE
The solution currently discovers only ExpressRoute private peerings.
NOTE
Only private peerings connected to the virtual networks associated with the subscription linked with this Log
Analytics workspace are discovered. If ExpressRoute is connected to virtual networks outside of the subscription
linked to this workspace, create a Log Analytics workspace in those subscriptions. Then use Network Performance
Monitor to monitor those peerings.
After the discovery is complete, the discovered private peering connections are listed in a table. The
monitoring for these peerings is initially in a disabled state.
Enable monitoring of the ExpressRoute peering connections
1. Select the private peering connection you want to monitor.
2. In the pane on the right, select the Monitor this Peering check box.
3. If you intend to create health events for this connection, select Enable Health Monitoring for this
peering.
4. Choose monitoring conditions. You can set custom thresholds for health event generation by entering
threshold values. Whenever the value of the condition goes above its selected threshold for the peering
connection, a health event is generated.
5. Select Add Agents to choose the monitoring agents you intend to use for monitoring this peering
connection. Make sure that you add agents on both ends of the connection. You need at least one agent in
the virtual network connected to this peering. You also need at least one on-premises agent connected to
this peering.
6. Select Save to save the configuration.
After you enable the rules and select values and agents, wait 30 to 60 minutes for the values to populate and the
ExpressRoute Monitoring tiles to appear. When you see the monitoring tiles, your ExpressRoute circuits and
connection resources are now monitored by Network Performance Monitor.
NOTE
This capability works reliably on workspaces that have upgraded to the new query language.
Walkthrough
The Network Performance Monitor dashboard shows an overview of the health of ExpressRoute circuits and
peering connections.
Circuits list
To see a list of all monitored ExpressRoute circuits, select theExpressRoute circuitstile. You can select a circuit and
view its health state, trend charts for packet loss, bandwidth utilization, and latency. The charts are interactive. You
can select a custom time window for plotting the charts. Drag the mouse over an area on the chart to zoom in and
see fine-grained data points.
Peerings list
To bring up a list of all connections to virtual networks over private peering, select the Private Peerings tile on the
dashboard. Here, you can select a virtual network connection and view its health state, trend charts for packet loss,
bandwidth utilization, and latency.
Circuit topology
To view circuit topology, select the Topology tile. This action takes you to the topology view of the selected circuit
or peering. The topology diagram provides the latency for each segment on the network, and each layer 3 hop is
represented by a node of the diagram. Selecting a hop reveals more details about the hop. To increase the level of
visibility to include on-premises hops, move the slider bar under FILTERS. Moving the slider bar to the left or right
increases or decreases the number of hops in the topology graph. The latency across each segment is visible, which
allows for faster isolation of high-latency segments on your network.
5511 The traffic is not passing through the intended virtual network
Circuit is down. Network Performance Monitor notifies you as soon as the connectivity between your on-
premises resources and Azure virtual networks is lost. This notification helps you take proactive action before you
receive user escalations and reduce downtime.
Traffic not flowing through intended circuit. Network Performance Monitor notifies you whenever traffic isn't
flowing through the intended ExpressRoute circuit. This issue can happen if the circuit is down and traffic is flowing
through the backup route. It also can happen if there's a routing issue. This information helps you proactively
manage any configuration issues in your routing policies and make sure that the most optimal and secure route is
used.
Traffic not flowing through primary circuit. Network Performance Monitor notifies you when traffic is flowing
through the secondary ExpressRoute circuit. Even though you won't experience any connectivity issues in this case,
proactively troubleshooting the issues with the primary circuit makes you better prepared.
Degradation due to peak utilization. You can correlate the bandwidth utilization trend with the latency trend to
identify whether the Azure workload degradation is due to a peak in bandwidth utilization or not. Then you can
take action accordingly.
Next steps
Search logs to view detailed network performance data records.
Pricing changes for Azure Network Performance
Monitor
10/24/2019 • 4 minutes to read • Edit Online
We have listened to your feedback and recently introduced a new pricing experience for various monitoring
services across Azure. This article captures the pricing changes related to Azure Network Performance Monitor
(NPM ) in an easy-to-read question and answer format.
Network Performance Monitor consists of three components:
Performance Monitor
Service Endpoint Monitor
ExpressRoute Monitor
The following sections explain the pricing changes for the NPM components.
Performance Monitor
How was usage of Performance Monitor billed in the old model?
The billing for NPM was based on the usage and consumption of two components:
Nodes: All synthetic transactions originate and terminate at the nodes. Nodes are also referred to as agents or
Microsoft Management Agents.
Data: The results of the various network tests are stored in the Log Analytics workspace.
Under the old model, the bill was computed based on the number of nodes and the volume of data generated.
How is usage of Performance Monitor charged under the new model?
The Performance Monitor feature in NPM is now billed based on a combination of:
Subnet links monitored
Data volume
What is a subnet link?
Performance Monitor monitors connectivity between two or more locations on the network. The connection
between a group of nodes or agents on one subnet, and a group of nodes on another subnet, is called a subnet link.
I have two subnets (A and B ), and I have several agents on each subnet. Performance Monitor monitors
connectivity from all agents on subnet A to all agents on subnet B. Will I be charged based on the
number of inter-subnet connections?
No. For billing purposes, all connections from subnet A to subnet B are grouped together into one subnet link.
You're billed for a single connection. Performance Monitor continues to monitor connectivity between various
agents on each subnet.
What are the costs for monitoring a subnet link?
For the cost of monitoring a single subnet link for the entire month, see the Ping Mesh section.
What are the charges for data that Performance Monitor generates?
The charge for ingestion (data upload to Log Analytics workspace in Azure Monitor, processing, and indexing) is
available on the pricing page for Log Analytics, in the Data Ingestion section. The charge for data retention (that is,
data retained at customer's option, beyond the first month) is also available on the pricing page, in the Data
Retention section.
ExpressRoute Monitor
What are the charges for usage of ExpressRoute Monitor?
Charges for ExpressRoute Monitor are billed based on the volume of data generated during monitoring. For more
information, see "What are the charges for data that Performance Monitor generates?"
I use ExpressRoute Monitor to monitor multiple ExpressRoute circuits. Am I charged based on the
number of circuits being monitored?
You are not charged based on either the number of circuits or the type of peering (for example, private peering,
Microsoft peering). You are charged based on the volume of data, as explained previously.
What is the volume of data generated when ExpressRoute monitors a single circuit?
The volume of data generated per month, when ExpressRoute monitors a private peering connection, is as follows:
50th 192
60th 256
70th 360
80th 498
90th 870
95th 1560
According to this table, customers at the 50th percentile pay for 192 MB of data. At USD $2.30/GB for the first
month, the cost incurred for monitoring a circuit is USD $0.43 (= 192 * 2.30 / 1024).
What are some reasons for variations in the volume of data?
The volume of monitoring data generated depends on several factors, such as:
Number of agents. The accuracy of fault isolation increases with an increase in the number of agents.
Number of hops on the network.
Number of paths between the source and the destination.
Customers at the higher percentiles (in the preceding table) usually monitor their circuits from several vantage
points on their on-premises network. Multiple agents are also placed deeper in the network, farther from the
service provider edge router. The agents are often placed at several user sites, branches, and racks in datacenters.
References
Log Analytics Pricing FAQ: The FAQ section has information on free tier, per node pricing and other pricing details.
Network Performance Monitor solution FAQ
11/26/2019 • 17 minutes to read • Edit Online
This article captures the frequently asked questions (FAQs) about Network Performance Monitor (NPM ) in Azure
Network Performance Monitor is a cloud-based hybrid network monitoring solution that helps you monitor
network performance between various points in your network infrastructure. It also helps you monitor network
connectivity to service and application endpoints and monitor the performance of Azure ExpressRoute.
Network Performance Monitor detects network issues like traffic blackholing, routing errors, and issues that
conventional network monitoring methods aren't able to detect. The solution generates alerts and notifies you
when a threshold is breached for a network link. It also ensures timely detection of network performance issues
and localizes the source of the problem to a particular network segment or device.
More information on the various capabilities supported by Network Performance Monitor is available online.
The script configures only Windows Firewall locally. If you have network firewall or Network Security Group
(NSG ) rules, make sure that they allow the traffic destined for the TCP port used by NPM.
How many agents should I use?
You should use at least one agent for each subnet that you want to monitor.
What is the maximum number of agents I can use or I see error ".... you've reached your Configuration limit"?
NPM limits the number of IPs to 5000 IPs per workspace. If a node has both IPv4 and IPv6 addresses, this will
count as 2 IPs for that node. Hence, this limit of 5000 IPs would decide the upper limit on the number of agents.
You can delete the inactive agents from Nodes tab in NPM >> Configure. NPM also maintains history of all the
IPs that were ever assigned to the VM hosting the agent and each is counted as separate IP contributing to that
upper limit of 5000 IPs. To free up IPs for your workspace, you can use the Nodes page to delete the IPs that are
not in use.
Monitoring
How are loss and latency calculated
Source agents send either TCP SYN requests (if TCP is chosen as the protocol for monitoring) or ICMP ECHO
requests (if ICMP is chosen as the protocol for monitoring) to destination IP at regular intervals to ensure that all
the paths between the source-destination IP combination are covered. The percentage of packets received and
packet round-trip time is measured to calculate the loss and latency of each path. This data is aggregated over the
polling interval and over all the paths to get the aggregated values of loss and latency for the IP combination for
the particular polling interval.
With what frequency does the source agent send packets to the destination for monitoring?
For Performance Monitor and ExpressRoute Monitor capabilities, the source sends packets every 5 seconds and
records the network measurements. This data is aggregated over a 3-minute polling interval to calculate the
average and peak values of loss and latency. For Service Connectivity Monitor capability, the frequency of sending
the packets for network measurement is determined by the frequency entered by the user for the specific test while
configuring the test.
How many packets are sent for monitoring?
The number of packets sent by the source agent to destination in a polling is adaptive and is decided by our
proprietary algorithm, which can be different for different network topologies. More the number of network paths
between the source-destination IP combination, more is the number of packets that are sent. The system ensures
that all paths between the source-destination IP combination are covered.
How does NPM discover network topology between source and destination?
NPM uses a proprietary algorithm based on Traceroute to discover all the paths and hops between source and
destination.
Does NPM provide routing and switching level info
Though NPM can detect all the possible routes between the source agent and the destination, it does not provide
visibility into which route was taken by the packets sent by your specific workloads. The solution can help you
identify the paths and underlying network hops, which are adding more latency than you expected.
Why are some of the paths unhealthy?
Different network paths can exist between the source and destination IPs and each path can have a different value
of loss and latency. NPM marks those paths as unhealthy (denoted with red color) for which the values of loss
and/or latency is greater than the respective threshold set in the monitoring configuration.
What does a hop in red color signify in the network topology map?
If a hop is red, it signifies that it is part of at-least one unhealthy path. NPM only marks the paths as unhealthy, it
does not segregate the health status of each path. To identify the troublesome hops, you can view the hop-by-hop
latency and segregate the ones adding more than expected latency.
How does fault localization in Performance Monitor work?
NPM uses a probabilistic mechanism to assign fault-probabilities to each network path, network segment, and the
constituent network hops based on the number of unhealthy paths they are a part of. As the network segments and
hops become part of more number of unhealthy paths, the fault-probability associated with them increases. This
algorithm works best when you have many nodes with NPM agent connected to each other as this increases the
data points for calculating the fault-probabilities.
How can I create alerts in NPM?
Refer to alerts section in the documentation for step-by-step instructions.
What are the default Log Analytics queries for alerts
Performance monitor query
NetworkMonitoring
| where (SubType == "SubNetwork" or SubType == "NetworkPath")
| where (LossHealthState == "Unhealthy" or LatencyHealthState == "Unhealthy") and RuleName == "<<your rule
name>>"
NetworkMonitoring
| where (SubType == "EndpointHealth" or SubType == "EndpointPath")
| where (LossHealthState == "Unhealthy" or LatencyHealthState == "Unhealthy" or ServiceResponseHealthState ==
"Unhealthy" or LatencyHealthState == "Unhealthy") and TestName == "<<your test name>>"
NetworkMonitoring
| where (SubType == "ERCircuitTotalUtilization") and (UtilizationHealthState == "Unhealthy") and
CircuitResourceId == "<<your circuit resource ID>>"
Private peering
NetworkMonitoring
| where (SubType == "ExpressRoutePeering" or SubType == "ERVNetConnectionUtilization" or SubType ==
"ExpressRoutePath")
| where (LossHealthState == "Unhealthy" or LatencyHealthState == "Unhealthy" or UtilizationHealthState ==
"Unhealthy") and CircuitName == "<<your circuit name>>" and VirtualNetwork == "<<vnet name>>"
Microsoft peering
NetworkMonitoring
| where (SubType == "ExpressRoutePeering" or SubType == "ERMSPeeringUtilization" or SubType ==
"ExpressRoutePath")
| where (LossHealthState == "Unhealthy" or LatencyHealthState == "Unhealthy" or UtilizationHealthState ==
"Unhealthy") and CircuitName == ""<<your circuit name>>" and PeeringType == "MicrosoftPeering"
Common query
NetworkMonitoring
| where (SubType == "ExpressRoutePeering" or SubType == "ERVNetConnectionUtilization" or SubType ==
"ERMSPeeringUtilization" or SubType == "ExpressRoutePath")
| where (LossHealthState == "Unhealthy" or LatencyHealthState == "Unhealthy" or UtilizationHealthState ==
"Unhealthy")
NetworkMonitoring
| where SubType == "ERMSPeeringUtilization"
| project
CircuitName,PeeringName,PrimaryBytesInPerSecond,PrimaryBytesOutPerSecond,SecondaryBytesInPerSecond,SecondaryByt
esOutPerSecond
For private peering level information, use the below mentioned query in Log Search
NetworkMonitoring
| where SubType == "ERVNetConnectionUtilization"
| project
CircuitName,PeeringName,PrimaryBytesInPerSecond,PrimaryBytesOutPerSecond,SecondaryBytesInPerSecond,SecondaryByt
esOutPerSecond
For circuit level information, use the below mentioned query in Log Search
NetworkMonitoring
| where SubType == "ERCircuitTotalUtilization"
| project CircuitName, PrimaryBytesInPerSecond,
PrimaryBytesOutPerSecond,SecondaryBytesInPerSecond,SecondaryBytesOutPerSecond
Troubleshoot
Why are some of the hops marked as unidentified in the network topology view?
NPM uses a modified version of traceroute to discover the topology from the source agent to the destination. An
unidentified hop represents that the network hop did not respond to the source agent's traceroute request. If three
consecutive network hops do not respond to the agent's traceroute, the solution marks the unresponsive hops as
unidentified and does not try to discover more hops.
A hop may not respond to a traceroute in one or more of the below scenarios:
The routers have been configured to not reveal their identity.
The network devices are not allowing ICMP_TTL_EXCEEDED traffic.
A firewall is blocking the ICMP_TTL_EXCEEDED response from the network device.
When either of the endpoints lies in Azure, traceroute shows up unidentified hops as Azure ndrastructure does not
reveal identity to traceroute.
I get alerts for unhealthy tests but I do not see the high values in NPM's loss and latency graph. How do I check
what is unhealthy?
NPM raises an alert if end to end latency between source and destination crosses the threshold for any path
between them. Some networks have multiple paths connecting the same source and destination. NPM raises an
alert is any path is unhealthy. The loss and latency seen in the graphs is the average value for all the paths, hence it
may not show the exact value of a single path. To understand where the threshold has been breached, look for the
"SubType" column in the alert. If the issue is caused by a path the SubType value will be NetworkPath (for
Performance Monitor tests), EndpointPath (for Service Connectivity Monitor tests) and ExpressRoutePath (for
ExpressRotue Monitor tests).
Sample Query to find is path is unhealthy:
NetworkMonitoring
| where ( SubType == "ExpressRoutePath")
| where (LossHealthState == "Unhealthy" or LatencyHealthState == "Unhealthy" or UtilizationHealthState ==
"Unhealthy") and CircuitResourceID =="<your ER circuit ID>" and ConnectionResourceId == "<your ER connection
resource id>"
| project SubType, LossHealthState, LatencyHealthState, MedianLatency
Why does my test show unhealthy but the topology does not
NPM monitors end-to-end loss, latency, and topology at different intervals. Loss and latency are measured once
every 5 seconds and aggregated every three minutes (for Performance Monitor and Express Route Monitor) while
topology is calculated using traceroute once every 10 minutes. For example, between 3:44 and 4:04, topology may
be updated three times (3:44, 3:54, 4:04), but loss and latency are updated about seven times (3:44, 3:47, 3:50, 3:53,
3:56, 3:59, 4:02). The topology generated at 3:54 will be rendered for the loss and latency that gets calculated at
3:56, 3:59 and 4:02. Suppose you get an alert that your ER circuit was unhealthy at 3:59. You log on to NPM and
try to set the topology time to 3:59. NPM will render the topology generated at 3:54. To understand the last known
topology of your network, compare the fields TimeProcessed (time at which loss and latency was calculated) and
TracerouteCompletedTime(time at which topology was calculated).
What is the difference between the fields E2EMedianLatency and AvgHopLatencyList in the
NetworkMonitoring table
E2EMedianLatency is the latency updated every three minutes after aggregating the results of tcp ping tests,
whereas AvgHopLatencyList is updated every 10 mins based on traceroute. To understand the exact time at which
E2EMedianLatency was calculated, use the field TimeProcessed. To understand the exact time at which traceroute
completed and updated AvgHopLatencyList, use the field TracerouteCompletedTime
Why does hop-by-hop latency numbers differ from HopLatencyValues
HopLatencyValues are source to endpoint. For Example: Hops - A,B,C. AvgHopLatency - 10,15,20. This means
source to A latency = 10, source to B latency = 15 and source to C latency is 20. UI will calculate A-B hop latency as
5 in the topology
The solution shows 100% loss but there is connectivity between the source and destination
This can happen if either the host firewall or the intermediate firewall (network firewall or Azure NSG ) is blocking
the communication between the source agent and the destination over the port being used for monitoring by NPM
(by default the port is 8084, unless the customer has changed this).
To verify that the host firewall is not blocking the communication on the required port, view the health status of
the source and destination nodes from the following view: Network Performance Monitor -> Configuration ->
Nodes. If they are unhealthy, view the instructions and take corrective action. If the nodes are healthy, move to
the step b. below.
To verify that an intermediate network firewall or Azure NSG is not blocking the communication on the
required port, use the third-party PsPing utility using the below instructions:
psping utility is available for download here
Run following command from the source node.
psping -n 15 <destination node IPAddress>:portNumber By default NPM uses 8084 port. In case
you have explicitly changed this by using the EnableRules.ps1 script, enter the custom port
number you are using). This is a ping from Azure machine to on-premises
Check if the pings are successful. If not, then it indicates that an intermediate network firewall or Azure NSG is
blocking the traffic on this port.
Now, run the command from destination node to source node IP.
There is loss from node A to B, but not from node B to A. Why?
As the network paths between A to B can be different from the network paths between B to A, different values for
loss and latency can be observed.
Why are all my ExpressRoute circuits and peering connections not being discovered?
NPM now discovers ExpressRoute circuits and peering connections in all subscriptions to which the user has
access. Choose all the subscriptions where your Express Route resources are linked and enable monitoring for each
discovered resource. NPM looks for connection objects when discovering a private peering, so please check if a
VNET is associated with your peering.
The ER Monitor capability has a diagnostic message "Traffic is not passing through ANY circuit". What does that
mean?
There can be a scenario where there is a healthy connection between the on-premises and Azure nodes but the
traffic is not going over the ExpressRoute circuit configured to be monitored by NPM.
This can happen if:
The ER circuit is down.
The route filters are configured in such a manner that they give priority to other routes (such as a VPN
connection or another ExpressRoute circuit) over the intended ExpressRoute circuit.
The on-premises and Azure nodes chosen for monitoring the ExpressRoute circuit in the monitoring
configuration, do not have connectivity to each other over the intended ExpressRoute circuit. Ensure that you
have chosen correct nodes that have connectivity to each other over the ExpressRoute circuit you intend to
monitor.
While configuring monitoring of my ExpressRoute circuit, the Azure nodes are not being detected.
This can happen if the Azure nodes are connected through Operations Manager. The ExpressRoute Monitor
capability supports only those Azure nodes that are connected as Direct Agents.
I cannot Discover by ExpressRoute circuits in the OMS portal
Though NPM can be used both from the Azure portal as well as the OMS portal, the circuit discovery in the
ExpressRoute Monitor capability works only through the Azure portal. Once the circuits are discovered through the
Azure portal, you can then use the capability in either of the two portals.
In the Service Connectivity Monitor capability, the service response time, network loss, as well as latency are
shown as NA
This can happen if one or more is true:
The service is down.
The node used for checking network connectivity to the service is down.
The target entered in the test configuration is incorrect.
The node doesn't have any network connectivity.
In the Service Connectivity Monitor capability, a valid service response time is shown but network loss as well as
latency are shown as NA
This can happen if one or more is true:
If the node used for checking network connectivity to the service is a Windows client machine, either the target
service is blocking ICMP requests or a network firewall is blocking ICMP requests that originate from the node.
The Perform network measurements check box is blank in the test configuration.
In the Service Connectivity Monitor capability, the service response time is NA but network loss as well as
latency are valid
This can happen if the target service is not a web application but the test is configured as a Web test. Edit the test
configuration, and choose the test type as Network instead of Web.
Miscellaneous
Is there a performance impact on the node being used for monitoring?
NPM process is configured to stop if it utilizes more than 5% of the host CPU resources. This is to ensure that you
can keep using the nodes for their usual workloads without impacting performance.
Does NPM edit firewall rules for monitoring?
NPM only creates a local Windows Firewall rule on the nodes on which the EnableRules.ps1 Powershell script is
run to allow the agents to create TCP connections with each other on the specified port. The solution does not
modify any network firewall or Network Security Group (NSG ) rules.
How can I check the health of the nodes being used for monitoring?
You can view the health status of the nodes being used for monitoring from the following view: Network
Performance Monitor -> Configuration -> Nodes. If a node is unhealthy, you can view the error details and take
the suggested action.
Can NPM report latency numbers in microseconds?
NPM rounds the latency numbers in the UI and in milliseconds. The same data is stored at a higher granularity
(sometimes up to four decimal places).
Next steps
Learn more about Network Performance Monitor by referring to Network Performance Monitor solution in
Azure.
Monitoring solutions in Azure Monitor
12/18/2019 • 5 minutes to read • Edit Online
Monitoring solutions leverage services in Azure to provide additional insight into the operation of a
particular application or service. This article provides a brief overview of monitoring solutions in Azure
and details on using and installing them.
NOTE
Monitoring solutions were previously referred to as management solutions.
Monitoring solutions typically collect log data and provide queries and views to analyze collected data.
They may also leverage other services such as Azure Automation to perform actions related to the
application or service.
You can add monitoring solutions to Azure Monitor for any applications and services that you use.
They're typically available at no cost but collect data that could invoke usage charges. In addition to
solutions provided by Microsoft, partners and customers can create management solutions to be used
in their own environment or made available to customers through the community.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still
stored in a Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We
are updating the terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology
changes for details.
Click on the name of a solution to open its summary page. This page displays any views included in
the solution and provides different options for the solution itself and its workspace. View the summary
page for a solution by using one of the procedures above to list solutions and then click on the name of
the solution.
Install a monitoring solution
Monitoring solutions from Microsoft and partners are available from the Azure Marketplace. You can
search available solutions and install them using the following procedure. When you install a solution,
you must select a Log Analytics workspace where the solution will be installed and where its data will
be collected.
1. From the list of solutions for your subscription, click Add.
2. Browse or search for a solution. You can also browse solutions from this search link.
3. Locate the monitoring solution you want and read through its description.
4. Click Create to start the installation process.
5. When the installation process starts, you're prompted to specify the Log Analytics workspace and
provide any required configuration for the solution.
When you add a monitoring solution to your subscription, it's automatically deployed by default to all Windows
and Linux agents connected to your Log Analytics workspace. You may want to manage your costs and limit the
amount of data collected for a solution by limiting it to a particular set of agents. This article describes how to use
Solution Targeting which is a feature that allows you to apply a scope to your solutions.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log
Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology
to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Next steps
Learn more about monitoring solutions including the solutions that are available to install in your environment
at Add Azure Log Analytics monitoring solutions to your workspace.
Learn more about creating computer groups at Computer groups in Azure Monitor log queries.
Optimize your Active Directory environment with the
Active Directory Health Check solution in Azure
Monitor
1/17/2020 • 9 minutes to read • Edit Online
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log
Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology
to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
You can use the Active Directory Health Check solution to assess the risk and health of your server environments
on a regular interval. This article helps you install and use the solution so that you can take corrective actions for
potential problems.
This solution provides a prioritized list of recommendations specific to your deployed server infrastructure. The
recommendations are categorized across four focus areas, which help you quickly understand the risk and take
action.
The recommendations are based on the knowledge and experience gained by Microsoft engineers from thousands
of customer visits. Each recommendation provides guidance about why an issue might matter to you and how to
implement the suggested changes.
You can choose focus areas that are most important to your organization and track your progress toward running
a risk free and healthy environment.
After you've added the solution and a check is completed, summary information for focus areas is shown on the
AD Health Check dashboard for the infrastructure in your environment. The following sections describe how to
use the information on the AD Health Check dashboard, where you can view and then take recommended
actions for your Active Directory server infrastructure.
Prerequisites
The Active Directory Health Check solution requires a supported version of .NET Framework 4.6.2 or above
installed on each computer that has the Log Analytics agent for Windows (also referred to as the Microsoft
Monitoring Agent (MMA)) installed. The agent is used by System Center 2016 - Operations Manager,
Operations Manager 2012 R2, and Azure Monitor.
The solution supports domain controllers running Windows Server 2008 and 2008 R2, Windows Server
2012 and 2012 R2, and Windows Server 2016.
A Log Analytics workspace to add the Active Directory Health Check solution from the Azure marketplace
in the Azure portal. There is no additional configuration required.
NOTE
After you've added the solution, the AdvisorAssessment.exe file is added to servers with agents. Configuration data is
read and then sent to Azure Monitor in the cloud for processing. Logic is applied to the received data and the cloud
service records the data.
To perform the health check against your domain controllers that are members of the domain to be evaluated,
each domain controller in that domain requires an agent and connectivity to Azure Monitor using one of the
following supported methods:
1. Install the Log Analytics agent for Windows if the domain controller is not already monitored by System Center
2016 - Operations Manager or Operations Manager 2012 R2.
2. If it is monitored with System Center 2016 - Operations Manager or Operations Manager 2012 R2 and the
management group is not integrated with Azure Monitor, the domain controller can be multi-homed with
Azure Monitor to collect data and forward to the service and still be monitored by Operations Manager.
3. Otherwise, if your Operations Manager management group is integrated with the service, you need to add the
domain controllers for data collection by the service following the steps under add agent-managed computers
after you enable the solution in your workspace.
The agent on your domain controller which reports to an Operations Manager management group, collects data,
forwards to its assigned management server, and then is sent directly from a management server to Azure
Monitor. The data is not written to the Operations Manager databases.
4. You can take corrective actions suggested in Suggested Actions. When the item has been addressed, later
assessments records that recommended actions were taken and your compliance score will increase.
Corrected items appear as Passed Objects.
Ignore recommendations
If you have recommendations that you want to ignore, you can create a text file that Azure Monitor will use to
prevent recommendations from appearing in your assessment results.
To identify recommendations that you will ignore
Use log analytics to create queries and analyze log data in Azure Monitor by clicking Logs in the Azure Monitor
menu in the Azure portal.
Use the following query to list recommendations that have failed for computers in your environment.
ADAssessmentRecommendation | where RecommendationResult == "Failed" | sort by Computer asc | project Computer,
RecommendationId, Recommendation
Choose recommendations that you want to ignore. You’ll use the values for RecommendationId in the next
procedure.
To create and use an IgnoreRecommendations.txt text file
1. Create a file named IgnoreRecommendations.txt.
2. Paste or type each RecommendationId for each recommendation that you want Azure Monitor to ignore on
a separate line and then save and close the file.
3. Put the file in the following folder on each computer where you want Azure Monitor to ignore
recommendations.
On computers with the Microsoft Monitoring Agent (connected directly or through Operations
Manager) - SystemDrive:\Program Files\Microsoft Monitoring Agent\Agent
On the Operations Manager 2012 R2 management server - SystemDrive:\Program Files\Microsoft
System Center 2012 R2\Operations Manager\Server
On the Operations Manager 2016 management server - SystemDrive:\Program Files\Microsoft System
Center 2016\Operations Manager\Server
To verify that recommendations are ignored
After the next scheduled health check runs, by default every seven days, the specified recommendations are
marked Ignored and will not appear on the dashboard.
1. You can use the following log queries to list all the ignored recommendations.
2. If you decide later that you want to see ignored recommendations, remove any IgnoreRecommendations.txt
files, or you can remove RecommendationIDs from them.
AD Health Check solutions FAQ
What checks are performed by the AD Assessment solution?
The following query shows a description of all checks currently performed:
ADAssessmentRecommendation
| distinct RecommendationId, FocusArea, ActionArea, Recommendation, Description
| sort by FocusArea,ActionArea, Recommendation
Next steps
Use Azure Monitor log queries to learn how to analyze detailed AD Health Check data and recommendations.
Monitor Active Directory replication status with
Azure Monitor
12/16/2019 • 8 minutes to read • Edit Online
Active Directory is a key component of an enterprise IT environment. To ensure high availability and high
performance, each domain controller has its own copy of the Active Directory database. Domain controllers
replicate with each other in order to propagate changes across the enterprise. Failures in this replication process
can cause a variety of problems across the enterprise.
The AD Replication Status solution regularly monitors your Active Directory environment for any replication
failures.
NOTE
Previous versions of this article referred to Log Analytics as its own service. Its functionality hasn't changed, but it's been
renamed to the log feature of Azure Monitor. With this rename, this article describes its data being stored in a Log Analytics
workspace in Azure Monitor. For more information, see Azure Monitor branding changes.
NOTE
These changes do not take effect until you restart the Microsoft Monitoring Agent service (HealthService.exe).
Install solution
Follow the process described in Install a monitoring solution to add the Active Directory Replication Status
solution to your Log Analytics workspace. There is no further configuration required.
SCOM AGENT
DATA SENT VIA
AZURE SCOM MANAGEMENT COLLECTION
PLATFORM DIRECT AGENT SCOM AGENT STORAGE REQUIRED? GROUP FREQUENCY
When you click the tile, you can view more information about the errors.
Destination Server Status and Source Server Status
These columns show the status of destination servers and source servers that are experiencing replication errors.
The number after each domain controller name indicates the number of replication errors on that domain
controller.
The errors for both destination servers and source servers are shown because some problems are easier to
troubleshoot from the source server perspective and others from the destination server perspective.
In this example, you can see that many destination servers have roughly the same number of errors, but there's
one source server (ADDC35) that has many more errors than all the others. It's likely that there is some problem
on ADDC35 that is causing it to fail to send data to its replication partners. Fixing the problems on ADDC35 might
resolve many of the errors that appear in the destination server area.
Replication Error Types
This area gives you information about the types of errors detected throughout your enterprise. Each error has a
unique numerical code and a message that can help you determine the root cause of the error.
The donut at the top gives you an idea of which errors appear more and less frequently in your environment.
It shows you when multiple domain controllers experience the same replication error. In this case, you may be able
to discover or identify a solution on one domain controller, then repeat it on other domain controllers affected by
the same error.
Tombstone Lifetime
The tombstone lifetime determines how long a deleted object, referred to as a tombstone, is retained in the Active
Directory database. When a deleted object passes the tombstone lifetime, a garbage collection process
automatically removes it from the Active Directory database.
The default tombstone lifetime is 180 days for most recent versions of Windows, but it was 60 days on older
versions, and it can be changed explicitly by an Active Directory administrator.
It's important to know if you're having replication errors that are approaching or are past the tombstone lifetime. If
two domain controllers experience a replication error that persists past the tombstone lifetime, replication is
disabled between those two domain controllers, even if the underlying replication error is fixed.
The Tombstone Lifetime area helps you identify places where disabled replication is in danger of happening. Each
error in the Over 100% TSL category represents a partition that has not replicated between its source and
destination server for at least the tombstone lifetime for the forest.
In this situation, simply fixing the replication error will not be enough. At a minimum, you need to manually
investigate to identify and clean up lingering objects before you can restart replication. You may even need to
decommission a domain controller.
In addition to identifying any replication errors that have persisted past the tombstone lifetime, you also want to
pay attention to any errors falling into the 50-75% TSL or 75-100% TSL categories.
These are errors that are clearly lingering, not transient, so they likely need your intervention to resolve. The good
news is that they have not yet reached the tombstone lifetime. If you fix these problems promptly and before they
reach the tombstone lifetime, replication can restart with minimal manual intervention.
As noted earlier, the dashboard tile for the AD Replication Status solution shows the number of critical replication
errors in your environment, which is defined as errors that are over 75% of tombstone lifetime (including errors
that are over 100% of TSL ). Strive to keep this number at 0.
NOTE
All the tombstone lifetime percentage calculations are based on the actual tombstone lifetime for your Active Directory
forest, so you can trust that those percentages are accurate, even if you have a custom tombstone lifetime value set.
From here, you can filter further, modify the log query, and so on. For more information about using the log
queries in Azure Monitor, see Analyze log data in Azure Monitor.
The HelpLink field shows the URL of a TechNet page with additional details about that specific error. You can
copy and paste this link into your browser window to see information about troubleshooting and fixing the error.
You can also click Export to export the results to Excel. Exporting the data can help you visualize replication error
data in any way you'd like.
AD Replication Status FAQ
Q: How often is AD replication status data updated? A: The information is updated every five days.
Q: Is there a way to configure how often this data is updated? A: Not at this time.
Q: Do I need to add all of my domain controllers to my Log Analytics workspace in order to see
replication status? A: No, only a single domain controller must be added. If you have multiple domain controllers
in your Log Analytics workspace, data from all of them is sent to Azure Monitor.
Q: I don't want to add any domain controllers to my Log Analytics workspace. Can I still use the AD
Replication Status solution?
A: Yes. You can set the value of a registry key to enable it. See Enable non-domain controller.
Q: What is the name of the process that does the data collection? A: AdvisorAssessment.exe
Q: How long does it take for data to be collected? A: Data collection time depends on the size of the Active
Directory environment, but usually takes less than 15 minutes.
Q: What type of data is collected? A: Replication information is collected via LDAP.
Q: Is there a way to configure when data is collected? A: Not at this time.
Q: What permissions do I need to collect data? A: Normal user permissions to Active Directory are sufficient.
Next steps
Use Log queries in Azure Monitor to view detailed Active Directory Replication status data.
Azure Key Vault Analytics solution in Azure Monitor
12/6/2019 • 6 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
You can use the Azure Key Vault solution in Azure Monitor to review Azure Key Vault AuditEvent logs.
To use the solution, you need to enable logging of Azure Key Vault diagnostics and direct the diagnostics to a Log
Analytics workspace. It is not necessary to write the logs to Azure Blob storage.
NOTE
In January 2017, the supported way of sending logs from Key Vault to Log Analytics changed. If the Key Vault solution you
are using shows (deprecated) in the title, refer to migrating from the old Key Vault solution for steps you need to follow.
$workspaceId = "/subscriptions/d2e37fee-1234-40b2-5678-0b2199de3b50/resourcegroups/oi-default-east-
us/providers/microsoft.operationalinsights/workspaces/rollingbaskets"
OPERATIONS
SYSTEMS MANAGER
CENTER AGENT DATA
OPERATIONS OPERATIONS SENT VIA
MANAGER MANAGER MANAGEMENT COLLECTION
PLATFORM DIRECT AGENT AGENT AZURE REQUIRED? GROUP FREQUENCY
Azure • on arrival
After you click the Key Vault Analytics tile, you can view summaries of your logs and then drill in to details for
the following categories:
Volume of all key vault operations over time
Failed operation volumes over time
Average operational latency by operation
Quality of service for operations with the number of operations that take more than 1000 ms and a list of
operations that take more than 1000 ms
To view details for any operation
1. On the Overview page, click the Key Vault Analytics tile.
2. On the Azure Key Vault dashboard, review the summary information in one of the blades, and then click
one to view detailed information about it in the log search page.
On any of the log search pages, you can view results by time, detailed results, and your log search history.
You can also filter by facets to narrow the results.
PROPERTY DESCRIPTION
Type AzureDiagnostics
SourceSystem Azure
Category AuditEvent
CorrelationId An optional GUID that the client can pass to correlate client-
side logs with service-side (Key Vault) logs.
httpStatusCode_d HTTP status code returned by the request (for example, 200)
OperationVersion REST API version requested by the client (for example 2015-
06-01)
ResourceId Azure Resource Manager Resource ID. For Key Vault logs, this
is the Key Vault resource ID.
ResourceProvider MICROSOFT.KEYVAULT
ResourceType VAULTS
Data collected before the change is not visible in the new solution. You can continue to query for this data using
the old Type and field names.
Troubleshooting
Troubleshoot Azure Diagnostics
If you receive the following error message, the Microsoft.insights resource provider is not registered:
Failed to update diagnostics for 'resource'. {"code":"Forbidden","message":"Please register the subscription
'subscription id' with Microsoft.Insights."}
To register the resource provider, perform the following steps in the Azure portal:
1. In the navigation pane on the left, click Subscriptions
2. Select the subscription identified in the error message
3. Click Resource Providers
4. Find the Microsoft.insights provider
5. Click the Register link
Update your version of Azure PowerShell, follow the instructions in the Install Azure PowerShell article.
Next steps
Use Log queries in Azure Monitor to view detailed Azure Key Vault data.
Office 365 management solution in Azure (Preview)
1/16/2020 • 17 minutes to read • Edit Online
IMPORTANT
Solution update
This solution has been replaced by the Office 365 General Availability solution in Azure Sentinel and the Azure AD reporting
and monitoring solution. Together they provide an updated version of the previous Azure Monitor Office 365 solution with
an improved configuration experience. You can continue to use the existing solution until March 30, 2020.
Azure Sentinel is a cloud native Security Information and Event Management solution that ingests logs and provides
additional SIEM functionality including detections, investigations, hunting and machine learning driven insights. Using Azure
Sentinel will now provide you with ingestion of Office 365 SharePoint activity and Exchange management logs.
Azure AD reporting provides a more comprehensive view of logs from Azure AD activity in your environment, including sign
in events, audit events, and changes to your directory. To connect Azure AD logs, you can use either the Azure Sentinel
Azure AD connector or configure Azure AD logs integration with Azure Monitor.
The collection of Azure AD log is subjected to Azure Monitor pricing. See Azure Monitor pricing for more information.
To use the Azure Sentinel Office 365 solution:
1. Using Office 365 connector in Azure Sentinel affects the pricing for your workspace. For more information, see Azure
Sentinel pricing.
2. If you are already using the Azure Monitor Office 365 solution, you must first uninstall it using the script in the Uninstall
section below.
3. Enable the Azure Sentinel solution on your workspace.
4. Go to the Data connectors page in Azure Sentinel and enable the Office 365 connector.
SigninLogs
| where ConditionalAccessStatus == "failure" or ConditionalAccessStatus == "notApplied"
| summarize count() by UserDisplayName
OfficeActivity
| where OfficeWorkload =~ "AzureActiveDirectory"
| sort by TimeGenerated desc
| summarize AggregatedValue = count() by Operation
AuditLogs
| summarize count() by OperationName
The Office 365 management solution allows you to monitor your Office 365 environment in Azure Monitor.
Monitor user activities on your Office 365 accounts to analyze usage patterns as well as identify behavioral
trends. For example, you can extract specific usage scenarios, such as files that are shared outside your
organization or the most popular SharePoint sites.
Monitor administrator activities to track configuration changes or high privilege operations.
Detect and investigate unwanted user behavior, which can be customized for your organizational needs.
Demonstrate audit and compliance. For example, you can monitor file access operations on confidential files,
which can help you with the audit and compliance process.
Perform operational troubleshooting by using log queries on top of Office 365 activity data of your
organization.
Uninstall
You can remove the Office 365 management solution using the process in Remove a management solution. This
will not stop data being collected from Office 365 into Azure Monitor though. Follow the procedure below to
unsubscribe from Office 365 and stop collecting data.
1. Save the following script as office365_unsubscribe.ps1.
param (
[Parameter(Mandatory=$True)][string]$WorkspaceName,
[Parameter(Mandatory=$True)][string]$ResourceGroupName,
[Parameter(Mandatory=$True)][string]$SubscriptionId,
[Parameter(Mandatory=$True)][string]$OfficeTennantId
)
$line='#-----------------------------------------------------------------------------------------------
--------------------------------------------------------------------------'
$line
IF ($Subscription -eq $null)
{Login-AzAccount -ErrorAction Stop}
$Subscription = (Select-AzSubscription -SubscriptionId $($SubscriptionId) -ErrorAction Stop)
$Subscription
$option = [System.StringSplitOptions]::RemoveEmptyEntries
$Workspace = (Set-AzOperationalInsightsWorkspace -Name $($WorkspaceName) -ResourceGroupName
$($ResourceGroupName) -ErrorAction Stop)
$Workspace
$WorkspaceLocation= $Workspace.Location
switch ($WorkspaceLocation) {
"USGov Virginia" {
"USGov Virginia" {
$domain='login.microsoftonline.us';
$authority = "https://login.microsoftonline.us/$adTenant";
$ARMResource ="https://management.usgovcloudapi.net/"; break} # US Gov
Virginia
default {
$domain='login.microsoftonline.com';
$authority = "https://login.windows.net/$adTenant";
$ARMResource ="https://management.azure.com/";break}
}
Function RESTAPI-Auth {
$global:SubscriptionID = $Subscription.SubscriptionId
# Set Resource URI to Azure Service Management API
$resourceAppIdURIARM=$ARMResource;
# Authenticate and Acquire Token
# Create Authentication Context tied to Azure AD Tenant
$authContext = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext" -
ArgumentList $authority
# Acquire token
$platformParameters = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.PlatformParameters" -
ArgumentList "Auto"
$global:authResultARM = $authContext.AcquireTokenAsync($resourceAppIdURIARM, $clientId, $redirectUri,
$platformParameters)
$global:authResultARM.Wait()
$authHeader = $global:authResultARM.Result.CreateAuthorizationHeader()
$authHeader
}
Function Office-UnSubscribe-Call{
#------------------------------------------------------------------------------------------------------
----------------------------------------
$authHeader = $global:authResultARM.Result.CreateAuthorizationHeader()
$ResourceName = "https://manage.office.com"
$SubscriptionId = $Subscription[0].Subscription.Id
$OfficeAPIUrl = $ARMResource + 'subscriptions/' + $SubscriptionId + '/resourceGroups/' +
$ResourceGroupName + '/providers/Microsoft.OperationalInsights/workspaces/' + $WorkspaceName +
'/datasources/office365datasources_' + $SubscriptionId + $OfficeTennantId + '?api-version=2015-11-01-
preview'
$Officeparams = @{
ContentType = 'application/json'
Headers = @{
'Authorization'="$($authHeader)"
'x-ms-client-tenant-id'=$xms_client_tenant_Id
'Content-Type' = 'application/json'
}
Method = 'Delete'
URI = $OfficeAPIUrl
}
#GetDetails
RESTAPI-Auth -ErrorAction Stop
Office-UnSubscribe-Call -ErrorAction Stop
You will be prompted for credentials. Provide the credentials for your Log Analytics workspace.
Data collection
It may take a few hours for data to initially be collected. Once it starts collecting, Office 365 sends a webhook
notification with detailed data to Azure Monitor each time a record is created. This record is available in Azure
Monitor within a few minutes after being received.
Click on the Office 365 tile to open the Office 365 dashboard.
The dashboard includes the columns in the following table. Each column lists the top ten alerts by count matching
that column's criteria for the specified scope and time range. You can run a log search that provides the entire list
by clicking See all at the bottom of the column or by clicking the column header.
COLUMN DESCRIPTION
Operations Provides information about the active users from your all
monitored Office 365 subscriptions. You will also be able to
see the number of activities that happen over time.
Azure Active Directory Includes top user activities, such as Reset User Password and
Login Attempts. When you drill down, you will be able to see
the details of these activities like the Result Status. This is
mostly helpful if you want to monitor suspicious activities on
your Azure Active Directory.
PROPERTY DESCRIPTION
Type OfficeActivity
ClientIP The IP address of the device that was used when the activity
was logged. The IP address is displayed in either an IPv4 or
IPv6 address format.
AzureActiveDirectory
Exchange
SharePoint
OrganizationId The GUID for your organization's Office 365 tenant. This value
will always be the same for your organization, regardless of
the Office 365 service in which it occurs.
UserId The UPN (User Principal Name) of the user who performed the
action that resulted in the record being logged; for example,
my_name@my_domain_name. Note that records for activity
performed by system accounts (such as SHAREPOINT\system
or NTAUTHORITY\SYSTEM) are also included.
Admin
Application
DcAdmin
Regular
Reserved
ServicePrincipal
System
PROPERTY DESCRIPTION
OfficeWorkload AzureActiveDirectory
RecordType AzureActiveDirectory
PROPERTY DESCRIPTION
OfficeWorkload AzureActiveDirectory
RecordType AzureActiveDirectoryAccountLogon
Application The application that triggers the account login event, such as
Office 15.
PROPERTY DESCRIPTION
Client Details about the client device, device OS, and device browser
that was used for the of the account login event.
PROPERTY DESCRIPTION
OfficeWorkload AzureActiveDirectory
RecordType AzureActiveDirectory
AADTarget The user that the action (identified by the Operation property)
was performed on.
ActorContextId The GUID of the organization that the actor belongs to.
InterSystemsId The GUID that track the actions across components within the
Office 365 service.
TargetContextId The GUID of the organization that the targeted user belongs
to.
PROPERTY DESCRIPTION
Exchange Admin
These records are created when changes are made to Exchange configuration.
PROPERTY DESCRIPTION
OfficeWorkload Exchange
RecordType ExchangeAdmin
ModifiedObjectResolvedName This is the user friendly name of the object that was modified
by the cmdlet. This is logged only if the cmdlet modifies the
object.
OriginatingServer The name of the server from which the cmdlet was executed.
Parameters The name and value for all parameters that were used with
the cmdlet that is identified in the Operations property.
Exchange Mailbox
These records are created when changes or additions are made to Exchange mailboxes.
PROPERTY DESCRIPTION
OfficeWorkload Exchange
RecordType ExchangeItem
ClientInfoString Information about the email client that was used to perform
the operation, such as a browser version, Outlook version,
and mobile device information.
PROPERTY DESCRIPTION
Client_IPAddress The IP address of the device that was used when the
operation was logged. The IP address is displayed in either an
IPv4 or IPv6 address format.
ClientProcessName The email client that was used to access the mailbox.
Logon_Type Indicates the type of user who accessed the mailbox and
performed the operation that was logged.
MailboxOwnerUPN The email address of the person who owns the mailbox that
was accessed.
PROPERTY DESCRIPTION
OfficeWorkload Exchange
RecordType ExchangeItem
Item Represents the item upon which the operation was performed
SendAsUserMailboxGuid The Exchange GUID of the mailbox that was accessed to send
email as.
SendonBehalfOfUserMailboxGuid The Exchange GUID of the mailbox that was accessed to send
mail on behalf of.
SendOnBehalfOfUserSmtp SMTP address of the user on whose behalf the email is sent.
PROPERTY DESCRIPTION
OfficeWorkload Exchange
OfficeWorkload ExchangeItemGroup
SharePoint Base
These properties are common to all SharePoint records.
PROPERTY DESCRIPTION
OfficeWorkload SharePoint
OfficeWorkload SharePoint
ItemType The type of object that was accessed or modified. See the
ItemType table for details on the types of objects.
Site_ The GUID of the site where the file or folder accessed by the
user is located.
SharePoint Schema
These records are created when configuration changes are made to SharePoint.
PROPERTY DESCRIPTION
OfficeWorkload SharePoint
OfficeWorkload SharePoint
PROPERTY DESCRIPTION
OfficeWorkload SharePoint
OfficeWorkload SharePointFileOperation
DestinationFileName The name of the file that is copied or moved. This property is
displayed only for FileCopied and FileMoved events.
Site_Url The URL of the site where the file or folder accessed by the
user is located.
SourceFileExtension The file extension of the file that was accessed by the user. This
property is blank if the object that was accessed is a folder.
SourceRelativeUrl The URL of the folder that contains the file accessed by the
user. The combination of the values for the SiteURL,
SourceRelativeURL, and SourceFileName parameters is the
same as the value for the ObjectID property, which is the full
path name for the file accessed by the user.
QUERY DESCRIPTION
Count of all the operations on your Office 365 subscription OfficeActivity | summarize count() by Operation
Next steps
Use log queries in Azure Monitor to view detailed update data.
Create your own dashboards to display your favorite Office 365 search queries.
Create alerts to be proactively notified of important Office 365 activities.
Monitor Surface Hubs with Azure Monitor to track
their health
12/13/2019 • 3 minutes to read • Edit Online
This article describes how you can use the Surface Hub solution in Azure Monitor to monitor Microsoft Surface
Hub devices. The solution helps you track the health of your Surface Hubs as well as understand how they are
being used.
Each Surface Hub has the Microsoft Monitoring Agent installed. Its through the agent that you can send data from
your Surface Hub to a Log Analytics workspace in Azure Monitor. Log files are read from your Surface Hubs and
are then are sent to Azure Monitor. Issues like servers being offline, the calendar not syncing, or if the device
account is unable to log into Skype are shown in the Surface Hub dashboard in Azure Monitor. By using the data
in the dashboard, you can identify devices that are not running, or that are having other problems, and potentially
apply fixes for the detected issues.
Set up monitoring
You can monitor the health and activity of your Surface Hub using Azure Monitor. You can enroll the Surface Hub
by using Intune, or locally by using Settings on the Surface Hub.
Intune then syncs the Log Analytics settings with the devices in the target group, enrolling them in your Log
Analytics workspace.
You can create alerts based on existing or custom log searches. Using the data Azure Monitor collects from your
Surface Hubs, you can search for issues and alert on the conditions that you define for your devices.
Next steps
Use Log queries in Azure Monitor to view detailed Surface Hub data.
Create alerts to notify you when issues occur with your Surface Hubs.
Custom metrics in Azure Monitor
11/6/2019 • 9 minutes to read • Edit Online
As you deploy resources and applications in Azure, you'll want to start collecting telemetry to gain insights into
their performance and health. Azure makes some metrics available to you out of the box. These metrics are called
standard or platform. However, they're limited in nature. You might want to collect some custom performance
indicators or business-specific metrics to provide deeper insights. These custom metrics can be collected via
your application telemetry, an agent that runs on your Azure resources, or even an outside-in monitoring system
and submitted directly to Azure Monitor. After they're published to Azure Monitor, you can browse, query, and
alert on custom metrics for your Azure resources and applications side by side with the standard metrics emitted
by Azure.
When you send custom metrics to Azure Monitor, each data point, or value, reported must include the following
information.
Authentication
To submit custom metrics to Azure Monitor, the entity that submits the metric needs a valid Azure Active
Directory (Azure AD ) token in the Bearer header of the request. There are a few supported ways to acquire a
valid bearer token:
1. Managed identities for Azure resources. Gives an identity to an Azure resource itself, such as a VM. Managed
Service Identity (MSI) is designed to give resources permissions to carry out certain operations. An example
is allowing a resource to emit metrics about itself. A resource, or its MSI, can be granted Monitoring Metrics
Publisher permissions on another resource. With this permission, the MSI can emit metrics for other
resources as well.
2. Azure AD Service Principal. In this scenario, an Azure AD application, or service, can be assigned permissions
to emit metrics about an Azure resource. To authenticate the request, Azure Monitor validates the application
token by using Azure AD public keys. The existing Monitoring Metrics Publisher role already has this
permission. It's available in the Azure portal. The service principal, depending on what resources it emits
custom metrics for, can be given the Monitoring Metrics Publisher role at the scope required. Examples are
a subscription, resource group, or specific resource.
NOTE
When you request an Azure AD token to emit custom metrics, ensure that the audience or resource the token is requested
for is https://monitoring.azure.com/. Be sure to include the trailing '/'.
Subject
This property captures which Azure resource ID the custom metric is reported for. This information will be
encoded in the URL of the API call being made. Each API can only submit metric values for a single Azure
resource.
NOTE
You can't emit custom metrics against the resource ID of a resource group or subscription.
Region
This property captures what Azure region the resource you're emitting metrics for is deployed in. Metrics must
be emitted to the same Azure Monitor regional endpoint as the region the resource is deployed in. For example,
custom metrics for a VM deployed in West US must be sent to the WestUS regional Azure Monitor endpoint.
The region information is also encoded in the URL of the API call.
NOTE
During the public preview, custom metrics are only available in a subset of Azure regions. A list of supported regions is
documented in a later section of this article.
Timestamp
Each data point sent to Azure Monitor must be marked with a timestamp. This timestamp captures the DateTime
at which the metric value is measured or collected. Azure Monitor accepts metric data with timestamps as far as
20 minutes in the past and 5 minutes in the future. The timestamp must be in ISO 8601 format.
Namespace
Namespaces are a way to categorize or group similar metrics together. By using namespaces, you can achieve
isolation between groups of metrics that might collect different insights or performance indicators. For example,
you might have a namespace called contosomemorymetrics that tracks memory-use metrics which profile
your app. Another namespace called contosoapptransaction might track all metrics about user transactions in
your application.
Name
Name is the name of the metric that's being reported. Usually, the name is descriptive enough to help identify
what's measured. An example is a metric that measures the number of memory bytes used on a given VM. It
might have a metric name like Memory Bytes In Use.
Dimension keys
A dimension is a key or value pair that helps describe additional characteristics about the metric being collected.
By using the additional characteristics, you can collect more information about the metric, which allows for
deeper insights. For example, the Memory Bytes In Use metric might have a dimension key called Process that
captures how many bytes of memory each process on a VM consumes. By using this key, you can filter the
metric to see how much memory specific processes use or to identify the top five processes by memory usage.
Dimensions are optional, not all metrics may have dimensions. A custom metric can have up to 10 dimensions.
Dimension values
When reporting a metric data point, for each dimension key on the metric being reported, there's a
corresponding dimension value. For example, you might want to report the memory used by the ContosoApp on
your VM:
The metric name would be Memory Bytes in Use.
The dimension key would be Process.
The dimension value would be ContosoApp.exe.
When publishing a metric value, you can only specify a single dimension value per dimension key. If you collect
the same memory utilization for multiple processes on the VM, you can report multiple metric values for that
timestamp. Each metric value would specify a different dimension value for the Process dimension key.
Dimensions are optional, not all metrics may have dimensions. If a metric post defines dimension keys,
corresponding dimension values are mandatory.
Metric values
Azure Monitor stores all metrics at one-minute granularity intervals. We understand that during a given minute,
a metric might need to be sampled several times. An example is CPU utilization. Or it might need to be
measured for many discrete events. An example is sign-in transaction latencies. To limit the number of raw
values you have to emit and pay for in Azure Monitor, you can locally pre-aggregate and emit the values:
Min: The minimum observed value from all the samples and measurements during the minute.
Max: The maximum observed value from all the samples and measurements during the minute.
Sum: The summation of all the observed values from all the samples and measurements during the minute.
Count: The number of samples and measurements taken during the minute.
For example, if there were 4 sign-in transactions to your app during a given a minute, the resulting measured
latencies for each might be as follows:
7 ms 4 ms 13 ms 16 ms
"baseData": {
NOTE
Application Insights, the diagnostics extension, and the InfluxData Telegraf agent are already configured to emit metric
values against the correct regional endpoint and carry all of the preceding properties in each emission.
NOTE
Azure Monitor doesn’t yet support defining Units for a custom metric.
Supported regions
During the public preview, the ability to publish custom metrics is available only in a subset of Azure regions.
This restriction means that metrics can be published only for resources in one of the supported regions. The
following table lists the set of supported Azure regions for custom metrics. It also lists the corresponding
endpoints that metrics for resources in those regions should be published to:
US and Canada
West US 2 https://westus2.monitoring.azure.com/
Central US https://centralus.monitoring.azure.com
East US https://eastus.monitoring.azure.com/
Europe
UK South https://uksouth.monitoring.azure.com
Africa
Asia
CATEGORY LIMIT
An active time series is defined as any unique combination of metric, dimension key, or dimension value that has
had metric values published in the past 12 hours.
Next steps
Use custom metrics from different services:
Virtual Machines
Virtual machine scale set
Azure Virtual Machines (classic)
Linux Virtual Machine using the Telegraf agent
REST API
Classic Cloud Services
Send custom metrics for an Azure resource to the
Azure Monitor metric store by using a REST API
11/21/2019 • 2 minutes to read • Edit Online
This article shows you how to send custom metrics for Azure resources to the Azure Monitor metrics store via a
REST API. After the metrics are in Azure Monitor, you can do all the things with them that you do with standard
metrics. Examples are charting, alerting, and routing them to other external tools.
NOTE
The REST API only permits sending custom metrics for Azure resources. To send metrics for resources in different
environments or on-premises, you can use Application Insights.
{
"time": "2018-09-13T16:34:20",
"data": {
"baseData": {
"metric": "QueueDepth",
"namespace": "QueueProcessing",
"dimNames": [
"QueueName",
"MessageType"
],
"series": [
{
"dimValues": [
"ImagesToProcess",
"JPEG"
],
"min": 3,
"max": 20,
"sum": 28,
"count": 3
}
]
}
}
}
Troubleshooting
If you receive an error message with some part of the process, consider the following troubleshooting
information:
1. You can't issue metrics against a subscription or resource group as your Azure resource.
2. You can't put a metric into the store that's over 20 minutes old. The metric store is optimized for alerting and
real-time charting.
3. The number of dimension names should match the values and vice versa. Check the values.
4. You might be emitting metrics against a region that doesn’t support custom metrics. See supported regions.
Next steps
Learn more about custom metrics.
Send log data to Azure Monitor with the HTTP
Data Collector API (public preview)
10/25/2019 • 15 minutes to read • Edit Online
This article shows you how to use the HTTP Data Collector API to send log data to Azure Monitor from a
REST API client. It describes how to format data collected by your script or application, include it in a request,
and have that request authorized by Azure Monitor. Examples are provided for PowerShell, C#, and Python.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in
a Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the
terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
NOTE
The Azure Monitor HTTP Data Collector API is in public preview.
Concepts
You can use the HTTP Data Collector API to send log data to a Log Analytics workspace in Azure Monitor
from any client that can call a REST API. This might be a runbook in Azure Automation that collects
management data from Azure or another cloud, or it might be an alternate management system that uses
Azure Monitor to consolidate and analyze log data.
All data in the Log Analytics workspace is stored as a record with a particular record type. You format your
data to send to the HTTP Data Collector API as multiple records in JSON. When you submit the data, an
individual record is created in the repository for each record in the request payload.
Create a request
To use the HTTP Data Collector API, you create a POST request that includes the data to send in JavaScript
Object Notation (JSON ). The next three tables list the attributes that are required for each request. We
describe each attribute in more detail later in the article.
Request URI
ATTRIBUTE PROPERTY
Method POST
URI https://<CustomerId>.ods.opinsights.azure.com/api/logs?
api-version=2016-04-01
API Version The version of the API to use with this request. Currently,
it's 2016-04-01.
Request headers
HEADER DESCRIPTION
Log-Type Specify the record type of the data that is being submitted.
Can only contain letters, numbers, and underscore (_), and
may not exceed 100 characters.
x-ms-date The date that the request was processed, in RFC 1123
format.
Authorization
Any request to the Azure Monitor HTTP Data Collector API must include an authorization header. To
authenticate a request, you must sign the request with either the primary or the secondary key for the
workspace that is making the request. Then, pass that signature as part of the request.
Here's the format for the authorization header:
Authorization: SharedKey <WorkspaceID>:<Signature>
WorkspaceID is the unique identifier for the Log Analytics workspace. Signature is a Hash-based Message
Authentication Code (HMAC ) that is constructed from the request and then computed by using the SHA256
algorithm. Then, you encode it by using Base64 encoding.
Use this format to encode the SharedKey signature string:
When you have the signature string, encode it by using the HMAC -SHA256 algorithm on the UTF -8-
encoded string, and then encode the result as Base64. Use this format:
Signature=Base64(HMAC-SHA256(UTF8(StringToSign)))
The samples in the next sections have sample code to help you create an authorization header.
Request body
The body of the message must be in JSON. It must include one or more records with the property name and
value pairs in the following format. The property name can only contain letters, numbers, and underscore (_).
[
{
"property 1": "value1",
"property 2": "value2",
"property 3": "value3",
"property 4": "value4"
}
]
You can batch multiple records together in a single request by using the following format. All the records
must be the same record type.
[
{
"property 1": "value1",
"property 2": "value2",
"property 3": "value3",
"property 4": "value4"
},
{
"property 1": "value1",
"property 2": "value2",
"property 3": "value3",
"property 4": "value4"
}
]
String _s
Boolean _b
Double _d
Date/time _t
The data type that Azure Monitor uses for each property depends on whether the record type for the new
record already exists.
If the record type does not exist, Azure Monitor creates a new one using the JSON type inference to
determine the data type for each property for the new record.
If the record type does exist, Azure Monitor attempts to create a new record based on existing properties.
If the data type for a property in the new record doesn’t match and can’t be converted to the existing type,
or if the record includes a property that doesn’t exist, Azure Monitor creates a new property that has the
relevant suffix.
For example, this submission entry would create a record with three properties, number_d, boolean_b, and
string_s:
If you then submitted this next entry, with all values formatted as strings, the properties would not change.
These values can be converted to existing data types:
But, if you then made this next submission, Azure Monitor would create the new properties boolean_d and
string_d. These values can't be converted:
If you then submitted the following entry, before the record type was created, Azure Monitor would create a
record with three properties, number_s, boolean_s, and string_s. In this entry, each of the initial values is
formatted as a string:
Reserved properties
The following properties are reserved and should not be used in a custom record type. You will receive an
error if your payload includes any of these property names.
tenant
Data limits
There are some constraints around the data posted to the Azure Monitor Data collection API.
Maximum of 30 MB per post to Azure Monitor Data Collector API. This is a size limit for a single post. If
the data from a single post that exceeds 30 MB, you should split the data up to smaller sized chunks and
send them concurrently.
Maximum of 32 KB limit for field values. If the field value is greater than 32 KB, the data will be truncated.
Recommended maximum number of fields for a given type is 50. This is a practical limit from a usability
and search experience perspective.
A table in a Log Analytics workspace only supports up to 500 columns (referred to as a field in this
article).
The maximum number of characters for the column name is 500.
Return codes
The HTTP status code 200 means that the request has been received for processing. This indicates that the
operation completed successfully.
This table lists the complete set of status codes that the service might return:
Query data
To query data submitted by the Azure Monitor HTTP Data Collector API, search for records with Type that is
equal to the LogType value that you specified, appended with _CL. For example, if you used
MyCustomLog, then you'd return all records with MyCustomLog_CL .
Sample requests
In the next sections, you'll find samples of how to submit data to the Azure Monitor HTTP Data Collector API
by using different programming languages.
For each sample, do these steps to set the variables for the authorization header:
1. In the Azure portal, locate your Log Analytics workspace.
2. Select Advanced Settings and then Connected Sources.
3. To the right of Workspace ID, select the copy icon, and then paste the ID as the value of the Customer
ID variable.
4. To the right of Primary Key, select the copy icon, and then paste the ID as the value of the Shared Key
variable.
Alternatively, you can change the variables for the log type and JSON data.
PowerShell sample
# You can use an optional field to specify the timestamp from the data. If the time field is not
specified, Azure Monitor assumes the time is the message ingestion time
$TimeStampField = "DateValue"
$bytesToHash = [Text.Encoding]::UTF8.GetBytes($stringToHash)
$keyBytes = [Convert]::FromBase64String($sharedKey)
$headers = @{
"Authorization" = $signature;
"Log-Type" = $logType;
"x-ms-date" = $rfc1123date;
"time-generated-field" = $TimeStampField;
}
$response = Invoke-WebRequest -Uri $uri -Method $method -ContentType $contentType -Headers $headers -
Body $body -UseBasicParsing
return $response.StatusCode
C# sample
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
namespace OIAPIExample
{
class ApiExample
{
// An example JSON object, with key/value pairs
static string json = @"[{""DemoField1"":""DemoValue1"",""DemoField2"":""DemoValue2""},
{""DemoField3"":""DemoValue3"",""DemoField4"":""DemoValue4""}]";
// For sharedKey, use either the primary or the secondary Connected Sources client authentication key
static string sharedKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";
// LogName is name of the event type that is being submitted to Azure Monitor
static string LogName = "DemoExample";
// You can use an optional field to specify the timestamp from the data. If the time field is not
specified, Azure Monitor assumes the time is the message ingestion time
static string TimeStampField = "";
Python 2 sample
import json
import requests
import datetime
import hashlib
import hmac
import base64
# For the shared key, use either the primary or the secondary Connected Sources client authentication key
shared_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
# The log type is the name of the event that is being submitted
log_type = 'WebMonitorTest'
#####################
######Functions######
#####################
headers = {
'content-type': content_type,
'Authorization': signature,
'Log-Type': log_type,
'x-ms-date': rfc1123date
}
Custom events: Native SDK-based Application Insights, typically Data that is generated within
ingestion in Application Insights instrumented through an SDK within your application, but not
your application, offers the ability for picked up by SDK through
you to send custom data through one of the default data types
Custom Events. (requests, dependencies,
exceptions, and so on).
Data that is most often
correlated to other application
data in Application Insights
ALTERNATIVE DESCRIPTION BEST SUITED FOR
Data Collector API in Azure Monitor The Data Collector API in Azure Data that is not necessarily
Logs Monitor Logs is a completely open- generated within an
ended way to ingest data. Any data application instrumented
formatted in a JSON object can be within Application Insights.
sent here. Once sent, it will be Examples include lookup and
processed, and available in Logs to be fact tables, reference data,
correlated with other data in Logs or pre-aggregated statistics, and
against other Application Insights so on.
data. Intended for data that will be
cross-referenced against other
It is fairly easy to upload the data as Azure Monitor data
files to an Azure Blob blob, from (Application Insights, other
where these files will be processed Logs data types, Security
and uploaded to Log Analytics. Please Center, Azure Monitor for
see this article for a sample Containers/VMs, and so on).
implementation of such a pipeline.
Azure Data Explorer Azure Data Explorer (ADX) is the data Data that will not be
platform that powers Application correlated to any other data
Insights Analytics and Azure Monitor under Application Insights or
Logs. Now Generally Available ("GA"), Logs.
using the data platform in its raw Data requiring advanced
form provides you complete flexibility ingestion or processing
(but requiring the overhead of capabilities not today available
management) over the cluster (RBAC, in Azure Monitor Logs.
retention rate, schema, and so on).
ADX provides many ingestion options
including CSV, TSV, and JSON files.
Next steps
Use the Log Search API to retrieve data from the Log Analytics workspace.
Learn more about how create a data pipeline with the Data Collector API using Logic Apps workflow
to Azure Monitor.
Create a data pipeline with the Data Collector API
12/13/2019 • 6 minutes to read • Edit Online
The Azure Monitor Data Collector API allows you to import any custom log data into a Log Analytics workspace in
Azure Monitor. The only requirements are that the data be JSON -formatted and split into 30 MB or less segments.
This is a completely flexible mechanism that can be plugged into in many ways: from data being sent directly from
your application, to one-off adhoc uploads. This article will outline some starting points for a common scenario:
the need to upload data stored in files on a regular, automated basis. While the pipeline presented here will not be
the most performant or otherwise optimized, it is intended to serve as a starting point towards building a
production pipeline of your own.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log
Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology to
better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Example problem
For the remainder of this article, we will examine page view data in Application Insights. In our hypothetical
scenario, we want to correlate geographical information collected by default by the Application Insights SDK to
custom data containing the population of every country/region in the world, with the goal of identifying where we
should be spending the most marketing dollars.
We use a public data source such as the UN World Population Prospects for this purpose. The data will have the
following simple schema:
In our example, we assume that we will upload a new file with the latest year’s data as soon as it becomes
available.
General design
We are using a classic ETL -type logic to design our pipeline. The architecture will look as follows:
This article will not cover how to create data or upload it to an Azure Blob Storage account. Rather, we pick the
flow up as soon as a new file is uploaded to the blob. From here:
1. A process will detect that new data has been uploaded. Our example uses an Azure Logic App, which has
available a trigger to detect new data being uploaded to a blob.
2. A processor reads this new data and converts it to JSON, the format required by Azure Monitor In this
example, we use an Azure Function as a lightweight, cost-efficient way of executing our processing code.
The function is kicked off by the same Logic App that we used to detect a the new data.
3. Finally, once the JSON object is available, it is sent to Azure Monitor. The same Logic App sends the data to
Azure Monitor using the built in Log Analytics Data Collector activity.
While the detailed setup of the blob storage, Logic App, or Azure Function is not outlined in this article, detailed
instructions are available on the specific products’ pages.
To monitor this pipeline, we use Application Insights to monitor our Azure Function details here, and Azure
Monitor to monitor our Logic App details here.
3. Switch to run.csx from the right pane, and replace the default code with the following.
NOTE
For your project, you have to replace the record model (the “PopulationRecord” class) with your own data schema.
using System.Net;
using Newtonsoft.Json;
using CsvHelper;
class PopulationRecord
{
public String Location { get; set; }
public int Time { get; set; }
public long Population { get; set; }
}
NOTE
It can take up to 30 minutes for the data to appear in Azure Monitor the first time you send a new data type.
The output should show the two data sources now joined.
Where do I start
Azure Monitor metrics explorer is a component of the Microsoft Azure portal that allows plotting charts,
visually correlating trends, and investigating spikes and dips in metrics' values. Use the metrics explorer to
investigate the health and utilization of your resources. Start in the following order:
1. Pick a resource and a metric and you see a basic chart. Then select a time range that is relevant for your
investigation.
2. Try applying dimension filters and splitting. The filters and splitting allow you to analyze which
segments of the metric contribute to the overall metric value and identify possible outliers.
3. Use advanced settings to customize the chart before pinning to dashboards. Configure alerts to receive
notifications when the metric value exceeds or drops below a threshold.
2. For some resources, you must pick a namespace. The namespace is just a way to organize metrics so
that you can easily find them. For example, storage accounts have separate namespaces for storing
Files, Tables, Blobs, and Queues metrics. Many resource types only have one namespace.
3. Select a metric from a list of available metrics.
4. Optionally, you can change the metric aggregation. For example, you might want your chart to show
minimum, maximum, or average values of the metric.
NOTE
Use the Add metric button and repeat these steps if you want to see multiple metrics plotted in the same chart. For
multiple charts in one view, select the Add chart button on top.
NOTE
Use the time brush to investigate an interesting area of the chart (spike or a dip). Put the mouse pointer at the
beginning of the area, click and hold the left mouse button, drag to the other side of area and then release the button.
The chart will zoom in on that time range.
Next steps
Learn about advanced features of Metrics Explorer
Troubleshooting Metrics Explorer
See a list of available metrics for Azure services
See examples of configured charts
Advanced features of Azure Metrics Explorer
4/5/2019 • 5 minutes to read • Edit Online
NOTE
This article assumes that you are familiar with basic features of Metrics Explorer. If you are a new user and want to learn
how to create your first metric chart, see Getting started with Azure Metrics Explorer.
Metrics in Azure
Metrics in Azure Monitor are the series of measured values and counts that are collected and stored over time.
There are standard (or “platform”) metrics, and custom metrics. The standard metrics are provided to you by
the Azure platform itself. Standard metrics reflect the health and usage statistics of your Azure resources.
Whereas custom metrics are sent to Azure by your applications using the Application Insights API for custom
events and metrics, Windows Azure Diagnostics (WAD ) extension, or by Azure Monitor REST API.
NOTE
You typically don’t want to have metrics with different units of measure (i.e. “milliseconds” and “kilobytes”) or with
significantly different scale on one chart. Instead, consider using multiple charts. Click on the Add Chart button to create
multiple charts in metrics explorer.
Multiple charts
Click the Add chart and create another chart with a different metric.
Order or delete multiple charts
To order or delete multiple charts, click on the ellipses ( ... ) symbol to open the chart menu and choose the
appropriate menu item of Move up, Move down, or Delete.
3. Select which dimension values you want to include when plotting the chart (this example shows filtering
out the successful storage transactions):
4. After selecting the filter values, click away from the Filter Selector to close it. Now the chart shows how
many storage transactions have failed:
5. You can repeat steps 1-4 to apply multiple filters to the same charts.
NOTE
Splitting cannot be used with charts that have multiple metrics. Also, you can have multiple filters but only one
splitting dimension applied to any single chart.
Now the chart now shows multiple lines, one for each segment of dimension:
3. Click away from the Grouping Selector to close it.
NOTE
Use both Filtering and Splitting on the same dimension to hide the segments that are irrelevant for your scenario
and make charts easier to read.
You will be taken to the alert rule creation pane with the underlying metric dimensions from your chart pre-
populated to make it easier to generate custom alert rules.
Check out this article to learn more about setting up metric alerts.
Troubleshooting
I don't see any data on my chart.
Filters apply to all the charts on the pane. Make sure that, while you're focusing on one chart, you didn't
set a filter that excludes all the data on another.
If you want to set different filters on different charts, create them in different blades, save them as
separate favorites. If you want, you can pin them to the dashboard so that you can see them alongside
each other.
If you segment a chart by a property that is not defined on the metric, then there will be nothing on the
chart. Try clearing the segmentation (splitting), or choose a different property.
Next steps
Read Creating custom KPI dashboards to learn about the best practices for creating actionable dashboards with
metrics.
Metric chart examples
11/26/2019 • 2 minutes to read • Edit Online
The Azure platform offers over a thousand metrics, many of which have dimensions. By using dimension filters,
applying splitting, controlling chart type, and adjusting chart settings you can create powerful diagnostic views
and dashboards that provide insight into the health of your infrastructure and applications. This article shows
some examples of the charts that you can build using Metrics Explorer and explains the necessary steps to
configure each of these charts.
Want to share your great charts examples with the world? Contribute to this page on GitHub and share your own
chart examples here!
Use this article if you run into issues with creating, customizing, or interpreting charts in Azure metrics explorer. If
you are new to metrics, learn about getting started with metrics explorer and advanced features of metrics
explorer. You can also see examples of the configured metric charts.
WARNING
For best performance, when you first open metrics explorer, the Resource group dropdown has no pre-selected
resource groups. You must pick at least one group before you can see any resources.
WARNING
You cannot use Log Analytics agent (also referred to as the Microsoft Monitoring Agent, or "MMA") to send Guest
OS into a storage account.
Next steps
Learn about getting started with Metric Explorer
Learn about advanced features of Metric Explorer
See a list of available metrics for Azure services
See examples of configured charts
Overview of log queries in Azure
Monitor
10/24/2019 • 5 minutes to read • Edit Online
Log queries help you to fully leverage the value of the data collected in Azure Monitor
Logs. A powerful query language allows you to join data from multiple tables, aggregate
large sets of data, and perform complex operations with minimal code. Virtually any
question can be answered and analysis performed as long as the supporting data has
been collected, and you understand how to construct the right query.
Some features in Azure Monitor such as insights and solutions process log data without
exposing you to the underlying queries. To fully leverage other features of Azure Monitor,
you should understand how queries are constructed and how you can use them to
interactively analyze data in Azure Monitor Logs.
Use this article as a starting point to learning about log queries in Azure Monitor. It
answers common questions and provides links to other documentation that provides
further details and lessons.
Syslog
Or it could filter for particular records, summarize them, and visualize the results in a
chart:
SecurityEvent
| where TimeGenerated > ago(7d)
| where EventID == 4625
| summarize count() by Computer, bin(TimeGenerated, 1h)
| render timechart
For more complex analysis, you might retrieve data from multiple tables using a join to
analyze the results together.
app("ContosoRetailWeb").requests
| summarize count() by bin(timestamp,1hr)
| join kind= inner (Perf
| summarize avg(CounterValue)
by bin(TimeGenerated,1hr))
on $left.timestamp == $right.TimeGenerated
Even if you aren't familiar with KQL, you should be able to at least figure out the basic
logic being used by these queries. They start with the name of a table and then add
multiple commands to filter and process that data. A query can use any number of
commands, and you can write more complex queries as you become familiar with the
different KQL commands available.
See Get started with log queries in Azure Monitor for a tutorial on log queries that
introduces the language and common functions, .
Next steps
Walk through a tutorial on using Log Analytics in the Azure portal.
Walk through a tutorial on writing queries.
Simple Logs experience in Azure Monitor (Preview)
10/25/2019 • 2 minutes to read • Edit Online
Azure Monitor provides a rich experience for creating log queries using the KQL language. You may not require the
full power of KQL though and prefer a simplified experience for basic query requirements. The Simple Logs
experience allows you to create basic queries without directly interacting with KQL. You can also use Simple Logs
as a learning tool for KQL as you require more sophisticated queries.
NOTE
Simple Logs is currently implemented as a test for only Cosmos DB and Key Vaults. Please share your experience with
Microsoft through User Voice to help us determine whether we will expand and release this feature.
Scope
The Simple Logs experience retrieves data from the AzureDiagnostics, AzureMetrics, and AzureActivity table for
the selected resource.
Select a Field and an Operator and specify a Value for comparison. Click + and specify And/Or to add additional
criteria.
Next steps
Complete a tutorial on using Log Analytics in the Azure portal.
Complete a tutorial on writing log queries.
Get started with Log Analytics in Azure Monitor
10/25/2019 • 7 minutes to read • Edit Online
NOTE
You can work through this exercise in your own environment if you are collecting data from at least one virtual
machine. If not then use our Demo environment, which includes plenty of sample data.
In this tutorial you will learn how to use Log Analytics in the Azure portal to write Azure Monitor log
queries. It will teach you how to:
Use Log Analytics to write a simple query
Understand the schema of your data
Filter, sort, and group results
Apply a time range
Create charts
Save and load queries
Export and share queries
For a tutorial on writing log queries, see Get started with log queries in Azure Monitor.
For more details on log queries, see Overview of log queries in Azure Monitor.
Firewall requirements
To use Log Analytics, your browser requires access to the following addresses. If your browser is accessing
the Azure portal through a firewall, you must enable access to these addresses.
URI IP PORTS
Basic queries
Queries can be used to search terms, identify trends, analyze patterns, and provide many other insights
based on your data. Start with a basic query:
This query searches the Event table for records that contain the term error in any property.
Queries can start with either a table name or a search command. The above example starts with the table
name Event, which retrieves all records from the Event table. The pipe (|) character separates commands, so
the output of the first one serves as the input of the following command. You can add any number of
commands to a single query.
Another way to write that same query would be:
In this example, search is scoped to the Event table, and all records in that table are searched for the term
error.
Running a query
Run a query by clicking the Run button or pressing Shift+Enter. Consider the following details which
determine the code that will be run and the data that's returned:
Line breaks: A single break makes your query easier to read. Multiple line breaks split it into separate
queries.
Cursor: Place your cursor somewhere inside the query to execute it. The current query is considered to
be the code up until a blank line is found.
Time range - A time range of last 24 hours is set by default. To use a different range, use the time-picker
or add an explicit time range filter to your query.
Event
When selecting a custom time range, the selected values are in UTC, which could be different than your
local time zone.
If the query explicitly contains a filter for TimeGenerated, the time picker title will show Set in query.
Manual selection will be disabled to prevent a conflict.
Charts
In addition to returning results in a table, query results can be presented in visual formats. Use the
following query as an example:
Event
| where EventLevelName == "Error"
| where TimeGenerated > ago(1d)
| summarize count() by Source
By default, results are displayed in a table. Click Chart to see the results in a graphic view:
The results are shown in a stacked bar chart. Click Stacked Column and select Pie to show another view of
the results:
Different properties of the view, such as x and y axes, or grouping and splitting preferences, can be changed
manually from the control bar.
You can also set the preferred view in the query itself, using the render operator.
Smart diagnostics
On a timechart, if there is a sudden spike or step in your data, you may see a highlighted point on the line.
This indicates that Smart Diagnostics has identified a combination of properties that filter out the sudden
change. Click the point to get more detail on the filter, and to see the filtered version. This may help you
identify what caused the change:
Pin to dashboard
To pin a diagram or table to one of your shared Azure dashboards, click the pin icon. Note that this icon has
moved to the top of the Log Analytics window, different from the screenshot below.
Certain simplifications are applied to a chart when you pin it to a dashboard:
Table columns and rows: In order to pin a table to the dashboard, it must have four or fewer columns.
Only the top seven rows are displayed.
Time restriction: Queries are automatically limited to the past 14 days.
Bin count restriction: If you display a chart that has a lot of discrete bins, less populated bins are
automatically grouped into a single others bin.
Save queries
Once you've created a useful query, you might want to save it or share with others. The Save icon is on the
top bar.
You can save either the entire query page, or a single query as a function. Functions are queries that can
also be referenced by other queries. In order to save a query as a function, you must provide a function
alias, which is the name used to call this query when referenced by other queries.
NOTE
The following characters are supported - a–z, A–Z, 0-9, -, _, ., <space>, (, ), | in the Name field when
saving or editing the saved query.
Log Analytics queries are always saved to a selected workspace, and shared with other users of that
workspace.
Load queries
The Query Explorer icon is at the top-right area. This lists all saved queries by category. It also enables you
to mark specific queries as Favorites to quickly find them in the future. Double-click a saved query to add it
to the current window.
Next steps
Learn more about writing Azure Monitor log queries.
Get started with log queries in Azure Monitor
12/13/2019 • 7 minutes to read • Edit Online
NOTE
You can work through this exercise in your own environment if you are collecting data from at least one virtual
machine. If not then use our Demo environment, which includes plenty of sample data.
In this tutorial you will learn to write log queries in Azure Monitor. It will teach you how to:
Understand query structure
Sort query results
Filter query results
Specify a time range
Select which fields to include in the results
Define and use custom fields
Aggregate and group results
For a tutorial on using Log Analytics in the Azure portal, see Get started with Azure Monitor Log Analytics.
For more details on log queries in Azure Monitor, see Overview of log queries in Azure Monitor.
Follow along with a video version of this tutorial below:
NOTE
The Kusto query language used by Azure Monitor is case-sensitive. Language keywords are typically written in lower-
case. When using names of tables or columns in a query, make sure to use the correct case, as shown on the schema
pane.
SecurityEvent
| take 10
The query shown above returns 10 results from the SecurityEvent table, in no specific order. This is a very
common way to take a glance at a table and understand its structure and content. Let's examine how it's built:
The query starts with the table name SecurityEvent - this part defines the scope of the query.
The pipe (|) character separates commands, so the output of the first one in the input of the following
command. You can add any number of piped elements.
Following the pipe is the take command, which returns a specific number of arbitrary records from the
table.
We could actually run the query even without adding | take 10 - that would still be valid, but it could return
up to 10,000 results.
Search queries
Search queries are less structured, and generally more suited for finding records that include a specific value
in any of their columns:
This query searches the SecurityEvent table for records that contain the phrase "Cryptographic". Of those
records, 10 records will be returned and displayed. If we omit the in (SecurityEvent) part and just run
search "Cryptographic" , the search will go over all tables, which would take longer and be less efficient.
WARNING
Search queries are typically slower than table-based queries because they have to process more data.
SecurityEvent
| sort by TimeGenerated desc
That could return too many results though and might also take some time. The above query sorts the entire
SecurityEvent table by the TimeGenerated column. The Analytics portal then limits the display to show only
10,000 records. This approach is of course not optimal.
The best way to get only the latest 10 records is to use top, which sorts the entire table on the server side and
then returns the top records:
SecurityEvent
| top 10 by TimeGenerated
Descending is the default sorting order, so we typically omit the desc argument.The output will look like this:
Where: filtering on a condition
Filters, as indicated by their name, filter the data by a specific condition. This is the most common way to limit
query results to relevant information.
To add a filter to a query, use the where operator followed by one or more conditions. For example, the
following query returns only SecurityEvent records where Level equals 8:
SecurityEvent
| where Level == 8
When writing filter conditions, you can use the following expressions:
SecurityEvent
| where Level == 8 and EventID == 4672
SecurityEvent
| where Level == 8
| where EventID == 4672
NOTE
Values can have different types, so you might need to cast them to perform comparison on the correct type. For
example, SecurityEvent Level column is of type String, so you must cast it to a numerical type such as int or long,
before you can use numerical operators on it: SecurityEvent | where toint(Level) >= 10
SecurityEvent
| where TimeGenerated > ago(30m)
| where toint(Level) >= 10
In the above time filter ago(30m) means "30 minutes ago" so this query only returns records from the last 30
minutes. Other units of time include days (2d), minutes (25m), and seconds (10s).
SecurityEvent
| top 10 by TimeGenerated
| project TimeGenerated, Computer, Activity
SecurityEvent
| top 10 by TimeGenerated
| project Computer, TimeGenerated, EventDetails=Activity, EventCode=substring(Activity, 0, 4)
extend keeps all original columns in the result set and defines additional ones. The following query uses
extend to add the EventCode column. Note that this column may not display at the end of the table results in
which case you would need to expand the details of a record to view it.
SecurityEvent
| top 10 by TimeGenerated
| extend EventCode=substring(Activity, 0, 4)
Perf
| where TimeGenerated > ago(1h)
| summarize count() by ObjectName
Sometimes it makes sense to define groups by multiple dimensions. Each unique combination of these values
defines a separate group:
Perf
| where TimeGenerated > ago(1h)
| summarize count() by ObjectName, CounterName
Another common use is to perform mathematical or statistical calculations on each group. For example, the
following calculates the average CounterValue for each computer:
Perf
| where TimeGenerated > ago(1h)
| summarize avg(CounterValue) by Computer
Unfortunately, the results of this query are meaningless since we mixed together different performance
counters. To make this more meaningful, we should calculate the average separately for each combination of
CounterName and Computer:
Perf
| where TimeGenerated > ago(1h)
| summarize avg(CounterValue) by Computer, CounterName
Perf
| where TimeGenerated > ago(7d)
| where Computer == "ContosoAzADDS2"
| where CounterName == "Available MBytes"
| summarize avg(CounterValue) by bin(TimeGenerated, 1h)
To make the output clearer, you select to display it as a time-chart, showing the available memory over time:
Next steps
Learn more about using string data in a log query with Work with strings in Azure Monitor log queries.
Learn more about aggregating data in a log query with Advanced aggregations in Azure Monitor log
queries.
Learn how to join data from multiple tables with Joins in Azure Monitor log queries.
Get documentation on the entire Kusto query language in the KQL language reference.
Structure of Azure Monitor Logs
12/6/2019 • 3 minutes to read • Edit Online
The ability to quickly gain insights into your data using a log query is a powerful feature of Azure Monitor. To
create efficient and useful queries, you should understand some basic concepts such as where the data you want is
located and how it's structured. This article provides the basic concepts you need to get started.
Overview
Data in Azure Monitor Logs is stored in either a Log Analytics workspace or an Application Insights application.
Both are powered by Azure Data Explorer meaning that they leverage its powerful data engine and query
language.
Data in both workspaces and applications is organized into tables, each of which stores different kinds of data and
has its own unique set of properties. Most data sources will write to their own tables in a Log Analytics workspace,
while Application Insights will write to a predefined set of tables in an Application Insights application. Log queries
are very flexible allowing you to easily combine data from multiple tables and even use a cross-resource query to
combine data from tables in multiple workspaces or to write queries that combine workspace and application data.
The following image shows examples of data sources that write to different tables that are used in sample queries.
See documentation for each data source for details of the tables they create. Examples include articles for agent
data sources, resource logs, and monitoring solutions.
Workspace permissions
See Designing an Azure Monitor Logs deployment to understand the access control strategy and
recommendations to provide access to data in a workspace. In addition to granting access to the workspace itself,
you can limit access to individual tables using Table Level RBAC.
TABLE DESCRIPTION
You can view the schema for each table in the Schema tab in Log Analytics for the application.
Standard properties
While each table in Azure Monitor Logs has its own schema, there are standard properties shared by all tables.
See Standard properties in Azure Monitor Logs for details of each.
Next steps
Learn about using Log Analytics to create and edit log searches.
Check out a tutorial on writing queries using the new query language.
Log query scope and time range in Azure Monitor
Log Analytics
12/13/2019 • 5 minutes to read • Edit Online
When you run a log query in Log Analytics in the Azure portal, the set of data evaluated by the query depends
on the scope and the time range that you select. This article describes the scope and time range and how you can
set each depending on your requirements. It also describes the behavior of different types of scopes.
Query scope
The query scope defines the records that are evaluated by the query. This will usually include all records in a
single Log Analytics workspace or Application Insights application. Log Analytics also allows you to set a scope
for a particular monitored Azure resource. This allows a resource owner to focus only on their data, even if that
resource writes to multiple workspaces.
The scope is always displayed at the top left of the Log Analytics window. An icon indicates whether the scope is
a Log Analytics workspace or an Application Insights application. No icon indicates another Azure resource.
The scope is determined by the method you use to start Log Analytics, and in some cases you can change the
scope by clicking on it. The following table lists the different types of scope used and different details for each.
Log Analytics workspace All records in the Log Select Logs from the Azure Can change scope to any
Analytics workspace. Monitor menu or the Log other resource type.
Analytics workspaces
menu.
Application Insights All records in the Select Analytics from Can only change scope to
application Application Insights Overview page of another Application Insights
application. Application Insights. application.
Resource group Records created by all Select Logs from the Cannot change scope.
resources in the resource resource group menu.
group. May include data
from multiple Log Analytics
workspaces.
Subscription Records created by all Select Logs from the Cannot change scope.
resources in the subscription menu.
subscription. May include
data from multiple Log
Analytics workspaces.
QUERY SCOPE RECORDS IN SCOPE HOW TO SELECT CHANGING SCOPE
Other Azure resources Records created by the Select Logs from the Can only change scope to
resource. May include data resource menu. same resource type.
from multiple Log Analytics OR
workspaces. Select Logs from the Azure
Monitor menu and then
select a new scope.
Query limits
You may have business requirements for an Azure resource to write data to multiple Log Analytics workspaces.
The workspace doesn't need to be in the same region as the resource, and a single workspace might gather data
from resources in a variety of regions.
Setting the scope to a resource or set of resources is a particularly powerful feature of Log Analytics since it
allows you to automatically consolidate distributed data in a single query. It can significantly affect performance
though if data needs to be retrieved from workspaces across multiple Azure regions.
Log Analytics helps protect against excessive overhead from queries that span workspaces in multiple regions by
issuing a warning or error when a certain number of regions are being used. Your query will receive a warning if
the scope includes workspaces in 5 or more regions. it will still run, but it may take excessive time to complete.
Your query will be blocked from running if the scope includes workspaces in 20 or more regions. In this case you
will be prompted to reduce the number of workspace regions and attempt to run the query again. The dropdown
will display all of the regions in the scope of the query, and you should reduce the number of regions before
attempting to run the query again.
Time range
The time range specifies the set of records that are evaluated for the query based on when the record was
created. This is defined by a standard property on every record in the workspace or application as specified in
the following table.
LOCATION PROPERTY
Set the time range by selecting it from the time picker at the top of the Log Analytics window. You can select a
predefined period or select Custom to specify a specific time range.
If you set a filter in the query that uses the standard time property as shown in the table above, the time picker
changes to Set in query, and the time picker is disabled. In this case, it's most efficient to put the filter at the top
of the query so that any subsequent processing only needs to work with the filtered records.
If you use the workspace or app command to retrieve data from another workspace or application, the time
picker may behave differently. If the scope is a Log Analytics workspace and you use app, or if the scope is an
Application Insights application and you use workspace, then Log Analytics may not understand that the
property used in the filter should determine the time filter.
In the following example, the scope is set to a Log Analytics workspace. The query uses workspace to retrieve
data from another Log Analytics workspace. The time picker changes to Set in query because it sees a filter that
uses the expected TimeGenerated property.
If the query uses app to retrieve data from an Application Insights application though, Log Analytics doesn't
recognize the timestamp property in the filter, and the time picker remains unchanged. In this case, both filters
are applied. In the example, only records created in the last 24 hours are included in the query even though it
specifies 7 days in the where clause.
Next steps
Walk through a tutorial on using Log Analytics in the Azure portal.
Walk through a tutorial on writing queries.
Standard properties in Azure Monitor Logs
1/20/2020 • 6 minutes to read • Edit Online
Data in Azure Monitor Logs is stored as a set of records in either a Log Analytics workspace or Application
Insights application, each with a particular data type that has a unique set of properties. Many data types will have
standard properties that are common across multiple types. This article describes these properties and provides
examples of how you can use them in queries.
NOTE
Some of the standard properties will not show in the schema view or intellisense in Log Analytics, and they won't show in
query results unless you explicitly specify the property in the output.
Event
| where EventLevelName == "Error"
| where TimeGenerated between(startofweek(ago(7days))..endofweek(ago(7days)))
| summarize count() by bin(TimeGenerated, 1day)
| sort by TimeGenerated asc
The following query returns the number of exceptions created for each day in the previous week.
exceptions
| where timestamp between(startofweek(ago(7days))..endofweek(ago(7days)))
| summarize count() by bin(TimeGenerated, 1day)
| sort by timestamp asc
_TimeReceived
The _TimeReceived property contains the date and time that the record was received by the Azure Monitor
ingestion point in the Azure cloud. This can be useful for identifying latency issues between the data source and
the cloud. An example would be a networking issue causing a delay with data being sent from an agent. See Log
data ingestion time in Azure Monitor for more details.
The following query gives the average latency by hour for event records from an agent. This includes the time
from the agent to the cloud and and the total time for the record to be available for log queries.
Event
| where TimeGenerated > ago(1d)
| project TimeGenerated, TimeReceived = _TimeReceived, IngestionTime = ingestion_time()
| extend AgentLatency = toreal(datetime_diff('Millisecond',TimeReceived,TimeGenerated)) / 1000
| extend TotalLatency = toreal(datetime_diff('Millisecond',IngestionTime,TimeGenerated)) / 1000
| summarize avg(AgentLatency), avg(TotalLatency) by bin(TimeGenerated,1hr)
search *
| where TimeGenerated > ago(1h)
| summarize count() by Type
_ItemId
The _ItemId property holds a unique identifier for the record.
_ResourceId
The _ResourceId property holds a unique identifier for the resource that the record is associated with. This gives
you a standard property to use to scope your query to only records from a particular resource, or to join related
data across multiple tables.
For Azure resources, the value of _ResourceId is the Azure resource ID URL. The property is currently limited to
Azure resources, but it will be extended to resources outside of Azure such as on-premises computers.
NOTE
Some data types already have fields that contain Azure resource ID or at least parts of it like subscription ID. While these
fields are kept for backward compatibility, it is recommended to use the _ResourceId to perform cross correlation since it will
be more consistent.
Examples
The following query joins performance and event data for each computer. It shows all events with an ID of 101
and processor utilization over 50%.
Perf
| where CounterName == "% User Time" and CounterValue > 50 and _ResourceId != ""
| join kind=inner (
Event
| where EventID == 101
) on _ResourceId
The following query joins AzureActivity records with SecurityEvent records. It shows all activity operations with
users that were logged in to these machines.
AzureActivity
| where
OperationName in ("Restart Virtual Machine", "Create or Update Virtual Machine", "Delete Virtual Machine")
and ActivityStatus == "Succeeded"
| join kind= leftouter (
SecurityEvent
| where EventID == 4624
| summarize LoggedOnAccounts = makeset(Account) by _ResourceId
) on _ResourceId
The following query parses _ResourceId and aggregates billed data volumes per Azure subscription.
union withsource = tt *
| where _IsBillable == true
| parse tolower(_ResourceId) with "/subscriptions/" subscriptionId "/resourcegroups/"
resourceGroup "/providers/" provider "/" resourceType "/" resourceName
| summarize Bytes=sum(_BilledSize) by subscriptionId | sort by Bytes nulls last
Use these union withsource = tt * queries sparingly as scans across data types are expensive to execute.
_IsBillable
The _IsBillable property specifies whether ingested data is billable. Data with _IsBillable equal to false are
collected for free and not billed to your Azure account.
Examples
To get a list of computers sending billed data types, use the following query:
NOTE
Use queries with union withsource = tt * sparingly as scans across data types are expensive to execute.
union withsource = tt *
| where _IsBillable == true
| extend computerName = tolower(tostring(split(Computer, '.')[0]))
| where computerName != ""
| summarize TotalVolumeBytes=sum(_BilledSize) by computerName
This can be extended to return the count of computers per hour that are sending billed data types:
union withsource = tt *
| where _IsBillable == true
| extend computerName = tolower(tostring(split(Computer, '.')[0]))
| where computerName != ""
| summarize dcount(computerName) by bin(TimeGenerated, 1h) | sort by TimeGenerated asc
_BilledSize
The _BilledSize property specifies the size in bytes of data that will be billed to your Azure account if _IsBillable
is true.
Examples
To see the size of billable events ingested per computer, use the _BilledSize property which provides the size in
bytes:
union withsource = tt *
| where _IsBillable == true
| summarize Bytes=sum(_BilledSize) by Computer | sort by Bytes nulls last
To see the size of billable events ingested per subscription, use the following query:
union withsource=table *
| where _IsBillable == true
| parse _ResourceId with "/subscriptions/" SubscriptionId "/" *
| summarize Bytes=sum(_BilledSize) by SubscriptionId | sort by Bytes nulls last
To see the size of billable events ingested per resource group, use the following query:
union withsource=table *
| where _IsBillable == true
| parse _ResourceId with "/subscriptions/" SubscriptionId "/resourcegroups/" ResourceGroupName "/" *
| summarize Bytes=sum(_BilledSize) by SubscriptionId, ResourceGroupName | sort by Bytes nulls last
To see the count of events ingested per computer, use the following query:
union withsource = tt *
| summarize count() by Computer | sort by count_ nulls last
To see the count of billable events ingested per computer, use the following query:
union withsource = tt *
| where _IsBillable == true
| summarize count() by Computer | sort by count_ nulls last
To see the count of billable data types from a specific computer, use the following query:
union withsource = tt *
| where Computer == "computer name"
| where _IsBillable == true
| summarize count() by tt | sort by count_ nulls last
Next steps
Read more about how Azure Monitor log data is stored.
Get a lesson on writing log queries.
Get a lesson on joining tables in log queries.
Azure Monitor log queries
10/24/2019 • 2 minutes to read • Edit Online
Azure Monitor logs are built on Azure Data Explorer, and Azure Monitor log queries use a version of the same
Kusto query language. The Kusto query language documentation has all of the details for the language and
should be your primary resource for writing Azure Monitor log queries. This page provides links to other
resources for learning how to write queries and on differences with the Azure Monitor implementation of the
language.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a
Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the
terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Getting started
Get started with Azure Monitor Log Analytics is a lesson for writing queries and working with results in the
Azure portal.
Get started with Azure Monitor log queries is a lesson for writing queries using Azure Monitor log data.
Concepts
Analyze log data in Azure Monitor gives a brief overview of log queries and describes how Azure Monitor log
data is structured.
Viewing and analyzing log data in Azure Monitor explains the portals where you create and run log queries.
Reference
Query language reference is the complete language reference for the Kusto query language.
Azure Monitor log query language differences describes differences between versions of the Kusto query
language.
Standard properties in Azure Monitor log records describes properties that are standard to all Azure Monitor
log data.
Perform cross-resource log queries in Azure Monitor describes how to write log queries that use data from
multiple Log Analytics workspaces and Application Insights applications.
Examples
Azure Monitor log query examples provides example queries using Azure Monitor log data.
Lessons
Working with strings in Azure Monitor log queries describes how to work with string data.
Working with date time values in Azure Monitor log queries describes how to work with date and time data.
Aggregations in Azure Monitor log queries and Advanced aggregations in Azure Monitor log queries describe
how to aggregate and summarize data.
Joins in Azure Monitor log queries describes how to join data from multiple tables.
Working with JSON and data Structures in Azure Monitor log queries describes how to parse json data.
Writing advanced log queries in Azure Monitor describes strategies for creating complex queries and reusing
code.
Creating charts and diagrams from Azure Monitor log queries describes how to visualize data from a log
query.
Cheatsheets
SQL to Azure Monitor log query assists users who are already familiar with SQL.
Splunk to Azure Monitor log query assists users who are already familiar with Splunk.
Next steps
Access the complete reference documentation for the Kusto query language.
Azure Monitor log query language differences
10/25/2019 • 2 minutes to read • Edit Online
While logs in Azure Monitor is built on Azure Data Explorer and uses the same Kusto query language, the version
of the language does have some differences. This article identifies elements that are different between the version
of the language used for Data Explorer and the version used for Azure Monitor log queries.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a
Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the
terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Next steps
Get references to different resources for writing Azure Monitor log queries.
Access the complete reference documentation for Kusto query language.
Useful operators in Azure Monitor log queries
12/23/2019 • 2 minutes to read • Edit Online
The table below provides some common functions to use for different scenarios in Azure Monitor log queries.
Useful operators
CATEGORY RELEVANT ANALYTICS FUNCTION
Next steps
Go through a lesson on the writing log queries in Azure Monitor.
Using functions in Azure Monitor log queries
10/24/2019 • 2 minutes to read • Edit Online
To use a log query with another query you can save it as a function. This allows you to simplify complex queries by
breaking them into parts and allows you to reuse common code with multiple queries.
Create a function
Create a function with Log Analytics in the Azure portal by clicking Save and then providing the information in the
following table.
SETTING DESCRIPTION
Save as Function
Function Alias Short name to use the function in other queries. May not
contain spaces and must be unique.
NOTE
A function in Azure Monitor cannot contain another function.
Use a function
Use a function by including its alias in another query. It can be used like any other table.
Example
The following sample query returns all missing security updates reported in the last day. Save this query as a
function with the alias security_updates_last_day.
Update
| where TimeGenerated > ago(1d)
| where Classification == "Security Updates"
| where UpdateState == "Needed"
Create another query and reference the security_updates_last_day function to search for SQL -related needed
security updates.
Next steps
See other lessons for writing Azure Monitor log queries:
String operations
Date and time operations
Aggregation functions
Advanced aggregations
JSON and data structures
Joins
Charts
Transition from Log Analytics log search to Azure
Monitor logs
12/13/2019 • 2 minutes to read • Edit Online
The log search in Log Analytics was recently replaced with a new experience for analyzing Azure Monitor logs. The
Log search page is currently still accessible through the Logs (classic) menu item in the Log Analytics
workspaces page in the Azure portal but will be removed February 15th, 2019. This article describes differences
between the two experiences to help you transition from log search.
In Azure Monitor logs, select Filter (preview ) to display filters. Click on the filter icon to display addition filters.
Select a filter and click Apply & Run to run the query with the selected filter.
Extract custom fields
In Log Search, you extract custom fields from the List view, where a field’s menu includes the action Extract fields
from Table.
In Azure Monitor logs, you extract custom fields from the table view. Expand a record by clicking the arrow to its
left then click the ellipsis to access the Extract fields action.
Functions and computer groups
To save a search in Log Search, select Saved searches and Add to provide a name, category, and query text.
Create a computer group by adding a function alias.
To save the current query in Azure Monitor logs, select Save. Change Save as to Function and provide a Function
Alias to create a function. Select Save this query as a computer group to use the function alias for a computer
group.
Saved queries
In Log Search, your saved queries are available through the action bar item Saved searches. In Azure Monitor
logs, access saved queries from Query Explorer.
In Azure Monitor logs, you must modify the query to return these records. Expand one of the rows in the results
and click the + next to the value to add it to the query. Then comment out the summarize command and run the
query again.
Take action
In Log Search, you can start a runbook from a search result by selecting Take action.
In Azure Monitor logs, create an alert from the log query. Configure an action group with one or more actions that
will run in response to the alert.
Next steps
Learn more about the new Azure Monitor logs experience.
Work with strings in Azure Monitor log queries
12/23/2019 • 7 minutes to read • Edit Online
NOTE
You should complete Get started with Azure Monitor Log Analytics and Getting started with Azure Monitor log queries
before completing this tutorial.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
This article describes how to edit, compare, search in and perform a variety of other operations on strings.
Each character in a string has an index number, according to its location. The first character is at index 0, the next
character is 1, and so on. Different string functions use index numbers as shown in the following sections. Many
of the following examples use the print command for to demonstrate string manipulation without using a
specific data source.
To prevent "\" from acting as an escape character, add "@" as a prefix to the string:
String comparisons
OPERATOR DESCRIPTION CASE-SENSITIVE EXAMPLE (YIELDS TRUE )
countof
Counts occurrences of a substring in a string. Can match plain strings or use regex. Plain string matches may
overlap while regex matches do not.
Syntax
Arguments:
text - The input string
search - Plain string or regular expression to match inside text.
kind - normal | regex (default: normal).
Returns
The number of times that the search string can be matched in the container. Plain string matches may overlap
while regex matches do not.
Examples
Plain string matches
Regex matches
extract
Gets a match for a regular expression from a given string. Optionally also converts the extracted substring to the
specified type.
Syntax
Arguments
regex - A regular expression.
captureGroup - A positive integer constant indicating the capture group to extract. 0 for the entire match, 1 for
the value matched by the first '('parenthesis')' in the regular expression, 2 or more for subsequent
parentheses.
text - A string to search.
typeLiteral - An optional type literal (for example, typeof(long)). If provided, the extracted substring is
converted to this type.
Returns
The substring matched against the indicated capture group captureGroup, optionally converted to typeLiteral. If
there's no match, or the type conversion fails, return null.
Examples
The following example extracts the last octet of ComputerIP from a heartbeat record:
Heartbeat
| where ComputerIP != ""
| take 1
| project ComputerIP, last_octet=extract("([0-9]*$)", 1, ComputerIP)
The following example extracts the last octet, casts it to a real type (number) and calculates the next IP value
Heartbeat
| where ComputerIP != ""
| take 1
| extend last_octet=extract("([0-9]*$)", 1, ComputerIP, typeof(real))
| extend next_ip=(last_octet+1)%255
| project ComputerIP, last_octet, next_ip
In the example below, the string Trace is searched for a definition of "Duration". The match is cast to real and
multiplied by a time constant (1 s) which casts Duration to type timespan.
isempty(value)
isnotempty(value)
Examples
Heartbeat | where isnotempty(ComputerIP) | take 1 // return 1 Heartbeat record in which ComputerIP isn't
empty
parseurl
Splits a URL into its parts (protocol, host, port, etc.), and returns a dictionary object containing the parts as
strings.
Syntax
parseurl(urlstring)
Examples
print parseurl("http://user:pass@contoso.com/icecream/buy.aspx?a=1&b=2#tag")
replace
Replaces all regex matches with another string.
Syntax
Arguments
regex - The regular expression to match by. It can contain capture groups in '('parentheses')'.
rewrite - The replacement regex for any match made by matching regex. Use \0 to refer to the whole match,
\1 for the first capture group, \2, and so on for subsequent capture groups.
input_text - The input string to search in.
Returns
The text after replacing all matches of regex with evaluations of rewrite. Matches don't overlap.
Examples
SecurityEvent
| take 1
| project Activity
| extend replaced = replace(@"(\d+) -", @"Activity ID \1: ", Activity)
ACTIVITY REPLACED
4663 - An attempt was made to access an object Activity ID 4663: An attempt was made to access an object.
split
Splits a given string according to a specified delimiter, and returns an array of the resulting substrings.
Syntax
Arguments:
source - The string to be split according to the specified delimiter.
delimiter - The delimiter that will be used in order to split the source string.
requestedIndex - An optional zero-based index. If provided, the returned string array will hold only that item
(if exists).
Examples
strcat
Concatenates string arguments (supports 1-16 arguments).
Syntax
Examples
strlen
Returns the length of a string.
Syntax
strlen("text_to_evaluate")
Examples
substring
Extracts a substring from a given source string, starting at the specified index. Optionally, the length of the
requested substring can be specified.
Syntax
Arguments:
source - The source string that the substring will be taken from.
startingIndex - The zero-based starting character position of the requested substring.
length - An optional parameter that can be used to specify the requested length of the returned substring.
Examples
print substring("abcdefg", 1, 2); // result: "bc"
print substring("123456", 1); // result: "23456"
print substring("123456", 2, 2); // result: "34"
print substring("ABCD", 0, 2); // result: "AB"
tolower, toupper
Converts a given string to all lower or upper case.
Syntax
tolower("value")
toupper("value")
Examples
Next steps
Continue with the advanced tutorials:
Aggregation functions
Advanced aggregations
Charts and diagrams
Working with JSON and data structures
Advanced query writing
Joins - cross analysis
Working with date time values in Azure Monitor log
queries
12/13/2019 • 3 minutes to read • Edit Online
NOTE
You should complete Get started with the Analytics portal and Getting started with queries before completing this lesson.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
This article describes how to work with date and time data in Azure Monitor log queries.
d day
h hour
m minute
s second
ms millisecond
microsecond microsecond
tick nanosecond
Datetimes can be created by casting a string using the todatetime operator. For example, to review the VM
heartbeats sent in a specific timeframe, use the between operator to specify a time range.
Heartbeat
| where TimeGenerated between(datetime("2018-06-30 22:46:42") .. datetime("2018-07-01 00:57:27"))
Another common scenario is comparing a datetime to the present. For example, to see all heartbeats over the last
two minutes, you can use the now operator together with a timespan that represents two minutes:
Heartbeat
| where TimeGenerated > now() - 2m
Heartbeat
| where TimeGenerated > now(-2m)
The shortest and most readable method though is using the ago operator:
Heartbeat
| where TimeGenerated > ago(2m)
Suppose that instead of knowing the start and end time, you know the start time and the duration. You can
rewrite the query as follows:
Event
| where TimeGenerated > ago(30m)
| where EventLevelName == "Error"
| extend timeAgo = now() - TimeGenerated
The timeAgo column holds values such as: "00:09:31.5118992", meaning they're formatted as hh:mm:ss.fffffff. If
you want to format these values to the numver of minutes since the start time, divide that value by "1 minute":
Event
| where TimeGenerated > ago(30m)
| where EventLevelName == "Error"
| extend timeAgo = now() - TimeGenerated
| extend timeAgoMinutes = timeAgo/1m
Event
| where TimeGenerated > ago(30m)
| summarize events_count=count() by bin(TimeGenerated, 5m)
This query produces the following table:
TIMEGENERATED(UTC) EVENTS_COUNT
2018-08-01T09:30:00.000 54
2018-08-01T09:35:00.000 41
2018-08-01T09:40:00.000 42
2018-08-01T09:45:00.000 41
2018-08-01T09:50:00.000 41
2018-08-01T09:55:00.000 16
Event
| where TimeGenerated > ago(4d)
| summarize events_count=count() by startofday(TimeGenerated)
TIMESTAMP COUNT_
2018-07-28T00:00:00.000 7,136
2018-07-29T00:00:00.000 12,315
2018-07-30T00:00:00.000 16,847
2018-07-31T00:00:00.000 12,616
2018-08-01T00:00:00.000 5,416
Time zones
Since all datetime values are expressed in UTC, it's often useful to convert these values into the local timezone.
For example, use this calculation to convert UTC to PST times:
Event
| extend localTimestamp = TimeGenerated - 8h
Related functions
CATEGORY FUNCTION
Next steps
See other lessons for using the Kusto query language with Azure Monitor log data:
String operations
Aggregation functions
Advanced aggregations
JSON and data structures
Advanced query writing
Joins
Charts
Aggregations in Azure Monitor log queries
10/24/2019 • 3 minutes to read • Edit Online
NOTE
You should complete Get started with the Analytics portal and Getting started with queries before completing this lesson.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
This article describes aggregation functions in Azure Monitor log queries that offer useful ways to analyze your
data. These functions all work with the summarize operator that produces a table with aggregated results of the
input table.
Counts
count
Count the number of rows in the result set after any filters are applied. The following example returns the total
number of rows in the Perf table from the last 30 minutes. The result is returned in a column named count_
unless you assign it a specific name:
Perf
| where TimeGenerated > ago(30m)
| summarize count()
Perf
| where TimeGenerated > ago(30m)
| summarize num_of_records=count()
Perf
| where TimeGenerated > ago(30m)
| summarize count() by bin(TimeGenerated, 5m)
| render timechart
The output from this example shows the perf record count trendline in 5 minutes' intervals:
dcount, dcountif
Use dcount and dcountif to count distinct values in a specific column. The following query evaluates how
many distinct computers sent heartbeats in the last hour:
Heartbeat
| where TimeGenerated > ago(1h)
| summarize dcount(Computer)
To count only the Linux computers that sent heartbeats, use dcountif :
Heartbeat
| where TimeGenerated > ago(1h)
| summarize dcountif(Computer, OSType=="Linux")
Evaluating subgroups
To perform a count or other aggregations on subgroups in your data, use the by keyword. For example, to
count the number of distinct Linux computers that sent heartbeats in each country/region:
Heartbeat
| where TimeGenerated > ago(1h)
| summarize distinct_computers=dcountif(Computer, OSType=="Linux") by RemoteIPCountry
REMOTEIPCOUNTRY DISTINCT_COMPUTERS
United States 19
Canada 3
Ireland 0
United Kingdom 0
Netherlands 2
To analyze even smaller subgroups of your data, add additional column names to the by section. For example,
you might want to count the distinct computers from each country/region per OSType:
Heartbeat
| where TimeGenerated > ago(1h)
| summarize distinct_computers=dcountif(Computer, OSType=="Linux") by RemoteIPCountry, OSType
Percentiles and Variance
When evaluating numerical values, a common practice is to average them using summarize avg(expression) .
Averages are affected by extreme values that characterize only a few cases. To address that issue, you can use
less sensitive functions such as median or variance .
Percentile
To find the median value, use the percentile function with a value to specify the percentile:
Perf
| where TimeGenerated > ago(30m)
| where CounterName == "% Processor Time" and InstanceName == "_Total"
| summarize percentiles(CounterValue, 50) by Computer
You can also specify different percentiles to get an aggregated result for each:
Perf
| where TimeGenerated > ago(30m)
| where CounterName == "% Processor Time" and InstanceName == "_Total"
| summarize percentiles(CounterValue, 25, 50, 75, 90) by Computer
This might show that some computer CPUs have similar median values, but while some are steady around the
median, other computers have reported much lower and higher CPU values meaning they experienced spikes.
Variance
To directly evaluate the variance of a value, use the standard deviation and variance methods:
Perf
| where TimeGenerated > ago(30m)
| where CounterName == "% Processor Time" and InstanceName == "_Total"
| summarize stdev(CounterValue), variance(CounterValue) by Computer
A good way to analyze the stability of the CPU usage is to combine stdev with the median calculation:
Perf
| where TimeGenerated > ago(130m)
| where CounterName == "% Processor Time" and InstanceName == "_Total"
| summarize stdev(CounterValue), percentiles(CounterValue, 50) by Computer
See other lessons for using the Kusto query language with Azure Monitor log data:
String operations
Date and time operations
Advanced aggregations
JSON and data structures
Advanced query writing
Joins
Charts
Advanced aggregations in Azure Monitor log
queries
12/13/2019 • 3 minutes to read • Edit Online
NOTE
You should complete Aggregations in Azure Monitor queries before completing this lesson.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
This article describes some of the more advanced aggregation options available to Azure Monitor queries.
Event
| where TimeGenerated > ago(12h)
| order by TimeGenerated desc
| summarize makelist(EventID) by Computer
COMPUTER LIST_EVENTID
computer1 [704,701,1501,1500,1085,704,704,701]
computer2 [326,105,302,301,300,102]
... ...
makelist generates a list in the order that data was passed into it. To sort events from oldest to newest, use
asc in the order statement instead of desc .
It is also useful to create a list of just distinct values. This is called a Set and can be generated with makeset :
Event
| where TimeGenerated > ago(12h)
| order by TimeGenerated desc
| summarize makeset(EventID) by Computer
COMPUTER LIST_EVENTID
computer1 [704,701,1501,1500,1085]
COMPUTER LIST_EVENTID
computer2 [326,105,302,301,300,102]
... ...
Like makelist , makeset also works with ordered data and will generate the arrays based on the order of the
rows that are passed into it.
Expanding lists
The inverse operation of makelist or makeset is mvexpand , which expands a list of values to separate rows. It
can expand across any number of dynamic columns, both JSON and array. For example, you could check the
Heartbeat table for solutions sending data from computers that sent a heartbeat in the last hour:
Heartbeat
| where TimeGenerated > ago(1h)
| project Computer, Solutions
COMPUTER SOLUTIONS
... ...
Use mvexpand to show each value in a separate row instead of a comma-separated list:
Heartbeat
| where TimeGenerated > ago(1h)
| project Computer, split(Solutions, ",")
| mvexpand Solutions
COMPUTER SOLUTIONS
computer1 "security"
computer1 "updates"
computer1 "changeTracking"
computer2 "security"
computer2 "updates"
computer3 "antiMalware"
computer3 "changeTracking"
COMPUTER SOLUTIONS
... ...
You could then use makelist again to group items together, and this time see the list of computers per solution:
Heartbeat
| where TimeGenerated > ago(1h)
| project Computer, split(Solutions, ",")
| mvexpand Solutions
| summarize makelist(Computer) by tostring(Solutions)
SOLUTIONS LIST_COMPUTER
"antiMalware" ["computer3"]
... ...
Heartbeat
| where TimeGenerated > ago(12h)
| summarize count() by Category, bin(TimeGenerated, 1h)
In these results though the bucket associated with "2017-06-06T19:00:00Z" is missing because there isn't any
heartbeat data for that hour. Use the make-series function to assign a default value to empty buckets. This will
generate a row for each category with two extra array columns, one for values, and one for matching time
buckets:
Heartbeat
| make-series count() default=0 on TimeGenerated in range(ago(1d), now(), 1h) by Category
The third element of the count_ array is a 0 as expected, and there is a matching timestamp of "2017-06-
06T19:00:00.0000000Z" in the TimeGenerated array. This array format is difficult to read though. Use mvexpand
to expand the arrays and produce the same format output as generated by summarize :
Heartbeat
| make-series count() default=0 on TimeGenerated in range(ago(1d), now(), 1h) by Category
| mvexpand TimeGenerated, count_
| project Category, TimeGenerated, count_
Next steps
See other lessons for using the Kusto query language with Azure Monitor log data:
String operations
Date and time operations
Aggregation functions
Advanced aggregations
JSON and data structures
Joins
Charts
Joins in Azure Monitor log queries
12/13/2019 • 3 minutes to read • Edit Online
NOTE
You should complete Get started with Azure Monitor Log Analytics and Azure Monitor log queries before completing this
lesson.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
Joins allow you to analyze data from multiple tables, in the same query. They merge the rows of two data sets by
matching values of specified columns.
SecurityEvent
| where EventID == 4624 // sign-in events
| project Computer, Account, TargetLogonId, LogonTime=TimeGenerated
| join kind= inner (
SecurityEvent
| where EventID == 4634 // sign-out events
| project TargetLogonId, LogoffTime=TimeGenerated
) on TargetLogonId
| extend Duration = LogoffTime-LogonTime
| project-away TargetLogonId1
| top 10 by Duration desc
In this example, the first dataset filters for all sign-in events. This is joined with a second dataset that filters for all
sign-out events. The projected columns are Computer, Account, TargetLogonId, and TimeGenerated. The
datasets are correlated by a shared column, TargetLogonId. The output is a single record per correlation, which
has both the sign-in and sign-out time.
If both datasets have columns with the same names, the columns of the right-side dataset would be given an
index number, so in this example the results would show TargetLogonId with values from the left-side table and
TargetLogonId1 with values from the right-side table. In this case, the second TargetLogonId1 column was
removed by using the project-away operator.
NOTE
To improve performance, keep only the relevant columns of the joined data-sets, using the project operator.
Use the following syntax to join two datasets and the joined key has a different name between the two tables:
Table1
| join ( Table2 )
on $left.key1 == $right.key2
Lookup Tables
A common use of joins is using static mapping of values using datatable that can help in transforming the
results into more presentable way. For example, to enrich the security event data with the event name for each
event ID.
Join kinds
Specify the type of join with the kind argument. Each type performs a different match between the records of the
given tables as described in the following table.
innerunique This is the default join mode. First the values of the matched
column on the left table are found, and duplicate values are
removed. Then the set of unique values is matched against
the right table.
leftouter All records in the left table and matching records in the right
table are included in the results. Unmatched output
properties contain nulls.
leftanti Records from the left side that do not have matches from
the right are included in the results. The results table has
only columns from the left table.
JOIN TYPE DESCRIPTION
leftsemi Records from the left side that have matches from the right
are included in the results. The results table has only columns
from the left table.
Best practices
Consider the following points for optimal performance:
Use a time filter on each table to reduce the records that must be evaluated for the join.
Use where and project to reduce the numbers of rows and columns in the input tables before the join.
If one table is always smaller than the other, use it as the left side of the join.
Next steps
See other lessons for using Azure Monitor log queries:
String operations
Aggregation functions
Advanced aggregations
JSON and data structures
Advanced query writing
Charts
Working with JSON and data Structures in Azure
Monitor log queries
12/13/2019 • 2 minutes to read • Edit Online
NOTE
You should complete Get started with Azure Monitor Log Analytics and Getting started with Azure Monitor log queries
before completing this lesson.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
Nested objects are objects that contain other objects in an array or a map of key-value pairs. These objects are
represented as JSON strings. This article describes how JSON is used to retrieve data and analyze nested
objects.
If there is only one element, you can use only the dot notation:
arraylength
Use arraylength to count the number of elements in an array:
mvexpand
Use mvexpand to break the properties of an object into separate rows.
buildschema
Use buildschema to get the schema that admits all values of an object:
This output describes the names of the object fields and their matching data types.
Nested objects may have different schemas such as in the following example:
Next steps
See other lessons for using log queries in Azure Monitor:
String operations
Date and time operations
Aggregation functions
Advanced aggregations
Advanced query writing
Joins
Charts
Writing advanced queries in Azure Monitor
10/24/2019 • 2 minutes to read • Edit Online
NOTE
You should complete Get started with Azure Monitor Log Analytics and Getting started with queries before completing this
lesson.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
You can also assign constant values to variables. This supports a method to set up parameters for the fields that
you need to change every time you execute the query. Modify those parameters as needed. For example, to
calculate the free disk space and free memory (in percentiles), in a given time window:
This makes it easy to change the start of end time the next time you run the query.
Local functions and parameters
Use let statements to create functions that can be used in the same query. For example, define a function that
takes a datetime field (in the UTC format) and converts it to a standard US format.
let utc_to_us_date_format = (t:datetime)
{
strcat(getmonth(t), "/", dayofmonth(t),"/", getyear(t), " ",
bin((t-1h)%12h+1h,1s), iff(t%24h<12h, "AM", "PM"))
};
Event
| where TimeGenerated > ago(1h)
| extend USTimeGenerated = utc_to_us_date_format(TimeGenerated)
| project TimeGenerated, USTimeGenerated, Source, Computer, EventLevel, EventData
Print
print will return a table with a single column and a single row, showing the result of a calculation. This is often
used in cases where you need a simple calculation. For example, to find the current time in PST and add a column
with EST:
Datatable
datatable allows you to define a set of data. You provide a schema and a set of values and then pipe the table
into any other query elements. For example to create a table of RAM usage and calculate their average value per
hour:
Datatable constructs are also very useful when creating a lookup table. For example, to map table data such as
event IDs from the SecurityEvent table, to event types listed elsewhere, create a lookup table with the event types
using datatable and join this datatable with SecurityEvent data:
let eventCodes = datatable (EventID: int, EventType:string)
[
4625, "Account activity",
4688, "Process action",
4634, "Account activity",
4672, "Access",
4624, "Account activity",
4799, "Access management",
4798, "Access management",
5059, "Key operation",
4648, "A logon was attempted using explicit credentials",
4768, "Access management",
4662, "Other",
8002, "Process action",
4904, "Security event management",
4905, "Security event management",
];
SecurityEvent
| where TimeGenerated > ago(1h)
| join kind=leftouter (
eventCodes
) on EventID
| project TimeGenerated, Account, AccountType, Computer, EventType
Next steps
See other lessons for using the Kusto query language with Azure Monitor log data:
String operations
Date and time operations
Aggregation functions
Advanced aggregations
JSON and data structures
Joins
Charts
Creating charts and diagrams from Azure Monitor
log queries
12/13/2019 • 2 minutes to read • Edit Online
NOTE
You should complete Advanced aggregations in Azure Monitor log queries before completing this lesson.
NOTE
You can work through this exercise in your own Log Analytics environment, or you can use our Demo environment, which
includes plenty of sample data.
This article describes various visualizations in Azure Monitor to display your log data in different ways.
Heartbeat
| where TimeGenerated > ago(1h)
| summarize count(Computer) by OSType
To get a better view, select Chart, and choose the Pie option to visualize the results:
Timecharts
Show the average, 50th and 95th percentiles of processor time in bins of 1 hour. The query generates multiple
series and you can then select which series to show in the time chart:
Perf
| where TimeGenerated > ago(1d)
| where CounterName == "% Processor Time"
| summarize avg(CounterValue), percentiles(CounterValue, 50, 95) by bin(TimeGenerated, 1h)
Reference line
A reference line can help you easily identifying if the metric exceeded a specific threshold. To add a line to a
chart, extend the dataset with a constant column:
Perf
| where TimeGenerated > ago(1d)
| where CounterName == "% Processor Time"
| summarize avg(CounterValue), percentiles(CounterValue, 50, 95) by bin(TimeGenerated, 1h)
| extend Threshold = 20
Multiple dimensions
Multiple expressions in the by clause of summarize create multiple rows in the results, one for each
combination of values.
SecurityEvent
| where TimeGenerated > ago(1d)
| summarize count() by tostring(EventID), AccountType, bin(TimeGenerated, 1h)
When you view the results as a chart, it uses the first column from the by clause. The following example shows
a stacked column chart using the EventID. Dimensions must be of string type, so in this example the EventID is
being cast to string.
You can switch between by selecting the dropdown with the column name.
Next steps
See other lessons for using the Kusto query language with Azure Monitor log data:
String operations
Date and time operations
Aggregation functions
Advanced aggregations
JSON and data structures
Advanced query writing
Joins
Search queries in Azure Monitor logs
10/25/2019 • 3 minutes to read • Edit Online
Azure Monitor log queries can start with either a table name or a search command. This tutorial covers search-
based queries. There are advantages to each method.
Table-based queries start by scoping the query and therefore tend to be more efficient than search queries. Search
queries are less structured which makes them the better choice when searching for a specific value across columns
or tables. search can scan all columns in a given table, or in all tables, for the specified value. The amount of data
being processed could be enormous, which is why these queries could take longer to complete and might return
very large result sets.
Search a term
The search command is typically used to search a specific term. In the following example, all columns in all tables
are scanned for the term "error":
search "error"
| take 100
While they're easy to use, unscoped queries like the one showed above are not efficient and are likely to return
many irrelevant results. A better practice would be to search in the relevant table, or even a specific column.
Table scoping
To search a term in a specific table, add in (table-name) just after the search operator:
or in multiple tables:
TIP
If you use == instead of : , the results would include records in which the Source column has the exact value "error", and
in this exact case. Using ':' will include records where Source has values such as "error code 404" or "Error".
Case-sensitivity
By default, term search is case-insensitive, so searching "dns" could yield results such as "DNS", "dns", or "Dns". To
make the search case-sensitive, use the kind option:
To search terms that starts with "corp" and ends in ".com", such as "corp.mydomain.com""
You can also get everything in a table by using just a wild card: search in (Event) * , but that would be the same
as writing just Event .
TIP
While you can use search * to get every column from every table, it's recommended that you always scope your queries
to specific tables. Unscoped queries may take a while to complete and might return too many results.
If you have multiple search conditions, you can combine them into the same query using parentheses:
search in (Event) "error" and ("register" or "marshal*")
| take 100
The results of this example would be records that contain the term "error" and also contain either "register" or
something that starts with "marshal".
Next steps
See further tutorials on the Kusto query language site.
SQL to Azure Monitor log query cheat sheet
12/23/2019 • 2 minutes to read • Edit Online
The table below helps users who are familiar with SQL to learn the Kusto query language to write log queries in
Azure Monitor. Have a look at the T-SQL command for solving common scenarios and the equivalent in an Azure
Monitor log query.
Select specific columns from a table SELECT name, resultCode FROM dependencies
dependencies | project name, resultCode
Select 100 records from a table SELECT TOP 100 * FROM dependencies
dependencies | take 100
Next steps
Go through the lessons on writing log queries in Azure Monitor.
Splunk to Azure Monitor log query
12/23/2019 • 5 minutes to read • Edit Online
This article is intended to assist users who are familiar with Splunk to learn the Kusto query language to write log
queries in Azure Monitor. Direct comparisons are made between the two to understand key differences and also
similarities where you can leverage your existing knowledge.
Data caches buckets Caching and retention policies Controls the period and
caching level for the data. This
setting directly impacts the
performance of queries and
cost of the deployment.
Structured event metadata N/A table Splunk does not have the
concept exposed to the
search language of event
metadata. Azure Monitor logs
has the concept of a table,
which has columns. Each
event instance is mapped to a
row.
Event ingestion time System Time ingestion_time() In Splunk, each event gets a
system timestamp of the time
that the event was indexed. In
Azure Monitor, you can
define a policy called
ingestion_time that exposes a
system column that can be
referenced through the
ingestion_time() function.
Functions
The following table specifies functions in Azure Monitor that are equivalent to Splunk functions.
if iff() (1)
(1) In Splunk, the function is invoked with the eval operator. In Azure Monitor, it is used as part of extend or
project .
(2) In Splunk, the function is invoked with the eval operator. In Azure Monitor, it can be used with the where
operator.
Operators
The following sections give examples of using different operators between Splunk and Azure Monitor.
NOTE
For the purpose of the following example, the Splunk field rule maps to a table in Azure Monitor, and Splunk's default timestamp
maps to the Logs Analytics ingestion_time() column.
Search
In Splunk, you can omit the search keyword and specify an unquoted string. In Azure Monitor you must start each
query with find , an unquoted string is a column name, and the lookup value must be a quoted string.
Filter
Azure Monitor log queries start from a tabular result set where the filter. In Splunk, filtering is the default operation on
the current index. You can also use where operator in Splunk, but it is not recommended.
Rename
Azure Monitor uses the project-rename operator to rename a field. project-rename allows the query to take
advantage of any indexes pre-built for a field. Splunk has a rename operator to do the same.
Format results/Projection
Splunk does not seem to have an operator similar to project-away . You can use the UI to filter away fields.
Aggregation
See the Aggregations in Azure Monitor log queries for the different aggregation functions.
Join
Join in Splunk has significant limitations. The subquery has a limit of 10000 results (set in the deployment
configuration file), and there a limited number of join flavors.
Sort
In Splunk, to sort in ascending order you must use the reverse operator. Azure Monitor also supports defining where
to put nulls, at the beginning or at the end.
Multivalue expand
This is a similar operator in both Splunk and Azure Monitor.
De-duplicate
You can use summarize arg_min() instead to reverse the order of which record gets chosen.
Next steps
Go through a lesson on the writing log queries in Azure Monitor.
Perform cross-resource log queries in Azure Monitor
12/13/2019 • 5 minutes to read • Edit Online
Previously with Azure Monitor, you could only analyze data from within the current workspace, and it limited
your ability to query across multiple workspaces defined in your subscription. Additionally, you could only
search telemetry items collected from your web-based application with Application Insights directly in
Application Insights or from Visual Studio. This also made it a challenge to natively analyze operational and
application data together.
Now you can query not only across multiple Log Analytics workspaces, but also data from a specific Application
Insights app in the same resource group, another resource group, or another subscription. This provides you
with a system-wide view of your data. You can only perform these types of queries in Log Analytics.
Qualified name - is the “full name” of the workspace, composed of the subscription name, resource group,
and component name in this format: subscriptionName/resourceGroup/componentName.
workspace('contoso/contosoretail/contosoretail-it').Update | count
NOTE
Because Azure subscription names are not unique, this identifier might be ambiguous.
Azure Resource ID – the Azure-defined unique identity of the workspace. You use the Resource ID when
the resource name is ambiguous. For workspaces, the format is:
/subscriptions/subscriptionId/resourcegroups/resourceGroup/providers/microsoft.OperationalInsights/wo
rkspaces/componentName.
For example:
workspace("/subscriptions/e427519-5645-8x4e-1v67-
3b84b59a1985/resourcegroups/ContosoAzureHQ/providers/Microsoft.OperationalInsights/workspaces/contoso
retail-it").Update | count
Identifying an application
The following examples return a summarized count of requests made against an app named fabrikamapp in
Application Insights.
Identifying an application in Application Insights can be accomplished with the app (Identifier ) expression. The
Identifier argument specifies the app using one of the following:
Resource name - is a human readable name of the app, sometimes referred to as the component name.
app("fabrikamapp")
NOTE
Identifying an application by name assumes uniqueness across all accessible subscriptions. If you have multiple
applications with the specified name, the query fails because of the ambiguity. In this case, you must use one of
the other identifiers.
Qualified name - is the “full name” of the app, composed of the subscription name, resource group, and
component name in this format: subscriptionName/resourceGroup/componentName.
app("AI-Prototype/Fabrikam/fabrikamapp").requests | count
NOTE
Because Azure subscription names are not unique, this identifier might be ambiguous.
Azure Resource ID - the Azure-defined unique identity of the app. You use the Resource ID when the
resource name is ambiguous. The format is:
/subscriptions/subscriptionId/resourcegroups/resourceGroup/providers/microsoft.OperationalInsights/co
mponents/componentName.
For example:
app("/subscriptions/b459b4f6-912x-46d5-9cb1-
b43069212ab4/resourcegroups/Fabrikam/providers/microsoft.insights/components/fabrikamapp").requests |
count
You can now use this function in a cross-resource query like the following. The function alias
applicationsScoping returns the union of the requests table from all the defined applications. The query then
filters for failed requests and visualizes the trends by application. The parse operator is optional in this example.
It extracts the application name from SourceApp property.
applicationsScoping
| where timestamp > ago(12h)
| where success == 'False'
| parse SourceApp with * '(' applicationName ')' *
| summarize count() by applicationName, bin(timestamp, 1h)
| render timechart
NOTE
This method can’t be used with log alerts because the access validation of the alert rule resources, including workspaces
and applications, is performed at alert creation time. Adding new resources to the function after the alert creation isn’t
supported. If you prefer to use function for resource scoping in log alerts, you need to edit the alert rule in the portal or
with a Resource Manager template to update the scoped resources. Alternatively, you can include the list of resources in
the log alert query.
Next steps
Review Analyze log data in Azure Monitor for an overview of log queries and how Azure Monitor log data is
structured.
Review Azure Monitor log queries to view all of the resources for Azure Monitor log queries.
app() expression in Azure Monitor query
10/25/2019 • 2 minutes to read • Edit Online
The app expression is used in an Azure Monitor query to retrieve data from a specific Application Insights app in
the same resource group, another resource group, or another subscription. This is useful to include application
data in an Azure Monitor log query and to query data across multiple applications in an Application Insights
query.
Syntax
app( Identifier )
Arguments
Identifier: Identifies the app using one of the formats in the table below.
Notes
You must have read access to the application.
Identifying an application by its name assumes that it is unique across all accessible subscriptions. If you have
multiple applications with the specified name, the query will fail because of the ambiguity. In this case you must
use one of the other identifiers.
Use the related expression workspace to query across Log Analytics workspaces.
The app() expression is currently not supported in the search query when using the Azure portal to create a
custom log search alert rule, unless an Application Insights application is used as the resource for the alert rule.
Examples
app("fabrikamapp").requests | count
app("AI-Prototype/Fabrikam/fabrikamapp").requests | count
app("b438b4f6-912a-46d5-9cb1-b44069212ab4").requests | count
app("/subscriptions/7293b69-db12-44fc-9a66-
9c2005c3051d/resourcegroups/Fabrikam/providers/microsoft.insights/components/fabrikamapp").requests | count
union
(workspace("myworkspace").Heartbeat | where Computer contains "Con"),
(app("myapplication").requests | where cloud_RoleInstance contains "Con")
| count
union
(workspace("myworkspace").Heartbeat), (app("myapplication").requests)
| where TimeGenerated between(todatetime("2018-02-08 15:00:00") .. todatetime("2018-12-08 15:05:00"))
Next steps
See the workspace expression to refer to a Log Analytics workspace.
Read about how Azure Monitor data is stored.
Access full documentation for the Kusto query language.
workspace() expression in Azure Monitor log query
12/23/2019 • 2 minutes to read • Edit Online
The workspace expression is used in an Azure Monitor query to retrieve data from a specific workspace in the
same resource group, another resource group, or another subscription. This is useful to include log data in an
Application Insights query and to query data across multiple workspaces in a log query.
Syntax
workspace( Identifier )
Arguments
Identifier: Identifies the workspace using one of the formats in the table below.
Notes
You must have read access to the workspace.
A related expression is app that allows you to query across Application Insights applications.
Examples
workspace("contosoretail").Update | count
workspace("b438b4f6-912a-46d5-9cb1-b44069212ab4").Update | count
workspace("/subscriptions/e427267-5645-4c4e-9c67-
3b84b59a6982/resourcegroups/ContosoAzureHQ/providers/Microsoft.OperationalInsights/workspaces/contosoretail")
.Event | count
union
(workspace("myworkspace").Heartbeat | where Computer contains "Con"),
(app("myapplication").requests | where cloud_RoleInstance contains "Con")
| count
union
(workspace("myworkspace").Heartbeat), (app("myapplication").requests)
| where TimeGenerated between(todatetime("2018-02-08 15:00:00") .. todatetime("2018-12-08 15:05:00"))
Next steps
See the app expression to refer to an Application Insights app.
Read about how Azure Monitor data is stored.
Access full documentation for the Kusto query language.
resource() expression in Azure Monitor log query
10/25/2019 • 2 minutes to read • Edit Online
The resource expression is used in a Azure Monitor query scoped to a resource to retrieve data from other
resources.
Syntax
resource( Identifier )
Arguments
Identifier: Resource ID of a resource.
Resource Group or Subscription Includes data for the resource and all resource("/subscriptions/xxxxxxx-xxxx-
resources that it contains. xxxx-xxxx-
xxxxxxxxxxxx/resourcesgroups/myresour
cegroup)
Notes
You must have read access to the resource.
Examples
union (Heartbeat),(resource("/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/resourcesgroups/myresourcegroup/providers/microsoft.compute/virtualmachines/myvm").Heartbeat) |
summarize count() by _ResourceId, TenantId
union (Heartbeat),(resource("/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/resourcesgroups/myresourcegroup).Heartbeat) | summarize count() by _ResourceId, TenantId
Next steps
See Log query scope and time range in Azure Monitor Log Analytics for details on a query scope.
Access full documentation for the Kusto query language.
Unify multiple Azure Monitor Application Insights
resources
12/23/2019 • 4 minutes to read • Edit Online
This article describes how to query and view all your Application Insights log data in one place, even when they are
in different Azure subscriptions, as a replacement for the deprecation of the Application Insights Connector. The
number of Application Insights resources that you can include in a single query is limited to 100.
ApplicationInsights
| summarize by ApplicationName
Create a function using union operator with the list of applications, then save the query in your workspace as
function with the alias applicationsScoping.
You can modify the listed applications at any time in the portal by navigating to Query explorer in your workspace
and selecting the function for editing and then saving, or using the SavedSearch PowerShell cmdlet.
NOTE
This method can’t be used with log alerts because the access validation of the alert rule resources, including workspaces and
applications, is performed at alert creation time. Adding new resources to the function after the alert creation isn’t
supported. If you prefer to use function for resource scoping in log alerts, you need to edit the alert rule in the portal or with
a Resource Manager template to update the scoped resources. Alternatively, you can include the list of resources in the log
alert query.
The withsource= SourceApp command adds a column to the results that designates the application that sent the
log. The parse operator is optional in this example and uses to extracts the application name from SourceApp
property.
union withsource=SourceApp
app('Contoso-app1').requests,
app('Contoso-app2').requests,
app('Contoso-app3').requests,
app('Contoso-app4').requests,
app('Contoso-app5').requests
| parse SourceApp with * "('" applicationName "')" *
You are now ready to use applicationsScoping function in the cross-resource query:
applicationsScoping
| where timestamp > ago(12h)
| where success == 'False'
| parse SourceApp with * '(' applicationName ')' *
| summarize count() by applicationName, bin(timestamp, 1h)
| render timechart
The query uses Application Insights schema, although the query is executed in the workspace since the
applicationsScoping function returns the Application Insights data structure. The function alias returns the union of
the requests from all the defined applications. The query then filters for failed requests and visualizes the trends by
application.
NOTE
Cross-resource query in log alerts is supported in the new scheduledQueryRules API. By default, Azure Monitor uses the
legacy Log Analytics Alert API for creating new log alert rules from Azure portal, unless you switch from legacy Log Alerts
API. After the switch, the new API becomes the default for new alert rules in Azure portal and it lets you create cross-
resource query log alerts rules. You can create cross-resource query log alert rules without making the switch by using the
ARM template for scheduledQueryRules API – but this alert rule is manageable though scheduledQueryRules API and not
from Azure portal.
For example, if the connector stopped working on 2018-11-01, when you query logs across Application Insights
resources and applications data in the workspace, your query would be constructed like the following example:
applicationsScoping //this brings data from Application Insights resources
| where timestamp between (datetime("2018-11-01") .. now())
| where success == 'False'
| where duration > 1000
| union (
ApplicationInsights //this is Application Insights data in Log Analytics workspace
| where TimeGenerated < (datetime("2018-12-01")
| where RequestSuccess == 'False'
| where RequestDuration > 1000
| extend duration = RequestDuration //align to Application Insights schema
| extend timestamp = TimeGenerated //align to Application Insights schema
| extend name = RequestName //align to Application Insights schema
| extend resultCode = ResponseCode //align to Application Insights schema
| project-away RequestDuration , RequestName , ResponseCode , TimeGenerated
)
| project timestamp , duration , name , resultCode
AnonUserId user_id
ApplicationId appId
ApplicationName appName
ApplicationTypeVersion application_Version
AvailabilityCount itemCount
AvailabilityDuration duration
AvailabilityMessage message
AvailabilityRunLocation location
AvailabilityTestId id
AvailabilityTestName name
AvailabilityTimestamp timestamp
Browser client_browser
City client_city
ClientIP client_IP
Computer cloud_RoleInstance
Country client_CountryOrRegion
LOG ANALYTICS WORKSPACE PROPERTIES APPLICATION INSIGHTS RESOURCE PROPERTIES
CustomEventCount itemCount
CustomEventDimensions customDimensions
CustomEventName name
DeviceModel client_Model
DeviceType client_Type
ExceptionCount itemCount
ExceptionHandledAt handledAt
ExceptionMessage message
ExceptionType type
OperationID operation_id
OperationName operation_Name
OS client_OS
PageViewCount itemCount
PageViewDuration duration
PageViewName name
ParentOperationID operation_Id
RequestCount itemCount
RequestDuration duration
RequestID id
RequestName name
RequestSuccess success
ResponseCode resultCode
Role cloud_RoleName
RoleInstance cloud_RoleInstance
SessionId session_Id
LOG ANALYTICS WORKSPACE PROPERTIES APPLICATION INSIGHTS RESOURCE PROPERTIES
SourceSystem operation_SyntheticSource
TelemetryTYpe type
URL url
UserAccountId user_AccountId
Next steps
Use Log Search to view detailed information for your Application Insights apps.
Azure Monitor log query examples
11/5/2019 • 9 minutes to read • Edit Online
This article includes various examples of queries using the Kusto query language to retrieve different types of log
data from Azure Monitor. Different methods are used to consolidate and analyze data, so you can use these
samples to identify different strategies that you might use for your own requirements.
See the Kusto language reference for details on the different keywords used in these samples. Go through a lesson
on creating queries if you're new to Azure Monitor.
Events
Search application-level events described as "Cryptographic"
This example searches the Events table for records in which EventLog is Application and RenderedDescription
contains cryptographic. Includes records from the last 24 hours.
Event
| where EventLog == "Application"
| where TimeGenerated > ago(24h)
| where RenderedDescription contains "cryptographic"
Heartbeat
Chart a week-over-week view of the number of computers sending data
The following example charts the number of distinct computers that sent heartbeats, each week.
Heartbeat
| where TimeGenerated >= startofweek(ago(21d))
| summarize dcount(Computer) by endofweek(TimeGenerated) | render barchart kind=default
Heartbeat
| where TimeGenerated > ago(1d)
| summarize LastHeartbeat = max(TimeGenerated) by Computer
| where isnotempty(Computer)
| where LastHeartbeat < ago(1h)
let start_time=startofday(datetime("2018-03-01"));
let end_time=now();
Heartbeat
| where TimeGenerated > start_time and TimeGenerated < end_time
| summarize heartbeat_per_hour=count() by bin_at(TimeGenerated, 1h, start_time), Computer
| extend available_per_hour=iff(heartbeat_per_hour>0, true, false)
| summarize total_available_hours=countif(available_per_hour==true) by Computer
| extend total_number_of_buckets=round((end_time-start_time)/1h)+1
| extend availability_rate=total_available_hours*100/total_number_of_buckets
union withsource=sourceTable *
| where TimeGenerated > ago(5h)
| summarize count() by bin(TimeGenerated,10m), sourceTable
| render timechart
search *
| where TimeGenerated > ago(1h)
| summarize CountOfRecords = count() by Type
| render barchart
AzureDiagnostics
Count Azure diagnostics records per category
This example counts all Azure diagnostics records for each unique category.
AzureDiagnostics
| where TimeGenerated > ago(1d)
| summarize count() by Category
AzureDiagnostics
| where TimeGenerated > ago(1d)
| summarize any(*) by Category
AzureDiagnostics
| where TimeGenerated > ago(1d)
| summarize arg_max(TimeGenerated, *) by Category
Network monitoring
Computers with unhealthy latency
This example creates a list of distinct computers with unhealthy latency.
NetworkMonitoring
| where LatencyHealthState <> "Healthy"
| where Computer != ""
| distinct Computer
Performance
Join computer perf records to correlate memory and CPU
This example correlates a particular computer's perf records and creates two time charts, the average CPU and
maximum memory.
Protection status
Computers with non-reporting protection status duration
This example lists computers that had a protection status of Not Reporting and the duration they were in this
status.
ProtectionStatus
| where ProtectionStatus == "Not Reporting"
| summarize count(), startNotReporting = min(TimeGenerated), endNotReporting = max(TimeGenerated) by Computer,
ProtectionStatusDetails
| join ProtectionStatus on Computer
| summarize lastReporting = max(TimeGenerated), startNotReporting = any(startNotReporting), endNotReporting =
any(endNotReporting) by Computer
| extend durationNotReporting = endNotReporting - startNotReporting
Security records
Count security events by activity ID
This example relies on the fixed structure of the Activity column: <ID>-<Name>. It parses the Activity value into
two new columns, and counts the occurrence of each activityID.
SecurityEvent
| where TimeGenerated > ago(30m)
| project Activity
| parse Activity with activityID " - " activityDesc
| summarize count() by activityID
SecurityEvent
| where TimeGenerated > ago(30m)
| summarize EventCount = countif(Activity has "Permissions")
Find accounts that failed to log in from computers with a security detection
This example finds and counts accounts that failed to log in from computers on which we identify a security
detection.
SecurityEvent
| where TimeGenerated > ago(30m)
| count
SecurityEvent
| take 100
| project Activity
| parse Activity with activityID " - " activityDesc
This example uses the split operator to create an array of separate values
SecurityEvent
| take 100
| project Activity
| extend activityArr=split(Activity, " - ")
| project Activity , activityArr, activityId=activityArr[0]
SecurityEvent
| where TimeGenerated > ago(7d)
// filter by id 4648 ("A logon was attempted using explicit credentials")
| where EventID == 4648
| summarize count() by Process
| render piechart
Find repeating failed login attempts by the same account from different IPs
The following example finds failed login attempts by the same account from more than five different IPs in the last
six hours.
SecurityEvent
| where AccountType == "User" and EventID == 4625 and TimeGenerated > ago(6h)
| summarize IPCount = dcount(IpAddress), makeset(IpAddress) by Account
| where IPCount > 5
| sort by IPCount desc
Using join, and let statements we can check if the same suspicious accounts were later able to log in successfully.
let timeframe = 1d;
let suspicious_users =
SecurityEvent
| where TimeGenerated > ago(timeframe)
| where AccountType == 'User' and EventID == 4625 // 4625 - failed login
| summarize failed_login_attempts=count(), latest_failed_login=arg_max(TimeGenerated, Account) by Account
| where failed_login_attempts > 5
| project-away Account1;
let suspicious_users_that_later_logged_in =
suspicious_users
| join kind=innerunique (
SecurityEvent
| where TimeGenerated > ago(timeframe)
| where AccountType == 'User' and EventID == 4624 // 4624 - successful login,
| summarize latest_successful_login=arg_max(TimeGenerated, Account) by Account
) on Account
| extend was_login_after_failures = iif(latest_successful_login>latest_failed_login, 1, 0)
| where was_login_after_failures == 1
;
suspicious_users_that_later_logged_in
Usage
Calculate the average size of perf usage reports per computer
This example calculates the average size of perf usage reports per computer, over the last 3 hours. The results are
shown in a bar chart.
Usage
| where TimeGenerated > ago(3h)
| where DataType == "Perf"
| where QuantityUnit == "MBytes"
| summarize avg(Quantity) by Computer
| sort by avg_Quantity desc nulls last
| render barchart
Usage
| where TimeGenerated > ago(24h)
| summarize percentiles(AvgLatencyInSeconds, 50, 95) by bin(TimeGenerated, 1h)
| render timechart
Usage
| where TimeGenerated > ago(1d)
| where Computer contains "ContosoFile"
| sort by TimeGenerated desc nulls last
Updates
Computers Still Missing Updates
This example shows a list of computers that were missing one or more critical updates a few days ago and are still
missing updates.
Next steps
Refer to the Kusto language reference for details on the language.
Walk through a lesson on writing log queries in Azure Monitor.
Log Analytics smart analytics examples
12/13/2019 • 7 minutes to read • Edit Online
This article includes examples that use smart analytics functions in Log Analytics to perform analysis of user
activity. You can either use these examples to analyze your own applications monitored by Application Insights or
use the concepts in these queries for similar analysis on other data.
See the Kusto language reference for details on the different keywords used in these samples. Go through a lesson
on creating queries if you're new to Log Analytics.
Cohorts analytics
Cohort analysis tracks the activity of specific groups of users, known as cohorts. It attempts to measure how
appealing a service is by measuring the rate of returning users. The users are grouped by the time they first used
the service. When analyzing cohorts, we expect to find a decrease in activity over the first tracked periods. Each
cohort is titled by the week its members were observed for the first time.
The following example analyzes the number of activities users perform over the course of 5 weeks, following their
first use of the service.
let startDate = startofweek(bin(datetime(2017-01-20T00:00:00Z), 1d));
let week = range Cohort from startDate to datetime(2017-03-01T00:00:00Z) step 7d;
// For each user we find the first and last timestamp of activity
let FirstAndLastUserActivity = (end:datetime)
{
customEvents
| where customDimensions["sourceapp"]=="ai-loganalyticsui-prod"
// Check 30 days back to see first time activity
| where timestamp > startDate - 30d
| where timestamp < end
| summarize min=min(timestamp), max=max(timestamp) by user_AuthenticatedId
};
let DistinctUsers = (cohortPeriod:datetime, evaluatePeriod:datetime) {
toscalar (
FirstAndLastUserActivity(evaluatePeriod)
// Find members of the cohort: only users that were observed in this period for the first time
| where min >= cohortPeriod and min < cohortPeriod + 7d
// Pick only the members that were active during the evaluated period or after
| where max > evaluatePeriod - 7d
| summarize dcount(user_AuthenticatedId))
};
week
| where Cohort == startDate
// Finally, calculate the desired metric for each cohort. In this sample we calculate distinct users but you
can change
// this to any other metric that would measure the engagement of the cohort members.
| extend
r0 = DistinctUsers(startDate, startDate+7d),
r1 = DistinctUsers(startDate, startDate+14d),
r2 = DistinctUsers(startDate, startDate+21d),
r3 = DistinctUsers(startDate, startDate+28d),
r4 = DistinctUsers(startDate, startDate+35d)
| union (week | where Cohort == startDate + 7d
| extend
r0 = DistinctUsers(startDate+7d, startDate+14d),
r1 = DistinctUsers(startDate+7d, startDate+21d),
r2 = DistinctUsers(startDate+7d, startDate+28d),
r3 = DistinctUsers(startDate+7d, startDate+35d) )
| union (week | where Cohort == startDate + 14d
| extend
r0 = DistinctUsers(startDate+14d, startDate+21d),
r1 = DistinctUsers(startDate+14d, startDate+28d),
r2 = DistinctUsers(startDate+14d, startDate+35d) )
| union (week | where Cohort == startDate + 21d
| extend
r0 = DistinctUsers(startDate+21d, startDate+28d),
r1 = DistinctUsers(startDate+21d, startDate+35d) )
| union (week | where Cohort == startDate + 28d
| extend
r0 = DistinctUsers (startDate+28d, startDate+35d) )
// Calculate the retention percentage for each cohort by weeks
| project Cohort, r0, r1, r2, r3, r4,
p0 = r0/r0*100,
p1 = todouble(r1)/todouble (r0)*100,
p2 = todouble(r2)/todouble(r0)*100,
p3 = todouble(r3)/todouble(r0)*100,
p4 = todouble(r4)/todouble(r0)*100
| sort by Cohort asc
Next steps
Refer to the Data Explorer language reference for details on the language.
Walk through a lesson on writing queries in Log Analytics.
Writing efficient log queries in Azure Monitor
12/13/2019 • 2 minutes to read • Edit Online
This article provides recommendations for writing efficient log queries in Azure Monitor. Using these strategies,
you can ensure that your queries will run quickly and with minimal overheard.
requests | ...
You can use search to search for a value across multiple columns in specific tables using a query like the following:
Filter records
To review only logs that match a given condition, use the where operator as in the following query that returns only
records in which the severityLevel value is higher than 0:
traces | where severityLevel > 0
String comparisons
When evaluating strings, you should usually use has instead of contains when looking for full tokens. has is
more efficient since it doesn't have to look-up for substrings.
Returned columns
Use project to narrow the set of columns being processed to only those you need:
traces
| project timestamp, severityLevel, client_City
| ...
While you can use extend to calculate values and create your own columns, it will typically be more efficient to filter
on a table column.
For example, the first query below that filters on operation_Name would be more efficient than the second which
creates a new subscription column and filters on it:
customEvents
| where split(operation_Name, "/")[2] == "acb"
customEvents
| extend subscription = split(operation_Name, "/")[2]
| where subscription == "acb"
Using joins
When using the join operator, choose the table with fewer rows to be on the left side of the query.
Next steps
To learn more about query best practices, see Query best practices.
Parse text data in Azure Monitor logs
12/13/2019 • 6 minutes to read • Edit Online
Some log data collected by Azure Monitor will include multiple pieces of information in a single property. Parsing
this data into multiple properties make it easier to use in queries. A common example is a custom log that collects
an entire log entry with multiple values into a single property. By creating separate properties for the different
values, you can search and aggregate on each.
This article describes different options for parsing log data in Azure Monitor when the data is ingested and when
it's retrieved in a query, comparing the relative advantages for each.
Parsing methods
You can parse data either at ingestion time when the data is collected or at query time when analyzing the data
with a query. Each strategy has unique advantages as described below.
Parse data at collection time
When you parse data at collection time, you configure Custom Fields that create new properties in the table.
Queries don't have to include any parsing logic and simply use these properties as any other field in the table.
Advantages to this method include the following:
Easier to query the collected data since you don't need to include parse commands in the query.
Better query performance since the query doesn't need to perform parsing.
Disadvantages to this method include the following:
Must be defined in advance. Can't include data that's already been collected.
If you change the parsing logic, it will only apply to new data.
Fewer parsing options than available in queries.
Increases latency time for collecting data.
Errors can be difficult to handle.
Parse data at query time
When you parse data at query time, you include logic in your query to parse data into multiple fields. The actual
table itself isn't modified.
Advantages to this method include the following:
Applies to any data, including data that's already been collected.
Changes in logic can be applied immediately to all data.
Flexible parsing options including predefined logic for particular data structures.
Disadvantages to this method include the following:
Requires more complex queries. This can be mitigated by using functions to simulate a table.
Must replicate parsing logic in multiple queries. Can share some logic through functions.
Can create overhead when running complex logic against very large record sets (billions of records).
The following query would parse this data into individual properties. The line with project is added to only return
the calculated properties and not RawData, which is the single property holding the entire entry from the custom
log.
MyCustomLog_CL
| parse RawData with * "Time=" EventTime " Event Code=" Code " Status=" Status " Message=" Message
| project EventTime, Code, Status, Message
Following is another example that breaks out the user name of a UPN in the AzureActivity table.
AzureActivity
| parse Caller with UPNUserPart "@" *
| where UPNUserPart != "" //Remove non UPN callers (apps, SPNs, etc)
| distinct UPNUserPart, Caller
Regular expressions
If your data can be identified with a regular expression, you can use functions that use regular expressions to
extract individual values. The following example uses extract to break out the UPN field from AzureActivity
records and then return distinct users.
AzureActivity
| extend UPNUserPart = extract("([a-z.]*)@", 1, Caller)
| distinct UPNUserPart, Caller
To enable efficient parsing at large scale, Azure Monitor uses re2 version of Regular Expressions, which is similar
but not identical to some of the other regular expression variants. Refer to the re2 expression syntax for details.
The following query would parse this data and summarize by two of the calculated properties. The first line splits
the RawData property into a string array. Each of the next lines gives a name to individual properties and adds
them to the output using functions to convert them to the appropriate data type.
MyCustomCSVLog_CL
| extend CSVFields = split(RawData, ',')
| extend EventTime = todatetime(CSVFields[0])
| extend Code = toint(CSVFields[1])
| extend Status = tostring(CSVFields[2])
| extend Message = tostring(CSVFields[3])
| where getyear(EventTime) == 2018
| summarize count() by Status,Code
AzureActivity
| extend parsedProp = parse_json(Properties)
| where parsedProp.isComplianceCheck == "True"
| summarize count() by ResourceGroup, tostring(parsedProp.tags.businessowner)
These parsing functions can be processor intensive, so they should be used only when your query uses multiple
properties from the formatted data. Otherwise, simple pattern matching processing will be faster.
The following example shows the breakdown of domain controller TGT Preauth type. The type exists only in the
EventData field, which is an XML string, but no other data from this field is needed. In this case, parse is used to
pick out the required piece of data.
SecurityEvent
| where EventID == 4768
| parse EventData with * 'PreAuthType">' PreAuthType '</Data>' *
| summarize count() by PreAuthType
MyCustomCSVLog_CL
| extend CSVFields = split(RawData, ',')
| extend DateTime = tostring(CSVFields[0])
| extend Code = toint(CSVFields[1])
| extend Status = tostring(CSVFields[2])
| extend Message = tostring(CSVFields[3])
You can now use the alias MyCustomCSVLog in place of the actual table name in queries like the following.
MyCustomCSVLog
| summarize count() by Status,Code
Next steps
Learn about log queries to analyze the data collected from data sources and solutions.
Automate Azure Monitor log processes with the
connector for Microsoft Flow
12/13/2019 • 2 minutes to read • Edit Online
Microsoft Flow allows you to create automated workflows using hundreds of actions for a variety of services.
Output from one action can be used as input to another allowing you to create integration between different
services. The Azure Log Analytics connector for Microsoft Flow allow you to build workflows that include data
retrieved by log queries from a Log Analytics workspace in Azure Monitor.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a Log
Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the terminology to
better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
For example, you can use Microsoft Flow to use Azure Monitor log data in an email notification from Office 365,
create a bug in Azure DevOps, or post a Slack message. You can trigger a workflow by a simple schedule or from
some action in a connected service such as when a mail or a tweet is received.
The tutorial in this article shows you how to create a flow that automatically sends the results of an Azure Monitor
log query by email, just one example of how you can use the Log Analytics connector in Microsoft Flow.
Event
| where EventLevelName == "Error"
| where TimeGenerated > ago(1day)
| summarize count() by Computer
| sort by Computer
4. When the flow completes, check the mail of the recipient that you specified. You should have received a mail
with a body similar to the following:
Next steps
Learn more about log queries in Azure Monitor.
Learn more about Microsoft Flow.
Automate Azure Application Insights processes with
the connector for Microsoft Flow
12/13/2019 • 3 minutes to read • Edit Online
Do you find yourself repeatedly running the same queries on your telemetry data to check that your service is
functioning properly? Are you looking to automate these queries for finding trends and anomalies and then build
your own workflows around them? The Azure Application Insights connector for Microsoft Flow is the right tool
for these purposes.
With this integration, you can now automate numerous processes without writing a single line of code. After you
create a flow by using an Application Insights action, the flow automatically runs your Application Insights
Analytics query.
You can add additional actions as well. Microsoft Flow makes hundreds of actions available. For example, you can
use Microsoft Flow to automatically send an email notification or create a bug in Azure DevOps. You can also use
one of the many templates that are available for the connector for Microsoft Flow. These templates speed up the
process of creating a flow.
If the connection box does not show up right away and instead goes straight to entering the query, click the ellipses
at the top right of the box. Then select my connections or use an existing one.
Click Create.
Step 5: Specify the Analytics query and chart type
This example query selects the failed requests within the last day and correlates them with exceptions that
occurred as part of the operation. Analytics correlates them based on the operation_Id identifier. The query then
segments the results by using the autocluster algorithm.
When you create your own queries, verify that they are working properly in Analytics before you add it to your
flow.
Add the following Analytics query, and select the HTML table chart type. Then select New step.
requests
| where timestamp > ago(1d)
| where success == "False"
| project name, operation_Id
| join ( exceptions
| project problemId, outerMessage, operation_Id
) on operation_Id
| evaluate autocluster()
Next steps
Learn more about creating Analytics queries.
Learn more about Microsoft Flow.
Automate Application Insights processes by using
Logic Apps
12/16/2019 • 3 minutes to read • Edit Online
Do you find yourself repeatedly running the same queries on your telemetry data to check whether your service is
functioning properly? Are you looking to automate these queries for finding trends and anomalies and then build
your own workflows around them? The Azure Application Insights connector for Logic Apps is the right tool for
this purpose.
With this integration, you can automate numerous processes without writing a single line of code. You can create a
logic app with the Application Insights connector to quickly automate any Application Insights process.
You can add additional actions as well. The Logic Apps feature of Azure App Service makes hundreds of actions
available. For example, by using a logic app, you can automatically send an email notification or create a bug in
Azure DevOps. You can also use one of the many available templates to help speed up the process of creating your
logic app.
requests
| where timestamp > ago(1d)
| where success == "False"
| project name, operation_Id
| join ( exceptions
| project problemId, outerMessage, operation_Id
) on operation_Id
| evaluate autocluster()
When your logic app runs, the recipients you specified in the email list will receive an email that looks like the
following:
Next steps
Learn more about creating Analytics queries.
Learn more about Logic Apps.
Azure Monitoring REST API walkthrough
12/13/2019 • 12 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
This article shows you how to perform authentication so your code can use the Microsoft Azure Monitor REST
API Reference.
The Azure Monitor API makes it possible to programmatically retrieve the available default metric definitions,
granularity, and metric values. The data can be saved in a separate data store such as Azure SQL Database, Azure
Cosmos DB, or Azure Data Lake. From there additional analysis can be performed as needed.
Besides working with various metric data points, the Monitor API also makes it possible to list alert rules, view
activity logs, and much more. For a full list of available operations, see the Microsoft Azure Monitor REST API
Reference.
To query the Azure Monitor API, the client application should use the previously created service principal to
authenticate. The following example PowerShell script shows one approach, using the Active Directory
Authentication Library (ADAL ) to obtain the JWT authentication token. The JWT token is passed as part of an
HTTP Authorization parameter in requests to the Azure Monitor REST API.
$clientId = $azureAdApplication.ApplicationId.Guid
$tenantId = $subscription.TenantId
$authUrl = "https://login.microsoftonline.com/${tenantId}"
$AuthContext = [Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext]$authUrl
$cred = New-Object -TypeName Microsoft.IdentityModel.Clients.ActiveDirectory.ClientCredential -ArgumentList
($clientId, $pwd)
$result = $AuthContext.AcquireTokenAsync("https://management.core.windows.net/",
$cred).GetAwaiter().GetResult()
After authenticating, queries can then be executed against the Azure Monitor REST API. There are two helpful
queries:
1. List the metric definitions for a resource
2. Retrieve the metric values
NOTE
For additional information on authenticating with the Azure REST API, please refer to the Azure REST API Reference.
Retrieve Metric Definitions (Multi-Dimensional API)
Use the Azure Monitor Metric definitions REST API to access the list of metrics that are available for a service.
Method: GET
Request URI:
https://management.azure.com/subscriptions/{subscriptionId }/resourceGroups/{resourceGroupName}/providers/
{resourceProviderNamespace}/{resourceType}/{resourceName}/providers/microsoft.insights/metricDefinitions?
api-version={apiVersion}
For example, to retrieve the metric definitions for an Azure Storage account, the request would appear as follows:
$request = "https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/ContosoStorage/providers/microsoft.insights/metricDefi
nitions?api-version=2018-01-01"
NOTE
To retrieve metric definitions using the multi-dimensional Azure Monitor metrics REST API, use "2018-01-01" as the API
version.
The resulting JSON response body would be similar to the following example: (Note that the second metric has
dimensions)
{
"value": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/ContosoStorage/providers/microsoft.insights/metricdefi
nitions/UsedCapacity",
"resourceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/ContosoStorage",
"namespace": "Microsoft.Storage/storageAccounts",
"category": "Capacity",
"name": {
"value": "UsedCapacity",
"localizedValue": "Used capacity"
},
"isDimensionRequired": false,
"unit": "Bytes",
"primaryAggregationType": "Average",
"supportedAggregationTypes": [
"Total",
"Average",
"Minimum",
"Maximum"
],
"metricAvailabilities": [
{
"timeGrain": "PT1H",
"retention": "P93D"
},
{
{
"timeGrain": "PT6H",
"retention": "P93D"
},
{
"timeGrain": "PT12H",
"retention": "P93D"
},
{
"timeGrain": "P1D",
"retention": "P93D"
}
]
},
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/ContosoStorage/providers/microsoft.insights/metricdefi
nitions/Transactions",
"resourceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/ContosoStorage",
"namespace": "Microsoft.Storage/storageAccounts",
"category": "Transaction",
"name": {
"value": "Transactions",
"localizedValue": "Transactions"
},
"isDimensionRequired": false,
"unit": "Count",
"primaryAggregationType": "Total",
"supportedAggregationTypes": [
"Total"
],
"metricAvailabilities": [
{
"timeGrain": "PT1M",
"retention": "P93D"
},
{
"timeGrain": "PT5M",
"retention": "P93D"
},
{
"timeGrain": "PT15M",
"retention": "P93D"
},
{
"timeGrain": "PT30M",
"retention": "P93D"
},
{
"timeGrain": "PT1H",
"retention": "P93D"
},
{
"timeGrain": "PT6H",
"retention": "P93D"
},
{
"timeGrain": "PT12H",
"retention": "P93D"
},
{
"timeGrain": "P1D",
"retention": "P93D"
}
],
"dimensions": [
{
"value": "ResponseType",
"localizedValue": "Response type"
"localizedValue": "Response type"
},
{
"value": "GeoType",
"localizedValue": "Geo type"
},
{
"value": "ApiName",
"localizedValue": "API name"
}
]
},
...
]
}
NOTE
To retrieve dimension values using the Azure Monitor REST API, use "2018-01-01" as the API version.
Method: GET
Request URI: https://management.azure.com/subscriptions/{subscription-id }/resourceGroups/{resource-group -
name}/providers/{resource-provider-namespace}/{resource-type}/{resource-
name}/providers/microsoft.insights/metrics?
metricnames={metric}×pan={starttime/endtime}&$filter={filter }&resultType=metadata&api-
version={apiVersion}
For example, to retrieve the list of dimension values that were emitted for the 'API Name dimension' for the
'Transactions' metric, where the GeoType dimension = 'Primary' during the specified time range, the request
would be as follows:
The resulting JSON response body would be similar to the following example:
{
"timespan": "2018-03-01T00:00:00Z/2018-03-02T00:00:00Z",
"value": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/ContosoStorage/providers/Microsoft.Insights/metrics/Tr
ansactions",
"type": "Microsoft.Insights/metrics",
"name": {
"value": "Transactions",
"localizedValue": "Transactions"
},
"unit": "Count",
"timeseries": [
{
"metadatavalues": [
{
"name": {
"value": "apiname",
"localizedValue": "apiname"
},
"value": "DeleteBlob"
}
]
},
{
"metadatavalues": [
{
"name": {
"value": "apiname",
"localizedValue": "apiname"
},
"value": "SetBlobProperties"
}
]
},
...
]
}
],
"namespace": "Microsoft.Storage/storageAccounts",
"resourceregion": "eastus"
}
NOTE
To retrieve multi-dimensional metric values using the Azure Monitor REST API, use "2018-01-01" as the API version.
Method: GET
Request URI: https://management.azure.com/subscriptions/*{subscription-id}*/resourceGroups/*{resource-
group-name}*/providers/*{resource-provider-namespace}*/*{resource-type}*/*{resource-
name}*/providers/microsoft.insights/metrics?metricnames=*{metric}*×pan=*
{starttime/endtime}*&$filter=*{filter}*&interval=*{timeGrain}*&aggregation=*{aggreation}*&api-version=*
{apiVersion}*
For example, to retrieve the top 3 APIs, in descending value, by the number of 'Transactions' during a 5 min range,
where the GeotType was 'Primary', the request would be as follows:
The resulting JSON response body would be similar to the following example:
{
"cost": 0,
"timespan": "2018-03-01T02:00:00Z/2018-03-01T02:05:00Z",
"interval": "PT1M",
"value": [
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/ContosoStorage/providers/Microsoft.Insights/metrics/Tr
ansactions",
"type": "Microsoft.Insights/metrics",
"name": {
"value": "Transactions",
"localizedValue": "Transactions"
},
"unit": "Count",
"timeseries": [
{
"metadatavalues": [
{
"name": {
"value": "apiname",
"localizedValue": "apiname"
},
"value": "GetBlobProperties"
}
],
"data": [
{
"timeStamp": "2017-09-19T02:00:00Z",
"total": 2
},
{
"timeStamp": "2017-09-19T02:01:00Z",
"total": 1
},
{
"timeStamp": "2017-09-19T02:02:00Z",
"total": 3
},
{
"timeStamp": "2017-09-19T02:03:00Z",
"total": 7
},
{
"timeStamp": "2017-09-19T02:04:00Z",
"total": 2
}
]
},
...
]
}
],
"namespace": "Microsoft.Storage/storageAccounts",
"resourceregion": "eastus"
}
$request = "https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/microsoft.insights/metricDefinitions?
api-version=2016-03-01"
NOTE
To retrieve metric definitions using the Azure Monitor REST API, use "2016-03-01" as the API version.
The resulting JSON response body would be similar to the following example:
{
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/microsoft.insights/metricdefinitions",
"value": [
{
"name": {
"value": "RunsStarted",
"localizedValue": "Runs Started"
},
"category": "AllMetrics",
"startTime": "0001-01-01T00:00:00Z",
"endTime": "0001-01-01T00:00:00Z",
"unit": "Count",
"primaryAggregationType": "Total",
"resourceUri": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets",
"resourceId": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets",
"metricAvailabilities": [
{
"timeGrain": "PT1M",
"retention": "P30D",
"location": null,
"blobLocation": null
},
{
"timeGrain": "PT1H",
"retention": "P30D",
"location": null,
"blobLocation": null
}
],
"properties": null,
"dimensions": null,
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/microsoft.insights/metricdefinitions/R
unsStarted",
"supportedAggregationTypes": [ "None", "Average", "Minimum", "Maximum", "Total", "Count" ]
}
]
}
For more information, see the List the metric definitions for a resource in Azure Monitor REST API
documentation.
NOTE
To retrieve metric values using the Azure Monitor REST API, use "2016-09-01" as the API version.
Method: GET
Request URI: https://management.azure.com/subscriptions/*{subscription-id}*/resourceGroups/*{resource-
group-name}*/providers/*{resource-provider-namespace}*/*{resource-type}*/*{resource-
name}*/providers/microsoft.insights/metrics?$filter=*{filter}*&api-version=*{apiVersion}*
For example, to retrieve the RunsSucceeded metric data points for the given time range and for a time grain of 1
hour, the request would be as follows:
The resulting JSON response body would be similar to the following example:
{
"value": [
{
"data": [
{
"timeStamp": "2017-08-18T19:00:00Z",
"total": 0.0
},
{
"timeStamp": "2017-08-18T20:00:00Z",
"total": 159.0
},
{
"timeStamp": "2017-08-18T21:00:00Z",
"total": 174.0
},
{
"timeStamp": "2017-08-18T22:00:00Z",
"total": 97.0
}
],
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/Microsoft.Insights/metrics/RunsSucceed
ed",
"name": {
"value": "RunsSucceeded",
"localizedValue": "Runs Succeeded"
},
"type": "Microsoft.Insights/metrics",
"unit": "Count"
}
]
}
To retrieve multiple data or aggregation points, add the metric definition names and aggregation types to the filter,
as seen in the following example:
$filter = "(name.value eq 'ActionsCompleted' or name.value eq 'RunsSucceeded') and (aggregationType eq 'Total'
or aggregationType eq 'Average') and startTime eq 2017-08-18T21:00:00 and endTime eq 2017-08-18T21:30:00 and
timeGrain eq duration'PT1M'"
$request = "https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/microsoft.insights/metrics?
`$filter=${filter}&api-version=2016-09-01"
Invoke-RestMethod -Uri $request `
-Headers $authHeader `
-Method Get `
-OutFile ".\contosotweets-metrics-multiple-results.json" `
-Verbose
The resulting JSON response body would be similar to the following example:
{
"value": [
{
"data": [
{
"timeStamp": "2017-08-18T21:03:00Z",
"total": 5.0,
"average": 1.0
},
{
"timeStamp": "2017-08-18T21:04:00Z",
"total": 7.0,
"average": 1.0
}
],
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/Microsoft.Insights/metrics/ActionsComp
leted",
"name": {
"value": "ActionsCompleted",
"localizedValue": "Actions Completed "
},
"type": "Microsoft.Insights/metrics",
"unit": "Count"
},
{
"data": [
{
"timeStamp": "2017-08-18T21:03:00Z",
"total": 5.0,
"average": 1.0
},
{
"timeStamp": "2017-08-18T21:04:00Z",
"total": 7.0,
"average": 1.0
}
],
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/Microsoft.Insights/metrics/RunsSucceed
ed",
"name": {
"value": "RunsSucceeded",
"localizedValue": "Runs Succeeded"
},
"type": "Microsoft.Insights/metrics",
"unit": "Count"
}
]
}
Use ARMClient
An additional approach is to use ARMClient on your Windows machine. ARMClient handles the Azure AD
authentication (and resulting JWT token) automatically. The following steps outline the use of ARMClient for
retrieving metric data:
1. Install Chocolatey and ARMClient.
2. In a terminal window, type armclient.exe login. Doing so prompts you to log in to Azure.
3. Type armclient GET [your_resource_id ]/providers/microsoft.insights/metricdefinitions?api-version=2016 -03 -
01
4. Type armclient GET [your_resource_id ]/providers/microsoft.insights/metrics?api-version=2016 -09 -01
For example, in order to retrieve the metric definitions for a specific Logic App, issue the following command:
armclient GET /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Logic/workflows/ContosoTweets/providers/microsoft.insights/metricDefinitions?
api-version=2016-03-01
Azure PowerShell
The resource ID can be retrieved using Azure PowerShell cmdlets as well. For example, to obtain the resource ID
for an Azure Logic App, execute the Get-AzureLogicApp cmdlet, as in the following example:
Azure CLI
To retrieve the resource ID for an Azure Storage account using the Azure CLI, execute the
az storage account show command, as shown in the following example:
{
"accessTier": null,
"creationTime": "2017-08-18T19:58:41.840552+00:00",
"customDomain": null,
"enableHttpsTrafficOnly": false,
"encryption": null,
"id": "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/azmon-rest-api-
walkthrough/providers/Microsoft.Storage/storageAccounts/contosotweets2017",
"identity": null,
"kind": "Storage",
"lastGeoFailoverTime": null,
"location": "centralus",
"name": "contosotweets2017",
"networkAcls": null,
"primaryEndpoints": {
"blob": "https://contosotweets2017.blob.core.windows.net/",
"file": "https://contosotweets2017.file.core.windows.net/",
"queue": "https://contosotweets2017.queue.core.windows.net/",
"table": "https://contosotweets2017.table.core.windows.net/"
},
"primaryLocation": "centralus",
"provisioningState": "Succeeded",
"resourceGroup": "azmon-rest-api-walkthrough",
"secondaryEndpoints": null,
"secondaryLocation": "eastus2",
"sku": {
"name": "Standard_GRS",
"tier": "Standard"
},
"statusOfPrimary": "available",
"statusOfSecondary": "available",
"tags": {},
"type": "Microsoft.Storage/storageAccounts"
}
NOTE
Azure Logic Apps are not yet available via the Azure CLI, thus an Azure Storage account is shown in the preceding example.
$apiVersion = "2015-04-01"
$filter = "eventTimestamp ge '2017-08-18' and eventTimestamp le '2017-08-19'and eventChannels eq 'Admin,
Operation'"
$request = "https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/providers/microsoft.insights/eventtypes/management/values?api-
version=${apiVersion}&`$filter=${filter}"
Invoke-RestMethod -Uri $request `
-Headers $authHeader `
-Method Get `
-Verbose
Next steps
Review the Overview of Monitoring.
View the Supported metrics with Azure Monitor.
Review the Microsoft Azure Monitor REST API Reference.
Review the Azure Management Library.
Create custom views by using View Designer in
Azure Monitor
12/23/2019 • 4 minutes to read • Edit Online
By using View Designer in Azure Monitor, you can create a variety of custom views in the Azure portal that can
help you visualize data in your Log Analytics workspace. This article presents an overview of View Designer and
procedures for creating and editing custom views.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a
Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the
terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Concepts
Views are displayed in the Azure Monitor Overview page in the Azure portal. Open this page from the Azure
Monitor menu by clicking More under the Insights section. The tiles in each custom view are displayed
alphabetically, and the tiles for the monitoring solutions are installed the same workspace.
The views that you create with View Designer contain the elements that are described in the following table:
PART DESCRIPTION
PART DESCRIPTION
Custom view Displayed when you select a tile. Each view contains one or
more visualization parts.
Required permissions
You require at least contributor level permissions in the Log Analytics workspace to create or modify views. If you
don't have this permission, then the View Designer option won't be displayed in the menu.
OPTION DESCRIPTION
Logs Opens the Log Analytics to analyze data with log queries.
Edit Opens the view in View Designer to edit its contents and
configuration.
Clone Creates a new view and opens it in View Designer. The name
of the new view is the same as the original name, but with
Copy appended to it.
Date range Set the date and time range filter for the data that's included
in the view. This date range is applied before any date ranges
set in queries in the view.
OPTION DESCRIPTION
Import Imports the omsview file that you exported from another
workspace. This action overwrites the configuration of the
existing view.
Clone Creates a new view and opens it in View Designer. The name
of the new view is the same as the original name, but with
Copy appended to it.
Next steps
Add Tiles to your custom view.
Add Visualization parts to your custom view.
Reference guide to View Designer tiles in Azure
Monitor
10/25/2019 • 10 minutes to read • Edit Online
By using View Designer in Azure Monitor, you can create a variety of custom views in the Azure portal that can
help you visualize data in your Log Analytics workspace. This article is a reference guide to the settings for the
tiles that are available in your custom views.
For more information about View Designer, see:
View Designer: Provides an overview of View Designer and procedures for creating and editing custom views.
Visualization part reference: Provides a reference guide to the settings for the visualization parts that are
available in your custom views.
The available View Designer tiles are described in the following table:
TILE DESCRIPTION
Line chart and callout A line chart that's based on a query, and a callout with a
summary value.
Two timelines A column chart with two series, each based on a separate
query.
The next sections describe the tile types and their properties in detail.
NOTE
Tiles in views are based on log queries in your Log Analytics workspace. They do not currently support cross resource
queries to retrieve data from Application Insights.
Number tile
The Number tile displays both the count of records from a log query and a label.
SETTING DESCRIPTION
Tile
Query The query that's run. The count of the records that are
returned by the query is displayed.
SETTING DESCRIPTION
First Tile
Query The query that's run. The count of the records that are
returned by the query is displayed.
Second Tile
SETTING DESCRIPTION
Query The query that's run. The count of the records that are
returned by the query is displayed.
Donut tile
The Donut tile displays a single number that summarizes a value column in a log query. The donut graphically
displays results of the top three records.
SETTING DESCRIPTION
Donut
Query The query that's run for the donut. The first property is a text
value, and the second property is a numeric value. This query
ordinarily uses the measure keyword to summarize results.
Text The text that's displayed under the value inside the donut.
SETTING DESCRIPTION
Result values used in center operation Optionally, select the plus sign (+) to add one or more values.
The results of the query are limited to records with the
property values you specify. If no values are added, all records
are included in the query.
Colors The color that's displayed for each of the three top properties.
To specify alternate colors for specific property values, use
Advanced Color Mapping.
Advanced Color Mapping Displays a color that represents specific property values. If the
value you specify is in the top three, the alternate color is
displayed instead of the standard color. If the property is not
in the top three, the color is not displayed.
Line chart
Query The query that's run for the line chart. The first property is a
text value, and the second property is a numeric value. This
query ordinarily uses the measure keyword to summarize
results. If the query uses the interval keyword, the x-axis uses
this time interval. If the query doesn't use the interval
keyword, the x-axis uses hourly intervals.
Use Logarithmic Scale Select this link to use a logarithmic scale for the y-axis.
Units Specify the units for the values returned by the query. This
information is used to display labels on the chart indicating
the value types and optionally for converting the values. The
Unit Type specifies the category of the unit and defines the
Current Unit Type values that are available. If you select a
value in Convert to then the numeric values are converted
from the Current Unit type to the Convert to type.
Custom label The text that's displayed for the y-axis next to the label for the
Unit type. If no label is specified, only the Unit type is
displayed.
Line chart
Query The query that's run for the line chart. The first property is a
text value, and the second property is a numeric value. This
query ordinarily uses the measure keyword to summarize
results. If the query uses the interval keyword, the x-axis uses
this time interval. If the query doesn't use the interval
keyword, the x-axis uses hourly intervals.
Callout title The text that's displayed above the callout value.
Series name The series property value to be used as the callout value. If no
series is provided, all records from the query are used.
Use Logarithmic Scale Select this link to use a logarithmic scale for the y-axis.
Units Specify the units for the values to be returned by the query.
This information is used to display chart labels that indicate
the value types and, optionally, to convert the values. The
Unit type specifies the category of the unit and defines the
available Current Unit type values. If you select a value in
Convert to, the numeric values are converted from the
Current Unit type to the Convert to type.
SETTING DESCRIPTION
Custom label The text that's displayed for the y-axis next to the label for the
Unit type. If no label is specified, only the Unit type is
displayed.
SETTING DESCRIPTION
First Chart
Legend The text that's displayed under the callout for the first series.
Color The color that's used for the columns in the first series.
Chart query The query that's run for the first series. The count of the
records over each time interval is represented by the chart
columns.
SETTING DESCRIPTION
Second chart
Legend The text that's displayed under the callout for the second
series.
Color The color that's used for the columns in the second series.
Chart Query The query that's run for the second series. The count of the
records over each time interval is represented by the chart
columns.
Next steps
Learn about log queries to support the queries in tiles.
Add visualization parts to your custom view.
Reference guide to View Designer visualization parts
in Azure Monitor
10/25/2019 • 22 minutes to read • Edit Online
By using View Designer in Azure Monitor, you can create a variety of custom views in the Azure portal that can
help you visualize data in your Log Analytics workspace. This article is a reference guide to the settings for the
visualization parts that are available in your custom views.
For more information about View Designer, see:
View Designer: Provides an overview of View Designer and procedures for creating and editing custom views.
Tile reference: Provides a reference to the settings for each available tile in your custom views.
The available View Designer tile types are described in the following table:
List of queries Displays a list of log queries. You can select each query to
display its results.
Number and list The header displays a single number that shows a count of
records from a log query. The list displays the top ten results
from a query, with a graph that indicates the relative value of
a numeric column or its change over time.
Two numbers and list The header displays two numbers that show counts of
records from separate log queries. The list displays the top
ten results from a query, with a graph that indicates the
relative value of a numeric column or its change over time.
Donut and list The header displays a single number that summarizes a value
column in a log query. The donut graphically displays results
of the top three records.
Two timelines and list The header displays the results of two log queries over time
as column charts, with a callout that displays a single number
that summarizes a value column in a log query. The list
displays the top ten results from a query, with a graph that
indicates the relative value of a numeric column or its change
over time.
Information The header displays static text and an optional link. The list
displays one or more items with a static title and text.
Line chart, callout, and list The header displays a line chart, with multiple series from a
log query over time and a callout with a summarized value.
The list displays the top ten results from a query, with a
graph that indicates the relative value of a numeric column or
its change over time.
VIEW TYPE DESCRIPTION
Line chart and list The header displays a line chart, with multiple series from a
log query over time. The list displays the top ten results from
a query, with a graph that indicates the relative value of a
numeric column or its change over time.
Stack of line charts part Displays three separate line charts, with multiple series from a
log query over time.
The next sections describe the tile types and their properties in detail.
NOTE
Parts in views are based on log queries in your Log Analytics workspace. They do not currently support cross resource
queries to retrieve data from Application Insights.
SETTING DESCRIPTION
General
New Group Select this link to create a new group in the view, starting at
the current view.
Render mode The initial view that's displayed when the query is selected.
You can select any available views after they open the query.
SETTING DESCRIPTION
Queries
SETTING DESCRIPTION
General
Group title The text that's displayed at the top of the view.
New Group Select this link to create a new group in the view, starting at
the current view.
Icon The image file that's displayed next to the result in the header.
Title
Query The query to run for the header. The count of the records
that are returned by the query is displayed.
Click-through navigation Action taken when you click on the header. For more
information, see Common Settings.
SETTING DESCRIPTION
List
Query The query to run for the list. The first two properties for the
first ten records in the results are displayed. The first property
is a text value, and the second property is a numeric value.
Bars are automatically created that are based on the relative
value of the numeric column.
Hide graph Select this link to disable the graph at the right of the
numeric column.
Name and value separator The single-character delimiter to use to parse the text
property into multiple values. For more information, see
Common Settings.
Click-through navigation Action taken when you click on an item in the list. For more
information, see Common Settings.
Name The text that's displayed at the top of the first column.
Value The text that's displayed at the top of the second column.
Enable Thresholds Select this link to enable thresholds. For more information,
see Common Settings.
General
Group title The text that's displayed at the top of the view.
New Group Select this link to create a new group in the view, starting at
the current view.
Icon The image file that's displayed next to the result in the header.
Title Navigation
Click-through navigation Action taken when you click on the header. For more
information, see Common Settings.
Title
Query The query to run for the header. The count of the records
that are returned by the query is displayed.
List
Query The query to run for the list. The first two properties for the
first ten records in the results are displayed. The first property
is a text value, and the second property is a numeric value.
Bars are automatically created based on the relative value of
the numeric column.
Hide graph Select this link to disable the graph at the right of the
numeric column.
Name and value separator The single-character delimiter to use to parse the text
property into multiple values. For more information, see
Common Settings.
Click-through navigation Action taken when you click on an item in the list. For more
information, see Common Settings.
Name The text that's displayed at the top of the first column.
Value The text that's displayed at the top of the second column.
Enable Thresholds Select this link to enable thresholds. For more information,
see Common Settings.
General
Group title The text that's displayed at the top of the tile.
New Group Select this link to create a new group in the view, starting at
the current view.
Icon The image file that's displayed next to the result in the header.
Header
Subtitle The text that's displayed under the title at the top of the
header.
Donut
Query The query to run for the donut. The first property is a text
value, and the second property is a numeric value.
Click-through navigation Action taken when you click on the header. For more
information, see Common Settings.
Text The text that's displayed under the value inside the donut.
Result values used in center operation Optionally, select the plus sign (+) to add one or more values.
The results of the query are limited to records with the
property values you specify. If no values are added, all records
are included in the query.
Color 1 Select the color for each of the values that are displayed in
Color 2 the donut.
Color 3
List
Query The query to run for the list. The count of the records that
are returned by the query is displayed.
Hide graph Select this link to disable the graph at the right of the
numeric column.
Name and value separator The single-character delimiter to use to parse the text
property into multiple values. For more information, see
Common Settings.
Click-through navigation Action taken when you click on an item in the list. For more
information, see Common Settings.
Name The text that's displayed at the top of the first column.
Value The text that's displayed at the top of the second column.
Enable Thresholds Select this link to enable thresholds. For more information,
see Common Settings.
General
Group title The text that's displayed at the top of the tile.
New Group Select this link to create a new group in the view, starting at
the current view.
Icon The image file that's displayed next to the result in the header.
Title Navigation
Click-through navigation Action taken when you click on the header. For more
information, see Common Settings.
First chart
Second chart
Legend The text that's displayed under the callout for the first series.
Query The query to run for the first series. The count of records over
each time interval is represented by the chart columns.
SETTING DESCRIPTION
List
Query The query to run for the list. The count of the records that
are returned by the query is displayed.
Hide graph Select this link to disable the graph at the right of the
numeric column.
Click-through navigation Action taken when you click on an item in the list. For more
information, see Common Settings.
Name The text that's displayed at the top of the first column.
Value The text that's displayed at the top of the second column.
Enable Thresholds Select this link to enable thresholds. For more information,
see Common Settings.
Information part
The header displays static text and an optional link. The list displays one or more items with a static title and text.
SETTING DESCRIPTION
General
Group title The text that's displayed at the top of the tile.
New Group Select this link to create a new group in the view, starting at
the current view.
Header
Information items
Title The text that's displayed for the title of each item.
General
Group title The text that's displayed at the top of the tile.
New Group Select this link to create a new group in the view, starting at
the current view.
Icon The image file that's displayed next to the result in the header.
Header
Subtitle The text that's displayed under the title at the top of the
header.
Line chart
Query The query to run for the line chart. The first property is a text
value, and the second property is a numeric value. This query
ordinarily uses the measure keyword to summarize results. If
the query uses the interval keyword, the x-axis of the chart
uses this time interval. If the query does not include the
interval keyword, the x-axis uses hourly intervals.
Click-through navigation Action taken when you click on the header. For more
information, see Common Settings.
Callout title The text that's displayed above the callout value.
SETTING DESCRIPTION
Series Name Property value for the series to use for the callout value. If no
series is provided, all records from the query are used.
Use Logarithmic Scale Select this link to use a logarithmic scale for the y-axis.
Units Specify the units for the values to be returned by the query.
This information is used to display chart labels that indicate
the value types and, optionally, to convert the values. The
Unit type specifies the category of the unit and defines the
available Current Unit type values. If you select a value in
Convert to, the numeric values are converted from the
Current Unit type to the Convert to type.
Custom label The text that's displayed for the y-axis next to the label for
the Unit type. If no label is specified, only the Unit type is
displayed.
List
Query The query to run for the list. The count of the records that
are returned by the query is displayed.
Hide graph Select this link to disable the graph at the right of the
numeric column.
Name and value separator The single-character delimiter to use to parse the text
property into multiple values. For more information, see
Common Settings.
Click-through navigation Action taken when you click on an item in the list. For more
information, see Common Settings.
SETTING DESCRIPTION
Name The text that's displayed at the top of the first column.
Value The text that's displayed at the top of the second column.
Enable Thresholds Select this link to enable thresholds. For more information,
see Common Settings.
SETTING DESCRIPTION
General
Group title The text that's displayed at the top of the tile.
New Group Select this link to create a new group in the view, starting at
the current view.
Icon The image file that's displayed next to the result in the header.
Header
Subtitle The text that's displayed under the title at the top of the
header.
Line chart
Query The query to run for the line chart. The first property is a text
value, and the second property is a numeric value. This query
ordinarily uses the measure keyword to summarize results. If
the query uses the interval keyword, the x-axis of the chart
uses this time interval. If the query does not include the
interval keyword, the x-axis uses hourly intervals.
Click-through navigation Action taken when you click on the header. For more
information, see Common Settings.
Use Logarithmic Scale Select this link to use a logarithmic scale for the y-axis.
Units Specify the units for the values to be returned by the query.
This information is used to display chart labels that indicate
the value types and, optionally, to convert the values. The
Unit type specifies the category of the unit and defines the
available Current Unit type values. If you select a value in
Convert to, the numeric values are converted from the
Current Unit type to the Convert to type.
Custom label The text that's displayed for the y-axis next to the label for
the Unit type. If no label is specified, only the Unit type is
displayed.
List
Query The query to run for the list. The count of the records that
are returned by the query is displayed.
Hide graph Select this link to disable the graph at the right of the
numeric column.
Name and value separator The single-character delimiter to use to parse the text
property into multiple values. For more information, see
Common Settings.
Click-through navigation Action taken when you click on an item in the list. For more
information, see Common Settings.
SETTING DESCRIPTION
Name The text that's displayed at the top of the first column.
Value The text that's displayed at the top of the second column.
Enable Thresholds Select this link to enable thresholds. For more information,
see Common Settings.
SETTING DESCRIPTION
General
Group title The text that's displayed at the top of the tile.
New Group Select this link to create a new group in the view, starting at
the current view.
Icon The image file that's displayed next to the result in the header.
Subtitle The text that's displayed under the title at the top of the
chart.
Query The query to run for the line chart. The first property is a text
value, and the second property is a numeric value. This query
ordinarily uses the measure keyword to summarize results. If
the query uses the interval keyword, the x-axis of the chart
uses this time interval. If the query does not include the
interval keyword, the x-axis uses hourly intervals.
Click-through navigation Action taken when you click on the header. For more
information, see Common Settings.
Use Logarithmic Scale Select this link to use a logarithmic scale for the y-axis.
Units Specify the units for the values to be returned by the query.
This information is used to display chart labels that indicate
the value types and, optionally, to convert the values. The
Unit type specifies the category of the unit and defines the
available Current Unit type values. If you select a value in
Convert to, the numeric values are converted from the
Current Unit type to the Convert to type.
Custom label The text that's displayed for the y-axis next to the label for
the Unit type. If no label is specified, only the Unit type is
displayed.
Common settings
The following sections describe settings that are common to several visualization parts.
Name and value separator
The name and value separator is the single-character delimiter to use to parse the text property from a list query
into multiple values. If you specify a delimiter, you can provide names for each field, separated by the same
delimiter in the Name box.
For example, consider a property called Location that included values such as Redmond -Building 41 and
Bellevue-Building 12. You can specify a dash (-) for the name and value separator and City-Building for the name.
This approach parses each value into two properties called City and Building.
Click-Through Navigation
Click-through navigation defines what action will be taken when you click on a header or list item in a view. This
will either open a query in the Log Analytics or launch another view.
The following table describes the settings for click-through navigation.
SETTING DESCRIPTION
Log Search (Auto) Log query to run when you select a header item. This is the
same log query that the item is based on.
Log Search Log query to run when you select an item in a list. Type the
query into the Navigation query box. Use {selected item} to
include the syntax for the item that the user selected. For
example, if the query has a column named Computer and the
navigation query is {selected item}, a query such as
Computer="MyComputer" is run when you select a computer.
If the navigation query is Type=Event {selected item}, the
query Type=Event Computer="MyComputer" is run.
Sparklines
A sparkline is a small line chart that illustrates the value of a list entry over time. For visualization parts with a list,
you can select whether to display a horizontal bar, which indicates the relative value of a numeric column, or a
sparkline, which indicates its value over time.
The following table describes the settings for sparklines:
SETTING DESCRIPTION
Thresholds
By using thresholds, you can display a colored icon next to each item in a list. Thresholds give you a quick visual
indicator of items that exceed a particular value or fall within a particular range. For example, you can display a
green icon for items with an acceptable value, yellow if the value is within a range that indicates a warning, and
red if it exceeds an error value.
When you enable thresholds for a part, you must specify one or more thresholds. If the value of an item is greater
than a threshold value and lower than the next threshold value, the color for that value is used. If the item is
greater than the highest threshold value, another color is used.
Each threshold set has one threshold with a value of Default. This is the color that's set if no other values are
exceeded. You can add or remove thresholds by selecting the Add (+) or Delete (x) button.
The following table describes the settings for thresholds:
SETTING DESCRIPTION
Enable Thresholds Select this link to display a color icon at the left of each value.
The icon indicates the value's health relative to specified
thresholds.
Threshold The value for the threshold. The health color for each list item
is set to the color of the highest threshold value that's
exceeded by the item's value. If no threshold values are
exceeded, a default color is used.
Next steps
Learn about log queries to support the queries in visualization parts.
Filters in Azure Monitor views
10/25/2019 • 2 minutes to read • Edit Online
A filter in an Azure Monitor view allows users to filter the data in the view by the value of a particular property
without modifying the view itself. For example, you could allow users of your view to filter the view for data only
from a particular computer or set of computers. You can create multiple filters on a single view to allow users to
filter by multiple properties. This article describes how to use a filter and add one to a custom view.
Using a filter
Click the date time range at the top of a view to open the drop down where you can change the date time range for
the view.
Click the + to add a filter using custom filters that are defined for the view. Either select a value for the filter from
the dropdown or type in a value. Continue to add filters by clicking the +.
If you remove all of the values for a filter, then that filter will no longer be applied.
Creating a filter
Create a filter from the Filters tab when editing a view. The filter is global for the view and applies to all parts in
the view.
Field Name Name of the field used for filtering. This field must match the
summarize field in Query for Values.
Query for Values Query to run to populate filter dropdown for the user. This
query must use either summarize or distinct to provide
unique values for a particular field, and it must match the
Field Name. You can use sort to sort the values that are
displayed to the user.
Tag Name for the field that's used in queries supporting the filter
and is also displayed to the user.
Examples
The following table includes a few examples of common filters.
For example, if your view has a query that returns events and uses a filter called Computers, you could use the
following query.
If you added another filter called Severity, you could use the following query to use both filters.
Next steps
Learn more about the Visualization Parts you can add to your custom view.
Import Azure Monitor log data into Power BI
12/13/2019 • 3 minutes to read • Edit Online
Power BI is a cloud based business analytics service from Microsoft that provides rich visualizations and reports
for analysis of different sets of data. You can import the results of an Azure Monitor log query into a Power BI
dataset so you can take advantage of its features such as combining data from different sources and sharing
reports on the web and mobile devices.
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored in a
Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating the
terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for details.
Overview
To import data from a Log Analytics workspace in Azure Monitor into Power BI, you create a dataset in Power BI
based on a log query in Azure Monitor. The query is run each time the dataset is refreshed. You can then build
Power BI reports that use data from the dataset. To create the dataset in Power BI, you export your query from
Log Analytics to Power Query (M ) language. You then use this to create a query in Power BI Desktop and then
publish it to Power BI as a dataset. The details for this process are described below.
Export query
Start by creating a log query that returns the data that you want to populate the Power BI dataset. You then
export that query to Power Query (M ) language which can be used by Power BI Desktop.
1. Create the log query in Log Analytics to extract the data for your dataset.
2. Select Export > Power BI Query (M ). This exports the query to a text file called PowerBIQuery.txt.
3. Open the text file and copy its contents.
3. The query runs, and its results are displayed. You may be prompted for credentials to connect to Azure.
4. Type in a descriptive name for the query. The default is Query1. Click Close and Apply to add the dataset
to the report.
Publish to Power BI
When you publish to Power BI, a dataset and a report will be created. If you create a report in Power BI Desktop,
then this will be published with your data. If not, then a blank report will be created. You can modify the report in
Power BI or create a new one based on the dataset.
1. Create a report based on your data. Use Power BI Desktop documentation if you're not familiar with it.
2. When you're ready to send it to Power BI, click Publish.
3. When prompted, select a destination in your Power BI account. Unless you have a specific destination in
mind, use My workspace.
4. When the publishing completes, click Open in Power BI to open Power BI with your new dataset.
Configure scheduled refresh
The dataset created in Power BI will have the same data that you previously saw in Power BI Desktop. You need
to refresh the dataset periodically to run the query again and populate it with the latest data from Azure Monitor.
1. Click on the workspace where you uploaded your report and select the Datasets menu.
2. Select the context menu next to your new dataset and select Settings.
3. Under Data source credentials you should have a message that the credentials are invalid. This is
because you haven't provided credentials yet for the dataset to use when it refreshes its data.
4. Click Edit credentials and specify credentials with access to the Log Analytics workspace in Azure
Monitor. If you require two-factor authentication, select OAuth2 for the Authentication method to be
prompted to login with your credentials.
5. Under Scheduled refresh turn on the option to Keep your data up to date. You can optionally change
the Refresh frequency and one or more specific times to run the refresh.
Next steps
Learn about log searches to build queries that can be exported to Power BI.
Learn more about Power BI to build visualizations based on Azure Monitor log exports.
Feed Power BI from Application Insights
11/6/2019 • 4 minutes to read • Edit Online
Power BI is a suite of business tools that helps you analyze data and share insights. Rich dashboards are
available on every device. You can combine data from many sources, including Analytics queries from Azure
Application Insights.
There are three methods of exporting Application Insights data to Power BI:
Export Analytics queries. This is the preferred method. Write any query you want and export it to Power
BI. You can place this query on a dashboard, along with any other data.
Continuous export and Azure Stream Analytics. This method is useful if you want to store your data for
long periods of time. If you don't have an extended data retention requirement, use the export analytics
query method. Continuous export and Stream Analytics involves more work to set up and additional
storage overhead.
Power BI adapter. The set of charts is predefined, but you can add your own queries from any other
sources.
NOTE
The Power BI adapter is now deprecated. The predefined charts for this solution are populated by static uneditable
queries. You do not have the ability to edit these queries and depending on certain properties of your data it is possible
for the connection to Power BI to be successful, but no data is populated. This is due to exclusion criteria that are set
within the hardcoded query. While this solution may still work for some customers, due to the lack of flexibility of the
adapter the recommended solution is to use the export Analytics query functionality.
5. To allow Power BI to access Azure, you might have to provide credentials. Use Organizational account
to sign in with your Microsoft account.
If you need to verify the credentials, use the Data Source Settings menu command in the query editor.
Be sure to specify the credentials you use for Azure, which might be different from your credentials for
Power BI.
6. Choose a visualization for your query, and select the fields for x-axis, y-axis, and segmenting dimension.
7. Publish your report to your Power BI cloud workspace. From there, you can embed a synchronized
version into other web pages.
8. Refresh the report manually at intervals, or set up a scheduled refresh on the options page.
Export a Funnel
1. Make your Funnel.
2. Select Power BI.
3. In Power BI Desktop, select Get Data > Blank Query. Then, in the query editor, under View, select
Advanced Editor.
Troubleshooting
You might encounter errors pertaining to credentials or the size of the dataset. Here is some information about
what to do about these errors.
Unauthorized (401 or 403)
This can happen if your refresh token has not been updated. Try these steps to ensure you still have access:
1. Sign in to the Azure portal, and make sure you can access the resource.
2. Try to refresh the credentials for the dashboard.
3. Try to clear the cache from your PowerBI Desktop.
If you do have access and refreshing the credentials does not work, please open a support ticket.
Bad Gateway (502)
This is usually caused by an Analytics query that returns too much data. Try using a smaller time range for the
query.
If reducing the dataset coming from the Analytics query doesn't meet your requirements, consider using the
API to pull a larger dataset. Here's how to convert the M -Query export to use the API.
1. Create an API key.
2. Update the Power BI M script that you exported from Analytics by replacing the Azure Resource Manager
URL with the Application Insights API.
Replace https://management.azure.com/subscriptions/...
with, https://api.applicationinsights.io/beta/apps/...
3. Finally, update the credentials to basic, and use your API key.
Existing script
Source = Json.Document(Web.Contents("https://management.azure.com/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/resourcegroups//providers/microsoft.insights/components//api/query?api-version=2014-12-01-
preview",[Query=[#"csl"="requests",#"x-ms-app"="AAPBI"],Timeout=#duration(0,0,4,0)]))
Updated script
Source = Json.Document(Web.Contents("https://api.applicationinsights.io/beta/apps/<APPLICATION_ID>/query?
api-version=2014-12-01-preview",[Query=[#"csl"="requests",#"x-ms-
app"="AAPBI"],Timeout=#duration(0,0,4,0)]))
About sampling
Depending on the amount of data sent by your application, you might want to use the adaptive sampling
feature, which sends only a percentage of your telemetry. The same is true if you have manually set sampling
either in the SDK or on ingestion. Learn more about sampling.
Next steps
Power BI - Learn
Analytics tutorial
Azure Monitor Workbooks
12/5/2019 • 5 minutes to read • Edit Online
Workbooks provide a flexible canvas for data analysis and the creation of rich visual reports within the Azure
portal. They allow you to tap into multiple data sources from across Azure, and combine them into unified
interactive experiences.
Data sources
Workbooks can query data from multiple sources within Azure. Authors of workbooks can transform this data to
provide insights into the availability, performance, usage, and overall health of the underlying components. For
instance, analyzing performance logs from virtual machines to identify high CPU or low memory instances and
displaying the results as a grid in an interactive report.
But the real power of workbooks is the ability to combine data from disparate sources within a single report. This
allows for the creation of composite resource views or joins across resources enabling richer data and insights that
would otherwise be impossible.
Workbooks are currently compatible with the following data sources:
Logs
Metrics
Azure Resource Graph
Alerts (Preview )
Workload Health (Preview )
Azure Resource Health (Preview )
Azure Data Explorer (Preview )
Visualizations
Workbooks provide a rich set of capabilities for visualizing your data. For detailed examples of each visualization
type you can consult the example links below:
Text
Charts
Grids
Tiles
Trees
Graphs
Getting started
To explore the workbooks experience, first navigate to the Azure Monitor service. This can be done by typing
Monitor into the search box in the Azure portal.
Then select Workbooks (preview).
Gallery
This takes you to the workbooks gallery:
Once you have switched to editing mode you will notice a number of Edit boxes appear to the right corresponding
with each individual aspect of your workbook.
If we select the edit button immediately under the grid of request data we can see that this part of our workbook
consists of a Kusto query against data from an Application Insights resource.
Clicking the other Edit buttons on the right will reveal a number of the core components that make up workbooks
like markdown-based text boxes, parameter selection UI elements, and other chart/visualization types.
Exploring the pre-built templates in edit-mode and then modifying them to fit your needs and save your own
custom workbook is an excellent way to start to learn about what is possible with Azure Monitor workbooks.
Pinning Visualizations
Text, query, and metrics steps in a workbook can be pinned by using the pin button on those items while the
workbook is in pin mode, or if the workbook author has enabled settings for that element to make the pin icon
visible.
To access pin mode, click Edit to enter editing mode, and select the blue pin icon in the top bar. An individual pin
icon will then appear above each corresponding workbook part's Edit box on the right-hand side of your screen.
NOTE
The state of the the workbook is saved at the time of the pin, and pinned workbooks on a dashboard will not update if the
underlying workbook is modified. In order to update a pinned workbook part, you will need to delete and re-pin that part.
Next step
Get started learning more about workbooks many rich visualizations options.
Control and share access to your workbook resources.
Azure Monitor workbook visualizations
12/13/2019 • 8 minutes to read • Edit Online
Azure Monitor workbooks support a number of different styles of visualizations to accommodate your reporting
needs. This article provides examples of each type of visualization.
Text
Workbooks allow authors to include text blocks in their workbooks. The text can be human analysis of telemetry,
information to help users interpret your data, section headings, etc.
Text is added through a Markdown control which provides for full formatting control.
TIP
Use this Markdown cheat sheet to learn about different formatting options.
Charts
Workbooks allow monitoring data to be presented as charts. Supported chart types include line, bar, bar
categorical, area, scatter plots, pie and time. Authors can choose to customize the height, width, color palette,
legend, titles, no-data message, etc. of the chart.
Workbooks support charts for both logs and metric data sources.
Adding a log chart
1. Switch the workbook to edit mode by clicking on the Edit toolbar item.
2. Use the Add query link to add a log query control to the workbook.
3. Select the query type as Log, resource type (for example, Application Insights) and the resources to target.
4. Use the Query editor to enter the KQL for your analysis (for example, trend of requests).
5. Set the visualization to one of: Area, Bar, Bar (categorical), Line, Pie, Scatter, or Time.
6. Set other parameters if needed - like time range, visualization, size, color palette and legend.
Query Type The type of query to use Log, Azure Resource Graph, etc.
Resource Type The resource type to target Application Insights, Log Analytics, or
Azure-first
Time Range The time window to view the log chart Last hour, Last 24 hours, etc.
Visualization The visualization to use Area, Bar, Line, Pie, Scatter, Time, bar
categorical
Size The vertical size of the control Small, medium, large or full
Color palette The color palette to use in the chart. Blue, green, red, etc.
Ignored in multi-metric or segmented
mode.
Legend The aggregation function to use for the Sum or Average of values or Max, Min,
legend First, Last value
Query Any KQL query that returns data in the requests | make-series Requests =
format expected by the chart count () default = 0 on timestamp from
visualization ago (1d) to now() step 1h
Time Range The time window to view the metric in Last hour, Last 24 hours, etc.
Color palette The color palette to use in the chart. Blue, green, red, etc.
Ignored if the Split by parameter is
used
Grids
Grids or tables are a common way to present data to users. Workbooks allow users to individually style the
columns of the grid to provide a rich UI for their reports.
The example below shows a grid that combines icons, heatmaps, and spark-bars to present complex information.
The workbook also provides sorting, a search box and a go-to-analytics button.
Adding a log-based grid
1. Switch the workbook to edit mode by clicking on the Edit toolbar item.
2. Use the Add query link to add a log query control to the workbook.
3. Select the query type as Log, resource type (for example, Application Insights) and the resources to target.
4. Use the Query editor to enter the KQL for your analysis (for example, VMs with memory below a threshold)
5. Set the visualization to Grid
6. Set other parameters if needed - like time range, size, color palette and legend.
Tiles
Tiles are a very useful way to present summary data in workbooks. The image below shows a common use case
of tiles - app level summary on top of a detailed grid.
Workbook tiles support showing a title, subtitle, large text, icons, metric based gradients, spark line/bars, footer,
etc.
Adding a Tile
1. Switch the workbook to edit mode by clicking on the Edit toolbar item.
2. Use the Add query link to add a log query control to the workbook.
3. Select the query type as Log, resource type (for example, Application Insights) and the resources to target.
4. Use the Query editor to enter the KQL for your analysis
requests
| summarize Requests = count() by appName, name
| top 7 by Requests desc
Trees
Workbooks support hierarchical views via tree-grids. Trees allow some rows to be expandable into the next level
for a drill-down experience.
The example below shows container health metrics (working set size) visualized as a tree grid. The top-level
nodes here are Azure Kubernetes Service (AKS ) nodes, the next level are pods and the final level are containers.
Notice that you can still format your columns like in a grid (heatmap, icons, link). The underlying data source in
this case is an Log Analytics workspace with AKS logs.
Tree Settings
SETTING EXPLANATION
Show the expander on The column on which to show the tree expander. It is
common for tree grids to hide their id and parent id field
because they are not very readable. Instead, the expander
appears on a field with a more readable value - like the name
of the entity
Expand the top level of the tree If checked, the tree grid will be expanded at the top level.
Useful if you want to show more information by default
Graphs
Workbooks support visualizing arbitrary graphs based on data from logs to show the relationships between
monitoring entities.
The graph below show data flowing in/out of a computer via various port to/from external computers. It is
colored by type (computer vs. port vs. external IP ) and the edge sizes correspond to the amount of data flowing
in-between. The underlying data comes from KQL query targeting VM connections.
Adding a Graph
1. Switch the workbook to edit mode by clicking on the Edit toolbar item.
2. Use the Add query link to add a log query control to the workbook.
3. Select the query type as Log, resource type (for example, Application Insights) and the resources to target.
4. Use the Query editor to enter the KQL for your analysis
let data = dependencies
| summarize Calls = count() by App = appName, Request = operation_Name, Dependency = name
| extend RequestId = strcat(App, '::', Request);
let links = data
| summarize Calls = sum(Calls) by App, RequestId
| project SourceId = App, TargetId = RequestId, Calls, Kind = 'App -> Request'
| union (data
| project SourceId = RequestId, TargetId = Dependency, Calls, Kind = 'Request -> Dependency');
let nodes = data
| summarize Calls = sum(Calls) by App
| project Id = App, Name = App, Calls, Kind = 'App'
| union (data
| summarize Calls = sum(Calls) by RequestId, Request
| project Id = RequestId, Name = Request, Calls, Kind = 'Request')
| union (data
| summarize Calls = sum(Calls) by Dependency
| project Id = Dependency, Name = Dependency, Calls, Kind = 'Dependency');
nodes
| union (links)
Workbooks are compatible with a large number of data sources. This article will walk you through data sources
which are currently available for Azure Monitor workbooks.
Logs
Workbooks allow querying logs from the following sources:
Azure Monitor Logs (Application Insights Resources and Log Analytics Workspaces.)
Resource-centric data (Activity logs)
Workbook authors can use KQL queries that transform the underlying resource data to select a result set that can
visualized as text, charts, or grids.
Workbook authors can easily query across multiple resources creating a truly unified rich reporting experience.
Metrics
Azure resources emit metrics that can be accessed via workbooks. Metrics can be accessed in workbooks through a
specialized control that allows you to specify the target resources, the desired metrics, and their aggregation. This
data can then be plotted in charts or grids.
Azure Resource Graph
Workbooks support querying for resources and their metadata using Azure Resource Graph (ARG ). This
functionality is primarily used to build custom query scopes for reports. The resource scope is expressed via a
KQL -subset that ARG supports – which is often sufficient for common use cases.
To make a query control use this data source, use the Query type drop-down to choose Azure Resource Graph and
select the subscriptions to target. Use the Query control to add the ARG KQL -subset that selects an interesting
resource subset.
Alerts (preview)
Workbooks allow users to visualize the active alerts related to their resources. This feature allows the creation of
reports that bring together notification data (alert) and diagnostic information (metrics, logs) into one report. This
information can also be joined together to create rich reports that combine insights across these data sources.
To make a query control use this data source, use the Query type drop-down to choose Alerts and select the
subscriptions, resource groups or resources to target. Use the alert filter drop downs to select an interesting subset
of alerts for your analytic needs.
Next steps
Get started learning more about workbooks many rich visualizations options.
Control and share access to your workbook resources.
Programmatically manage workbooks
12/5/2019 • 3 minutes to read • Edit Online
Resource owners have the option to create and manage their workbooks programmatically via Resource Manager
templates.
This can be useful in scenarios like:
Deploying org- or domain-specific analytics reports along with resources deployments. For instance, you may
deploy org-specific performance and failure workbooks for your new apps or virtual machines.
Deploying standard reports or dashboards using workbooks for existing resources.
The workbook will be created in the desired sub/resource-group and with the content specified in the Resource
Manager templates.
Template parameters
PARAMETER EXPLANATION
workbookDisplayName The friendly name for the workbook that is used in the Gallery
or Saved List. Needs to be unique in the scope of the resource
group and source
workbookType The gallery that the workbook will be shown under. Supported
values include workbook, tsg , Azure Monitor, etc.
workbookId The unique guid for this workbook instance. Use [newGuid()]
to automatically create a new guid.
location The Azure location where the workbook will be created. Use
[resourceGroup().location] to create it in the same location as
the resource group
Workbook types
Workbook types specify which workbook gallery type the new workbook instance will show up under. Options
include:
Limitations
For a technical reason, this mechanism cannot be used to create workbook instances in the Workbooks gallery of
Application Insights. We are working on addressing this limitation. In the meanwhile, we recommend that you use
the Troubleshooting Guide gallery (workbookType: tsg ) to deploy Application Insights related workbooks.
Next steps
Explore how workbooks are being used to power the new Azure Monitor for Storage experience.
Access control
12/5/2019 • 2 minutes to read • Edit Online
Next steps
Get started learning more about workbooks many rich visualizations options.
Workbook parameters
12/5/2019 • 2 minutes to read • Edit Online
Parameters allow workbook authors to collect input from the consumers and reference it in other parts of the
workbook – usually to scope the result set or setting the right visual. It is a key capability that allows authors to
build interactive reports and experiences.
Workbooks allow you to control how your parameter controls are presented to consumers – text box vs. drop
down, single- vs. multi-select, values from text, JSON, KQL, or Azure Resource Graph, etc.
Supported parameter types include:
Time - allows a user to select from prepopulated time ranges or select a custom range
Drop down - allows a user to select from a value or set of values
Text - allows a user to enter arbitrary text
Resource - allows a user to select one or more Azure resources
Subscription - allows a user to select one or more Azure subscription resources
Resource Type - allows a user to select one or more Azure resource type values
Location - allows a user to select one or more Azure location values
These parameter values can be referenced in other parts of workbooks either via bindings or value expansions.
Creating a parameter
1. Start with an empty workbook in edit mode.
2. Choose Add parameters from the links within the workbook.
3. Click on the blue Add Parameter button.
4. In the new parameter pane that pops up enter:
a.Parameter name: TimeRange (note that parameter names cannot include spaces or special characters)
b.Display name: Time Range (however, display names can include spaces, special characters, emoji, etc.)
c.Parameter type: Time range picker
d.Required: checked
e.Available time ranges: Last hour, Last 12 hours, Last 24 hours, Last 48 hours, Last 3 days, Last 7 days
and Allow custom time range selection
5. Choose 'Save' from the toolbar to create the parameter.
This is how the workbook will look like in read-mode, in the "Pills" style.
Referencing a parameter
Via Bindings
1. Add a query control to the workbook and select an Application Insights resource.
2. Open the Time Range drop down and select the Time Range option from the Parameters section at the
bottom.
3. This binds the time range parameter to the time range of the chart. The time scope of the sample query is
now Last 24 hours.
4. Run query to see the results
In KQL
1. Add a query control to the workbook and select an Application Insights resource.
2. In the KQL, enter a time scope filter using the parameter: | where timestamp {TimeRange}
3. This expands on query evaluation time to | where timestamp > ago(1d) , which is the time range value of the
parameter.
4. Run query to see the results
In Text
1. Add a text control to the workbook.
2. In the markdown, enter The chosen time range is {TimeRange:label}
3. Choose Done Editing
4. The text control will show text: The chosen time range is Last 24 hours
Parameter options
The In Text section used the label of the parameter instead of its value. Parameters expose various such options
depending on its type - e.g. time range pickers allow value, label, query, start, end, and grain.
Use the Previews section of the Edit Parameter pane to see the expansion options for your parameter:
Next steps
Get started learning more about workbooks many rich visualizations options.
Control and share access to your workbook resources.
Workbook drop down parameters
12/5/2019 • 3 minutes to read • Edit Online
Drop downs allow user to collect one or more input values from a known set (for example, select one of your app’s
requests). Drop downs provide a user-friendly way to collect arbitrary inputs from users. Drop downs are
especially useful in enabling filtering in your interactive reports.
The easiest way to specify a drop-down is by providing a static list in the parameter setting. A more interesting way
is to get the list dynamically via a KQL query. Parameter settings also allow you to specify whether it is single or
multi-select, and if it is multi-select, how the result set should be formatted (delimiter, quotation, etc.).
[
{ "value":"dev", "label":"Development" },
{ "value":"ppe", "label":"Pre-production" },
{ "value":"prod", "label":"Production", "selected":true }
]
[
{ "value":"dev", "label":"Development", "group":"Development" },
{ "value":"dev-cloud", "label":"Development (Cloud)", "group":"Development" },
{ "value":"ppe", "label":"Pre-production", "group":"Test" },
{ "value":"ppe-test", "label":"Pre-production (Test)", "group":"Test" },
{ "value":"prod1", "label":"Prod 1", "selected":true, "group":"Production" },
{ "value":"prod2", "label":"Prod 2", "group":"Production" }
]
requests
| summarize by name
| order by name asc
6. Hit the blue Run Query button.
7. Choose 'Save' from the toolbar to create the parameter.
8. The RequestName parameter will be a drop-down the names of all requests in the app.
requests
| where name == '{RequestName}'
| summarize Requests = count() by bin(timestamp, 1h)
requests
| where name == 'GET Home/Index'
| summarize Requests = count() by bin(timestamp, 1h)
dependencies
| summarize by operation_Name, name
| where name !contains ('.')
| order by name asc
| serialize Rank = row_number()
| project value = name, label = strcat(' ', name), selected = iff(Rank == 1, true, false), group =
operation_Name
![Image showing a drop-down parameter using value, label, selection and group options](./media/workbook-
dropdowns/dropdown-more-options.png)
Multiple selection
The examples so far explicitly set the parameter to select only one value in the drop-down. Drop down parameters
also support multiple selection - enabling this is as simple as checking the Allow multiple selection option.
The user also has the option of specifying the format of the result set via the delimiter and quote with settings.
The default just returns the values as a collection in this form: 'a', 'b', 'c'. They also have the option to limit the
number of selections.
The KQL referencing the parameter will need to change to work with the format of the result. The most common
way to enable it is via the in operator.
dependencies
| where name in ({DependencyName})
| summarize Requests = count() by bin(timestamp, 1h), name
Next steps
Get started learning more about workbooks many rich visualizations options.
Control and share access to your workbook resources.
Interactive Workbooks
12/5/2019 • 5 minutes to read • Edit Online
Workbooks allow authors to create interactive reports and experiences for their consumers. Interactivity is
supported in a number of ways.
Parameter Changes
When a workbook user updates a parameter, any control that uses the parameter automatically refreshes and
redraws to reflect the new state. This is how most of the Azure portal reports support interactivity. Workbooks
provide this in a very straight forward manner with minimal user effort.
Learn more about Parameters in Workbooks
requests
| summarize AllRequests = count(), FailedRequests = countif(success == false) by Request = name
| order by AllRequests desc
requests
| where name == '{SelectedRequest}' or 'All Requests' == '{SelectedRequest}'
| summarize ['{SelectedRequest}'] = count() by bin(timestamp, 1h)
13. Click on a row in the first grid. Note how the area chart below filters to the selected request.
The resulting report looks like this in edit mode:
The image below shows a more elaborate interactive report in read mode based on the same principles. The report
uses grid clicks to export parameters - which in turn is used in two charts and a text block.
Exporting the contents of an entire row
It is sometimes desirable to export the entire contents of the selected row instead of just a particular column. In
such cases, leave the Field to export property unset in step 7.1 above. Workbooks will export the entire row
contents as a json to the parameter.
On the referencing KQL control, use the todynamic function to parse the json and access the individual columns.
requests
| summarize Count = count(), Sample = any(pack_all()) by Request = name
| order by Count desc
Generic Details Shows the row values in a property grid context blade
Cell Details Shows the cell value in a property grid context blade. Useful
when the cell contains a dynamic type with information (for
example, json with request properties like location, role
instance, etc.).
Cell Details Shows the cell value in a property grid context blade. Useful
when the cell contains a dynamic type with information (for
example, json with request properties like location, role
instance, etc.).
Custom Event Details Opens the Application Insights search details with the custom
event ID (itemId) in the cell
Custom Event User Flows Opens the Application Insights User Flows experience pivoted
on the custom event name in the cell
* User Flows Similar to Custom Event User Flows except for exceptions,
page views and requests
User Timeline Opens the user timeline with the user ID (user_Id) in the cell
Session Timeline Opens the Application Insights search experience for the value
in the cell (for example, search for text 'abc' where abc is the
value in the cell)
Resource overview Open the resource's overview in the portal based on the
resource ID value in the cell
Conditional Visibility
Workbook allows users to make certain controls appear or disappear based on values of the parameters. This
allows authors to have reports look different based on user input or telemetry state. An example is showing
consumers just a summary when things are good but show full details when something is wrong.
Setting up interactivity using conditional visibility
1. Follow the steps in the Setting up interactivity on grid row click section to set up two interactive controls.
2. Add a new parameter at the top:
a. Name: ShowDetails
b. Parameter type: Drop down
c. Required: checked
d. Get data from: JSON
e. JSON Input: ["Yes", "No"]
f. Save to commit changes.
3. Set parameter value to Yes
4. In the query control with the area chart, click the Advanced Settings icon (gear icon)
5. Check the setting Make this item conditionally visible
a. This item is visible if ShowDetails parameter value equals Yes
6. Click Done Editing to commit changes.
7. Click Done Editing on the workbook tool bar to enter read mode.
8. Switch the value of parameter ShowDetails to No . Notice that the chart below disappears.
The image below shows the visible case where ShowDetails is Yes
Time parameters allow users to set the time context of analysis and is used by almost all reports. It is relatively
simple to setup and use - allowing authors to specify the time ranges to show in the drop-down, including the
option for custom time ranges.
In KQL
1. Add a query control to the workbook and select an Application Insights resource.
2. In the KQL, enter a time scope filter using the parameter: | where timestamp {TimeRange}
3. This expands on query evaluation time to | where timestamp > ago(1d) , which is the time range value of the
parameter.
4. Run query to see the results
In Text
1. Add a text control to the workbook.
2. In the markdown, enter The chosen time range is {TimeRange:label}
3. Choose Done Editing
4. The text control will show text: The chosen time range is Last 24 hours
requests
| make-series Requests = count() default = 0 on timestamp from {TimeRange:start} to {TimeRange:end} step
{TimeRange:grain}
Next steps
Get started learning more about workbooks many rich visualizations options.
Control and share access to your workbook resources.
Workbook text parameters
12/5/2019 • 2 minutes to read • Edit Online
Textbox parameters provide a simple way to collect text input from workbook users. They are used when it is not
practical to use a drop-down to collect the input (for example, an arbitrary threshold or generic filters). Workbooks
allow authors to get the default value of the textbox from a query. This allows interesting scenarios like setting the
default threshold based on the p95 of the metric.
A common use of textboxes is as internal variables used by other workbook controls. This is done by leveraging a
query for default values, and making the input control invisible in read-mode. For example, a user may want a
threshold to come from a formula (not a user) and then use the threshold in subsequent queries.
requests
| summarize AllRequests = count(), SlowRequests = countif(duration >= {SlowRequestThreshold}) by name
| extend SlowRequestPercent = 100.0 * SlowRequests / AllRequests
| order by SlowRequests desc
3. By using the text parameter with a value of 500 coupled with the query control you effectively running the
query below:
requests
| summarize AllRequests = count(), SlowRequests = countif(duration >= 500) by name
| extend SlowRequestPercent = 100.0 * SlowRequests / AllRequests
| order by SlowRequests desc
requests
| summarize round(percentile(duration, 95), 2)
This query sets the default value of the text box to the 95th percentile duration for all requests in the app.
6. Run query to see the result
7. Choose 'Save' from the toolbar to create the parameter.
NOTE
While this example queries Application Insights data, the approach can be used for any log based data source - Log Analytics,
Azure Resource Graph, etc.
Next steps
Get started learning more about workbooks many rich visualizations options.
Control and share access to your workbook resources.
Create and share dashboards of Log Analytics data
12/13/2019 • 3 minutes to read • Edit Online
Log Analytics dashboards can visualize all of your saved log queries, giving you the ability to find, correlate, and
share IT operational data in the organization. This tutorial covers creating a log query that will be used to
support a shared dashboard that will be accessed by your IT operations support team. You learn how to:
Create a shared dashboard in the Azure portal
Visualize a performance log query
Add a log query to a shared dashboard
Customize a tile in a shared dashboard
To complete the example in this tutorial, you must have an existing virtual machine connected to the Log
Analytics workspace.
Here you can bring together operational data that is most important to IT across all your Azure resources,
including telemetry from Azure Log Analytics. Before we step into visualizing a log query, let's first create a
dashboard and share it. We can then focus on our example performance log query, which will render as a line
chart, and add it to the dashboard.
To create a dashboard, select the New dashboard button next to the current dashboard's name.
This action creates a new, empty, private dashboard and puts you into customization mode where you can name
your dashboard and add or rearrange tiles. Edit the name of the dashboard and specify Sample Dashboard for
this tutorial, and then select Done customizing.
When you create a dashboard, it is private by default, which means you are the only person who can see it. To
make it visible to others, use the Share button that appears alongside the other dashboard commands.
You are asked to choose a subscription and resource group for your dashboard to be published to. For
convenience, the portal's publishing experience guides you towards a pattern where you place dashboards in a
resource group called dashboards. Verify the subscription selected and then click Publish. Access to the
information displayed in the dashboard is controlled with Azure Resource Based Access Control.
Perf
| where CounterName == "% Processor Time" and ObjectName == "Processor" and InstanceName == "_Total"
| summarize AggregatedValue = avg(CounterValue) by bin(TimeGenerated, 1hr), Computer
| render timechart
Save the query by selecting the Save button from the top of the page.
In the Save Query control panel, provide a name such as Azure VMs - Processor Utilization and a category such
as Dashboards and then click Save. This way you can create a library of common queries that you can use and
modify. Finally, pin this to the shared dashboard created earlier by selecting the Pin to dashboard button from
the top right corner of the page and then selecting the dashboard name.
Now that we have a query pinned to the dashboard, you will notice it has a generic title and comment below it.
We should rename it to something meaningful that can be easily understood by those viewing it. Click the edit
button to customize the title and subtitle for the tile, and then click Update. A banner will appear asking you to
publish changes or discard. Click Save a copy.
Next steps
In this tutorial, you learned how to create a dashboard in the Azure portal and add a log query to it. Advance to
the next tutorial to learn the different responses you can implement based on log query results.
Respond to events with Log Analytics Alerts
Create custom KPI dashboards using Azure
Application Insights
10/24/2019 • 5 minutes to read • Edit Online
You can create multiple dashboards in the Azure portal that each include tiles visualizing data from multiple
Azure resources across different resource groups and subscriptions. You can pin different charts and views from
Azure Application Insights to create custom dashboards that provide you with complete picture of the health and
performance of your application. This tutorial walks you through the creation of a custom dashboard that
includes multiple types of data and visualizations from Azure Application Insights. You learn how to:
Create a custom dashboard in Azure
Add a tile from the Tile Gallery
Add standard metrics in Application Insights to the dashboard
Add a custom metric chart Application Insights to the dashboard
Add the results of a Logs (Analytics) query to the dashboard
Prerequisites
To complete this tutorial:
Deploy a .NET application to Azure and enable the Application Insights SDK.
Sign in to Azure
Sign in to the Azure portal at https://portal.azure.com.
6. Click Done customizing at the top of the screen to exit tile customization mode.
2. Keep the Dashboard name the same and select the Subscription Name to share the dashboard. Click
Publish. The dashboard is now available to other services and subscriptions. You can optionally define
specific users who should have access to the dashboard.
3. Select your Application Insights resource in the home screen.
4. Click Logs (Analytics) on the left under monitoring to open the Logs (Analytics) portal.
5. Type the following query, which returns the top 10 most requested pages and their request count:
requests
| summarize count() by name
| sort by count_ desc
| take 10
exceptions
| summarize count() by operation_Name
| sort by count_ desc
| take 10
10. Click the pin icon on the top right to pin the chart to your dashboard and this time select the link to
return to your dashboard.
11. The results of the queries are now added to your dashboard in the format that you selected. Click and drag
each into position and then click Done customizing.
12. Select the pencil icon on each title to give them a descriptive title.
13. Select Share to republish your changes to your dashboard that now includes a variety of charts and
visualizations from Application Insights.
Next steps
Now that you've learned how to create custom dashboards, have a look at the rest of the Application Insights
documentation including a case study.
Deep diagnostics
Monitor your Azure services in Grafana
10/17/2019 • 6 minutes to read • Edit Online
You can now monitor Azure services and applications from Grafana using the Azure Monitor data source plugin.
The plugin gathers application performance data collected by Azure Monitor, including various logs and metrics.
You can then display this data on your Grafana dashboard.
Use the following steps to set up a Grafana server and build dashboards for metrics and logs from Azure
Monitor.
If you select the network security group (grafana -nsg in this case), you can see that port 3000 is used to
access Grafana server.
7. Get the public IP address of your Grafana server - go back to the list of resources and select Public IP
address.
Sign in to Grafana
1. Using the IP address of your server, open the Login page at http://<IP address>:3000 or the
<DNSName>:3000 in your browser. While 3000 is the default port, note you might have selected a
different port during setup. You should see a login page for the Grafana server you built.
2. Sign in with the user name admin and the Grafana server admin password you created earlier. If you're
using a local setup, the default password would be admin, and you'd be requested to change it on your first
login.
1. Select Add data source to add and configure the Azure Monitor data source.
2. Pick a name for the data source and select Azure Monitor as the type from the dropdown.
3. Create a service principal - Grafana uses an Azure Active Directory service principal to connect to Azure
Monitor APIs and collect data. You must create, or use an existing service principal, to manage access to
your Azure resources.
See these instructions to create a service principal. Copy and save your tenant ID (Directory ID ), client
ID (Application ID ) and client secret (Application key value).
See Assign application to role to assign the Reader role to the Azure Active Directory application on the
subscription, resource group or resource you want to monitor. The Log Analytics API requires the Log
Analytics Reader role, which includes the Reader role's permissions and adds to it.
4. Provide the connection details to the APIs you'd like to use. You can connect to all or to some of them.
If you connect to both metrics and logs in Azure Monitor, you can reuse the same credentials by
selecting Same details as Azure Monitor API.
When configuring the plugin, you can indicate which Azure Cloud you would like the plugin to
monitor (Public, Azure US Government, Azure Germany, or Azure China).
If you use Application Insights, you can also include your Application Insights API and application
ID to collect Application Insights based metrics. For more information, see Getting your API key and
Application ID.
NOTE
Some data source fields are named differently than their correlated Azure settings:
Tenant ID is the Azure Directory ID
Client ID is the Azure Active Directory Application ID
Client Secret is the Azure Active Directory Application key value
5. If you use Application Insights, you can also include your Application Insights API and application ID to
collect Application Insights based metrics. For more information, see Getting your API key and Application
ID.
6. Select Save, and Grafana will test the credentials for each API. You should see a message similar to the
following one.
Build a Grafana dashboard
1. Go to the Grafana Home page, and select New Dashboard.
2. In the new dashboard, select the Graph. You can try other charting options but this article uses Graph as
an example.
3. A blank graph shows up on your dashboard. Click on the panel title and select Edit to enter the details of
the data you want to plot in this graph chart.
4. Select the Azure Monitor data source you've configured.
Collecting Azure Monitor metrics - select Azure Monitor in the service dropdown. A list of
selectors shows up, where you can select the resources and metric to monitor in this chart. To collect
metrics from a VM, use the namespace Microsoft.Compute/VirtualMachines. Once you have
selected VMs and metrics, you can start viewing their data in the dashboard.
Collecting Azure Monitor log data - select Azure Log Analytics in the service dropdown. Select the
workspace you'd like to query and set the query text. You can copy here any log query you already
have or create a new one. As you type in your query, IntelliSense will show up and suggest
autocomplete options. Select the visualization type, Time series Table, and run the query.
NOTE
The default query provided with the plugin uses two macros: "$__timeFilter() and $__interval. These macros
allow Grafana to dynamically calculate the time range and time grain, when you zoom in on part of a chart.
You can remove these macros and use a standard time filter, such as TimeGenerated > ago (1h), but that
means the graph would not support the zoom in feature.
5. Following is a simple dashboard with two charts. The one on left shows the CPU percentage of two VMs.
The chart on the right shows the transactions in an Azure Storage account broken down by the Transaction
API type.
Optional: Monitor your custom metrics in the same Grafana server
You can also install Telegraf and InfluxDB to collect and plot both custom and agent-based metrics same Grafana
instance. There are many data source plugins that you can use to bring these metrics together in a dashboard.
You can also reuse this set up to include metrics from your Prometheus server. Use the Prometheus data source
plugin in Grafana's plugin gallery.
Here are good reference articles on how to use Telegraf, InfluxDB, Prometheus, and Docker
How To Monitor System Metrics with the TICK Stack on Ubuntu 16.04
A monitoring solution for Docker hosts, containers, and containerized services
Here is an image of a full Grafana dashboard that has metrics from Azure Monitor and Application Insights.
Usage
| where $__timeFilter(TimeGenerated)
| summarize total_KBytes=sum(Quantity)*1024 by bin(TimeGenerated, $__interval)
| sort by TimeGenerated
You can configure a variable that will list all available Solution values, and then update your query to use it. To
create a new variable, click the dashboard's Settings button in the top right area, select Variables, and then New.
On the variable page, define the data source and query to run in order to get the list of values.
Once created, adjust the query to use the selected value(s) and your charts will respond accordingly:
Usage
| where $__timeFilter(TimeGenerated) and Solution in ($Solutions)
| summarize total_KBytes=sum(Quantity)*1024 by bin(TimeGenerated, $__interval)
| sort by TimeGenerated
Next steps
Overview of Azure Monitor Metrics
Monitor the availability of any website
10/23/2019 • 6 minutes to read • Edit Online
After you've deployed your web app/website, you can set up recurring tests to monitor
availability and responsiveness. Azure Application Insights sends web requests to your
application at regular intervals from points around the world. It can alert you if your application
isn't responding, or if it responds too slowly.
You can set up availability tests for any HTTP or HTTPS endpoint that is accessible from the
public internet. You don't have to make any changes to the website you're testing. In fact, it
doesn't even have to be a site you own. You can test the availability of a REST API that your
service depends on.
Types of availability tests:
There are three types of availability tests:
URL ping test: a simple test that you can create in the Azure portal.
Multi-step web test: A recording of a sequence of web requests, which can be played back to
test more complex scenarios. Multi-step web tests are created in Visual Studio Enterprise
and uploaded to the portal for execution.
Custom Track Availability Tests: If you decide to create a custom application to run
availability tests, the TrackAvailability() method can be used to send the results to
Application Insights.
You can create up to 100 availability tests per Application Insights resource.
URL The URL can be any web page you want to test, but
it must be visible from the public internet. The URL
can include a query string. So, for example, you can
exercise your database a little. If the URL resolves to
a redirect, we follow it up to 10 redirects.
Parse dependent requests Test requests images, scripts, style files, and other
files that are part of the web page under test. The
recorded response time includes the time taken to
get these files. The test fails if any of these
resources cannot be successfully downloaded within
the timeout for the whole test. If the option is not
checked, the test only requests the file at the URL
you specified. Enabling this option results in a
stricter check. The test could fail for cases, which
may not be noticeable when manually browsing the
site.
Enable retries when the test fails, it is retried after a short interval.
A failure is reported only if three successive
attempts fail. Subsequent tests are then performed
at the usual test frequency. Retry is temporarily
suspended until the next success. This rule is
applied independently at each test location. We
recommend this option. On average, about 80%
of failures disappear on retry.
Test frequency Sets how often the test is run from each test
location. With a default frequency of five minutes
and five test locations, your site is tested on
average every minute.
Test locations Are the places from where our servers send web
requests to your URL. Our minimum number of
recommended test locations is five in order to
insure that you can distinguish problems in your
website from network issues. You can select up to
16 locations.
If your URL is not visible from the public internet, you can choose to selectively open
up your firewall to allow only the test transactions through. To learn more about the
firewall exceptions for our availability test agents, consult the IP address guide.
NOTE
We strongly recommend testing from multiple locations with a minimum of five locations. This is to
prevent false alarms that may result from transient issues with a specific location. In addition we have
found that the optimal configuration is to have the number of test locations be equal to the alert
location threshold + 2.
Success criteria
SETTING EXPLANATION
Alerts
SETTING EXPLANATION
Select a particular test, location, or reduce the time period to see more results around the time
period of interest. Use Search Explorer to see results from all executions, or use Analytics
queries to run custom reports on this data.
From an availability test result, you can see the transaction details across all components. Here
you can:
Inspect the response received from your server.
Diagnose failure with correlated server-side telemetry collected while processing the failed
availability test.
Log an issue or work item in Git or Azure Boards to track the problem. The bug will contain a
link to this event.
Open the web test result in Visual Studio.
Learn more about the end to end transaction diagnostics experience here.
Click on the exception row to see the details of the server-side exception that caused the
synthetic availability test to fail. You can also get the debug snapshot for richer code level
diagnostics.
In addition to the raw results, you can also view two key Availability metrics in Metrics Explorer:
1. Availability: Percentage of the tests that were successful, across all test executions.
2. Test Duration: Average test duration across all test executions.
Automation
Use PowerShell scripts to set up an availability test automatically.
Set up a webhook that is called when an alert is raised.
Troubleshooting
Dedicated troubleshooting article.
Next steps
Availability Alerts
Multi-step web tests
Set Alerts in Application Insights
10/20/2019 • 8 minutes to read • Edit Online
Azure Application Insights can alert you to changes in performance or usage metrics in your web app.
Application Insights monitors your live app on a wide variety of platforms to help you diagnose performance
issues and understand usage patterns.
There are multiple types of alerts:
Metric alerts tell you when a metric crosses a threshold value for some period - such as response times,
exception counts, CPU usage, or page views.
Log Alerts is used to describe alerts where the alert signal is based on a custom Kusto query.
Web tests tell you when your site is unavailable on the internet, or responding slowly. Learn more.
Proactive diagnostics are configured automatically to notify you about unusual performance patterns.
NOTE
In the alerts blade, you see that there's already an alert set up: Proactive Diagnostics. The automatic alert monitors one
particular metric, request failure rate. Unless you decide to disable the proactive alert, you don't need to set your own
alert on request failure rate.
7. Under "Alert logic", choose whether it's based on number of results or metric measurement. Then pick
the condition (greater than, equal to, less than) and a threshold. While you are changing these values,
you may notice the condition preview sentence changes. In this example we are using "equal to".
8. Under "Evaluated based on", set the period and frequency. The period here must match the value that
we put for period in the query above. Then click done.
9. We now see the condition we created with the estimated monthly cost. Below under "Action Groups"
you can create a new group or select an existing one. If you want, you can customize the actions.
10. Finally add your alert details (alert rule name, description, severity). When you are done, click Create
alert rule at the bottom.
How to unsubscribe from classic alert e-mail notifications
This section applies to classic availability alerts, classic Application Insights metric alerts, and to classic
failure anomalies alerts.
You are receiving e-mail notifications for these classic alerts if any of the following applies:
Your e-mail address is listed in the Notification e-mail recipients field in the alert rule settings.
The option to send e-mail notifications to users holding certain roles for the subscription is activated,
and you hold a respective role for that particular Azure subscription.
To better control your security and privacy we generally recommend that you explicitly specify the notification
recipients for your classic alerts in the Notification email recipients field. The option to notify all users
holding certain roles is provided for backward compatibility.
To unsubscribe from e-mail notifications generated by a certain alert rule, remove your e-mail address from
the Notification email recipients field.
If your email address is not listed explicitly, we recommend that you disable the option to notify all members
of certain roles automatically, and instead list all user e-mails who need to receive notifications for that alert
rule in the Notification e-mail recipients field.
Use the new alert experience/near-realtime alerts if you need to notify users based on their roles. With action
groups, you can configure email notifications to users with any of the contributor/owner/reader roles (not
combined together as a single option).
Automation
Use PowerShell to automate setting up alerts
Use webhooks to automate responding to alerts
See also
Availability web tests
Automate setting up alerts
Proactive diagnostics
Create, view, and manage metric alerts using
Azure Monitor
1/23/2020 • 4 minutes to read • Edit Online
Metric alerts in Azure Monitor provide a way to get notified when one of your metrics cross a threshold.
Metric alerts work on a range of multi-dimensional platform metrics, custom metrics, Application Insights
standard and custom metrics. In this article, we will describe how to create, view and manage metric alert
rules through Azure portal and Azure CLI. You can also create metric alert rules using Azure Resource
Manager templates which is described in a separate article.
You can learn more about how metric alerts work from metric alerts overview.
TIP
Most resource blades also have Alerts in their resource menu under Monitoring, you could create alerts from
there as well.
3. Click Select target, in the context pane that loads, select a target resource that you want to alert on.
Use Subscription and Resource type drop-downs to find the resource you want to monitor. You can
also use the search bar to find your resource.
4. If the selected resource has metrics you can create alerts on, Available signals on the bottom right
will include metrics. You can view the full list of resource types supported for metric alerts in this
article.
5. Once you have selected a target resource, click on Add condition.
6. You will see a list of signals supported for the resource, select the metric you want to create an alert
on.
7. Optionally, refine the metric by adjusting Period and Aggregation. If the metric has dimensions, you
will see Dimensions table presented. Select one or more values per dimension. The metric alert will
run evaluate the condition for all combinations of values selected. Learn more about how alerting on
multi-dimensional metrics works. You can also Select * for any of the dimensions. Select * will
dynamically scale the selection to all current and future values for a dimension.
8. You will see a chart for the metric for the last 6 hours. Define the alert parameters; Condition Type,
Frequency, Operator and Threshold or Sensitivity, this will determine the logic which the metric
alert rule will evaluate. Learn more about Dynamic Thresholds condition type and sensitivity options.
9. If you are using a static threshold, the metric chart can help determine what might be a reasonable
threshold. If you are using a Dynamic Thresholds, the metric chart will display the calculated
thresholds based on recent data.
10. Click Done
11. Optionally, add another criteria if you want to monitor a complex alert rule. Currently users can have
alert rules with Dynamic Thresholds criteria as a single criterion.
12. Fill in Alert details like Alert Rule Name, Description and Severity
13. Add an action group to the alert either by selecting an existing action group or creating a new action
group.
14. Click Done to save the metric alert rule.
NOTE
Metric alert rules created through portal are created in the same resource group as the target resource.
TIP
In the Manage rules blade, you can select multiple alert rules and enable/disable them. This might be useful
when certain target resources need to be put under maintenance
4. Click on the name of the metric alert rule you want to edit
5. In the Edit Rule, click on the Alert criteria you want to edit. You can change the metric, threshold
condition and other fields as required
NOTE
You can't edit the Target resource and Alert Rule Name after the metric alert is created.
3. You can create a simple metric alert rule that monitors if average Percentage CPU on a VM is greater
than 90
4. You can view all the metric alerts in a resource group using the following command
5. You can see the details of a particular metric alert rule using the name or the resource ID of the rule.
6. You can disable a metric alert rule using the following command.
7. You can delete a metric alert rule using the following command.
Next steps
Create metric alerts using Azure Resource Manager Templates.
Understand how metric alerts work.
Understand how metric alerts with Dynamic Thresholds condition work.
Understand the web hook schema for metric alerts
Metric Alerts with Dynamic Thresholds in Azure
Monitor
12/3/2019 • 9 minutes to read • Edit Online
Metric Alert with Dynamic Thresholds detection leverages advanced machine learning (ML ) to learn metrics'
historical behavior, identify patterns and anomalies that indicate possible service issues. It provides support of
both a simple UI and operations at scale by allowing users to configure alert rules through the Azure Resource
Manager API, in a fully automated manner.
Once an alert rule is created, it will fire only when the monitored metric doesn’t behave as expected, based on its
tailored thresholds.
We would love to hear your feedback, keep it coming at azurealertsfeedback@microsoft.com.
To trigger an alert when there was a violation from a Dynamic Thresholds in 20 minutes out of the last 30 minutes
with period of 5 minutes, use the following settings:
Ignore data before - Users may also optionally define a start date from which the system should begin
calculating the thresholds from. A typical use case may occur when a resource was a running in a testing mode
and is now promoted to serve a production workload, and therefore the behavior of any metric during the testing
phase should be disregarded.
How do you find out why a Dynamic Thresholds alert was triggered?
You can explore triggered alert instances in the alerts view either by clicking on the link in the email or text
message, or browser to see the alerts view in the Azure portal. Learn more about the alerts view.
The alert view displays:
All the metric details at the moment the Dynamic Thresholds alert fired.
A chart of the period in which the alert was triggered that includes the Dynamic Thresholds used at that point
in time.
Ability to provide feedback on Dynamic Thresholds alert and the alerts view experience, which could improve
future detections.
3. Click Select target, in the context pane that loads, select a target resource that you want to alert on. Use
Subscription and 'Virtual Machines' Resource type drop-downs to find the resource you want to
monitor. You can also use the search bar to find your resource.
4. Once you have selected a target resource, click on Add condition.
5. Select the 'CPU Percentage'.
6. Optionally, refine the metric by adjusting Period and Aggregation. It is discouraged to use 'Maximum'
aggregation type for this metric type as it is less representative of behavior. For 'Maximum' aggregation
type static threshold maybe more appropriate.
7. You will see a chart for the metric for the last 6 hours. Define the alert parameters:
a. Condition Type - Choose 'Dynamic' option.
b. Sensitivity - Choose Medium/Low sensitivity to reduce alert noise.
c. Operator - Choose 'Greater Than' unless behavior represents the application usage.
d. Frequency - Consider lowering based on business impact of the alert.
e. Failing Periods (Advanced Option) - The look back window should be at least 15 minutes. For example,
if the period is set to five minutes, then failing periods should be at least three or more.
8. The metric chart will display the calculated thresholds based on recent data.
9. Click Done.
10. Fill in Alert details like Alert Rule Name, Description, and Severity.
11. Add an action group to the alert either by selecting an existing action group or creating a new action group.
12. Click Done to save the metric alert rule.
NOTE
Metric alert rules created through portal are created in the same resource group as the target resource.
TIP
Most resource blades also have Alerts in their resource menu under Monitoring, you could create alerts from there
as well.
3. Click Select target, in the context pane that loads, select a target resource that you want to alert on. Use
Subscription and 'Application Insights' Resource type drop-downs to find the resource you want to
monitor. You can also use the search bar to find your resource.
4. Once you have selected a target resource, click on Add condition.
5. Select the 'HTTP request execution time'.
6. Optionally, refine the metric by adjusting Period and Aggregation. It is discouraged to use 'Maximum'
aggregation type for this metric type as it is less representative of behavior. For 'Maximum' aggregation
type static threshold maybe more appropriate.
7. You will see a chart for the metric for the last 6 hours. Define the alert parameters:
a. Condition Type - Choose 'Dynamic' option.
b. Operator - Choose 'Greater Than' to reduce alerts fired on improvement in duration.
c. Frequency - Consider lowering based on business impact of the alert.
8. The metric chart will display the calculated thresholds based on recent data.
9. Click Done.
10. Fill in Alert details like Alert Rule Name, Description, and Severity.
11. Add an action group to the alert either by selecting an existing action group or creating a new action group.
12. Click Done to save the metric alert rule.
NOTE
Metric alert rules created through portal are created in the same resource group as the target resource.
Overview
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Azure Monitor supports metric alert type which has benefits over the classic alerts. Metrics are available for large
list of Azure services. This article explains usage of a subset (that is) for resource -
Microsoft.OperationalInsights/workspaces .
You can use metric alerts on popular Log Analytics logs extracted as metrics as part of Metrics from Logs
including resources in Azure or on-premises. The supported Log Analytics solutions are listed below:
Performance counters for Windows & Linux machines
Heartbeat records for Agent Health
Update management records
Event data logs
There are many benefits for using Metric Alerts for Logs over query based Log Alerts in Azure; some of them
are listed below:
Metric Alerts offer near-real time monitoring capability and Metric Alerts for Logs forks data from log source
to ensure the same.
Metric Alerts are stateful - only notifying once when alert is fired and once when alert is resolved; as opposed
to Log alerts, which are stateless and keep firing at every interval if the alert condition is met.
Metric Alerts for Log provide multiple dimensions, allowing filtering to specific values like Computers, OS
Type, etc. simpler; without the need for penning query in analytics.
NOTE
Specific metric and/or dimension will only be shown if data for it exists in chosen period. These metrics are available for
customers with Azure Log Analytics workspaces.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"convertRuleName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the rule to convert log to metric"
}
},
"convertRuleDescription": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Description for log converted to metric"
}
},
"convertRuleRegion": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the region used by workspace"
}
},
"convertRuleStatus": {
"type": "string",
"defaultValue": "true",
"metadata": {
"description": "Specifies whether the log conversion rule is enabled"
}
},
"convertRuleMetric": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric once extraction done from logs."
}
},
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Full Resource ID of the resource emitting the metric that will be used for the
comparison. For example /subscriptions/00000000-0000-0000-0000-0000-
00000000/resourceGroups/ResourceGroupName/providers/Microsoft.compute/virtualMachines/VM_xyz"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"Equals",
"NotEquals",
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "0",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": {
"convertRuleTag": "hidden-link:/subscriptions/1234-56789-1234-
567a/resourceGroups/resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/workspaceName",
"convertRuleSourceWorkspace": {
"SourceId": "/subscriptions/1234-56789-1234-
567a/resourceGroups/resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/workspaceName"
}
},
"resources": [
{
"name": "[parameters('convertRuleName')]",
"type": "Microsoft.Insights/scheduledQueryRules",
"apiVersion": "2018-04-16",
"location": "[parameters('convertRuleRegion')]",
"tags": {
"[variables('convertRuleTag')]": "Resource"
},
"properties": {
"description": "[parameters('convertRuleDescription')]",
"enabled": "[parameters('convertRuleStatus')]",
"source": {
"dataSourceId": "[variables('convertRuleSourceWorkspace').SourceId]"
},
"action": {
"odata.type":
"Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.Microsoft.AppInsights.Nexus.DataContracts.Resource
s.ScheduledQueryRules.LogToMetricAction",
"criteria": [{
"metricName": "[parameters('convertRuleMetric')]",
"dimensions": []
}
]
}
}
},
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"dependsOn":["
[resourceId('Microsoft.Insights/scheduledQueryRules',parameters('convertRuleName'))]"],
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria",
"allOf": [
{
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"threshold" : "[parameters('threshold')]",
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
Say the above JSON is saved as metricfromLogsAlertStatic.json - then it can be coupled with a parameter JSON
file for Resource Template based creation. A sample parameter JSON file is listed below:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"convertRuleName": {
"value": "TestLogtoMetricRule"
},
"convertRuleDescription": {
"value": "Test rule to extract metrics from logs via template"
},
"convertRuleRegion": {
"value": "West Central US"
},
"convertRuleStatus": {
"value": "true"
},
"convertRuleMetric": {
"value": "Average_% Idle Time"
},
"alertName": {
"value": "TestMetricAlertonLog"
},
"alertDescription": {
"value": "New multi-dimensional metric alert created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/1234-56789-1234-
567a/resourceGroups/myRG/providers/Microsoft.OperationalInsights/workspaces/workspaceName"
},
"metricName":{
"value": "Average_% Idle Time"
},
"operator": {
"value": "GreaterThan"
},
"threshold":{
"value": "1"
},
"timeAggregation":{
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/1234-56789-1234-
567a/resourceGroups/myRG/providers/microsoft.insights/actionGroups/actionGroupName"
}
}
}
Assuming the above parameter file is saved as metricfromLogsAlertStatic.parameters.json; then one can create
metric alert for logs using Resource Template for creation in Azure portal.
Alternatively, one can use the Azure Powershell command below as well:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"convertRuleName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the rule to convert log to metric"
}
},
"convertRuleDescription": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Description for log converted to metric"
}
},
"convertRuleRegion": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the region used by workspace"
}
},
"convertRuleStatus": {
"type": "string",
"defaultValue": "true",
"metadata": {
"description": "Specifies whether the log conversion rule is enabled"
}
},
"convertRuleMetric": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric once extraction done from logs."
}
},
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Full Resource ID of the resource emitting the metric that will be used for the
comparison. For example /subscriptions/00000000-0000-0000-0000-0000-
00000000/resourceGroups/ResourceGroupName/providers/Microsoft.compute/virtualMachines/VM_xyz"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterOrLessThan",
"allowedValues": [
"GreaterThan",
"LessThan",
"GreaterOrLessThan"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"alertSensitivity": {
"type": "string",
"defaultValue": "Medium",
"allowedValues": [
"High",
"Medium",
"Low"
],
"metadata": {
"description": "Tunes how 'noisy' the Dynamic Thresholds alerts will be: 'High' will result in
more alerts while 'Low' will result in fewer alerts."
}
},
"numberOfEvaluationPeriods": {
"type": "string",
"defaultValue": "4",
"metadata": {
"description": "The number of periods to check in the alert evaluation."
}
},
"minFailingPeriodsToAlert": {
"type": "string",
"defaultValue": "3",
"metadata": {
"metadata": {
"description": "The number of unhealthy periods to alert on (must be lower or equal to
numberOfEvaluationPeriods)."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": {
"convertRuleTag": "hidden-link:/subscriptions/1234-56789-1234-
567a/resourceGroups/resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/workspaceName",
"convertRuleSourceWorkspace": {
"SourceId": "/subscriptions/1234-56789-1234-
567a/resourceGroups/resourceGroupName/providers/Microsoft.OperationalInsights/workspaces/workspaceName"
}
},
"resources": [
{
"name": "[parameters('convertRuleName')]",
"type": "Microsoft.Insights/scheduledQueryRules",
"apiVersion": "2018-04-16",
"location": "[parameters('convertRuleRegion')]",
"tags": {
"[variables('convertRuleTag')]": "Resource"
},
"properties": {
"description": "[parameters('convertRuleDescription')]",
"enabled": "[parameters('convertRuleStatus')]",
"source": {
"dataSourceId": "[variables('convertRuleSourceWorkspace').SourceId]"
},
"action": {
"odata.type":
"Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.Microsoft.AppInsights.Nexus.DataContracts.Resource
s.ScheduledQueryRules.LogToMetricAction",
s.ScheduledQueryRules.LogToMetricAction",
"criteria": [{
"metricName": "[parameters('convertRuleMetric')]",
"dimensions": []
}
]
}
}
},
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"dependsOn":["
[resourceId('Microsoft.Insights/scheduledQueryRules',parameters('convertRuleName'))]"],
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"criterionType": "DynamicThresholdCriterion",
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"alertSensitivity": "[parameters('alertSensitivity')]",
"failingPeriods": {
"numberOfEvaluationPeriods": "[parameters('numberOfEvaluationPeriods')]",
"minFailingPeriodsToAlert": "[parameters('minFailingPeriodsToAlert')]"
},
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
Say the above JSON is saved as metricfromLogsAlertDynamic.json - then it can be coupled with a parameter
JSON file for Resource Template based creation. A sample parameter JSON file is listed below:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"convertRuleName": {
"value": "TestLogtoMetricRule"
},
"convertRuleDescription": {
"value": "Test rule to extract metrics from logs via template"
},
"convertRuleRegion": {
"value": "West Central US"
},
"convertRuleStatus": {
"value": "true"
},
"convertRuleMetric": {
"value": "Average_% Idle Time"
},
"alertName": {
"value": "TestMetricAlertonLog"
},
"alertDescription": {
"value": "New multi-dimensional metric alert created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/1234-56789-1234-
567a/resourceGroups/myRG/providers/Microsoft.OperationalInsights/workspaces/workspaceName"
},
"metricName":{
"value": "Average_% Idle Time"
},
"operator": {
"value": "GreaterOrLessThan"
},
"alertSensitivity": {
"value": "Medium"
},
"numberOfEvaluationPeriods": {
"value": "4"
},
"minFailingPeriodsToAlert": {
"value": "3"
},
"timeAggregation":{
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/1234-56789-1234-
567a/resourceGroups/myRG/providers/microsoft.insights/actionGroups/actionGroupName"
}
}
}
Assuming the above parameter file is saved as metricfromLogsAlertDynamic.parameters.json; then one can create
metric alert for logs using Resource Template for creation in Azure portal.
Alternatively, one can use the Azure Powershell command below as well:
New-AzResourceGroupDeployment -ResourceGroupName "myRG" -TemplateFile metricfromLogsAlertDynamic.json
TemplateParameterFile metricfromLogsAlertDynamic.parameters.json
Next steps
Learn more about the metric alerts.
Learn about log alerts in Azure.
Learn about alerts in Azure.
Create a metric alert with a Resource Manager
template
1/14/2020 • 45 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
This article shows how you can use an Azure Resource Manager template to configure newer metric alerts in
Azure Monitor. Resource Manager templates enable you to programmatically set up alerts in a consistent and
reproducible way across your environments. Newer metric alerts are currently available on this set of resource
types.
IMPORTANT
Resource template for creating metric alerts for resource type: Azure Log Analytics Workspace (i.e.)
Microsoft.OperationalInsights/workspaces , requires additional steps. For details, see the article on Metric Alert for Logs
- Resource Template.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Full Resource ID of the resource emitting the metric that will be used for the
comparison. For example /subscriptions/00000000-0000-0000-0000-0000-
00000000/resourceGroups/ResourceGroupName/providers/Microsoft.compute/virtualMachines/VM_xyz"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"Equals",
"NotEquals",
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "0",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H",
"PT6H",
"PT12H",
"PT24H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between one minute and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria",
"allOf": [
{
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"threshold" : "[parameters('threshold')]",
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
An explanation of the schema and properties for an alert rule is available here.
You can set the values for the parameters either on the command line or through a parameter file. A sample
parameter file is provided below.
Save the json below as simplestaticmetricalert.parameters.json and modify it as required.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "New Metric Alert"
},
"alertDescription": {
"value": "New metric alert created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resourceGroup-
name/providers/Microsoft.Compute/virtualMachines/replace-with-resource-name"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterThan"
},
"threshold": {
"value": "80"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI.
Using Azure PowerShell
Connect-AzAccount
az login
NOTE
While the metric alert could be created in a different resource group to the target resource, we recommend using the same
resource group as your target resource.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Full Resource ID of the resource emitting the metric that will be used for the
comparison. For example /subscriptions/00000000-0000-0000-0000-0000-
00000000/resourceGroups/ResourceGroupName/providers/Microsoft.compute/virtualMachines/VM_xyz"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterOrLessThan",
"allowedValues": [
"GreaterThan",
"LessThan",
"GreaterOrLessThan"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"alertSensitivity": {
"type": "string",
"defaultValue": "Medium",
"allowedValues": [
"High",
"Medium",
"Low"
],
"metadata": {
"description": "Tunes how 'noisy' the Dynamic Thresholds alerts will be: 'High' will result in
more alerts while 'Low' will result in fewer alerts."
}
},
"numberOfEvaluationPeriods": {
"type": "string",
"defaultValue": "4",
"metadata": {
"description": "The number of periods to check in the alert evaluation."
}
},
"minFailingPeriodsToAlert": {
"type": "string",
"defaultValue": "3",
"metadata": {
"description": "The number of unhealthy periods to alert on (must be lower or equal to
numberOfEvaluationPeriods)."
}
},
"ignoreDataBefore": {
"ignoreDataBefore": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Use this option to set the date from which to start learning the metric
historical data and calculate the dynamic thresholds (in ISO8601 format, e.g. '2019-12-31T22:00:00Z')."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one hour. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"criterionType": "DynamicThresholdCriterion",
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"alertSensitivity": "[parameters('alertSensitivity')]",
"failingPeriods": {
"numberOfEvaluationPeriods": "[parameters('numberOfEvaluationPeriods')]",
"minFailingPeriodsToAlert": "[parameters('minFailingPeriodsToAlert')]"
},
"ignoreDataBefore": "[parameters('ignoreDataBefore')]",
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
An explanation of the schema and properties for an alert rule is available here.
You can set the values for the parameters either on the command line or through a parameter file. A sample
parameter file is provided below.
Save the json below as simpledynamicmetricalert.parameters.json and modify it as required.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "New Metric Alert with Dynamic Thresholds"
},
"alertDescription": {
"value": "New metric alert with Dynamic Thresholds created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resourceGroup-
name/providers/Microsoft.Compute/virtualMachines/replace-with-resource-name"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterOrLessThan"
},
"alertSensitivity": {
"value": "Medium"
},
"numberOfEvaluationPeriods": {
"value": "4"
},
"minFailingPeriodsToAlert": {
"value": "3"
},
"ignoreDataBefore": {
"value": ""
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI.
Using Azure PowerShell
Connect-AzAccount
NOTE
While the metric alert could be created in a different resource group to the target resource, we recommend using the same
resource group as your target resource.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Resource ID of the resource emitting the metric that will be used for the
comparison."
}
},
"criterion1":{
"type": "object",
"metadata": {
"description": "Criterion includes metric name, dimension values, threshold and an operator.
The alert rule fires when ALL criteria are met"
}
},
"criterion2": {
"type": "object",
"metadata": {
"description": "Criterion includes metric name, dimension values, threshold and an operator.
The alert rule fires when ALL criteria are met"
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H",
"PT6H",
"PT12H",
"PT24H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between one minute and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": {
"criterion1": "[array(parameters('criterion1'))]",
"criterion2": "[array(parameters('criterion2'))]",
"criteria": "[concat(variables('criterion1'),variables('criterion2'))]"
},
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria",
"allOf": "[variables('criteria')]"
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template along with the parameter file provided below.
Save and modify the json below as advancedstaticmetricalert.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "New Multi-dimensional Metric Alert (Replace with your alert name)"
},
"alertDescription": {
"value": "New multi-dimensional metric alert created via template (Replace with your alert
description)"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resourcegroup-
name/providers/Microsoft.Storage/storageAccounts/replace-with-storage-account"
},
"criterion1": {
"value": {
"name": "1st criterion",
"metricName": "Transactions",
"dimensions": [
{
"name":"ResponseType",
"operator": "Include",
"values": ["Success"]
},
{
"name":"ApiName",
"operator": "Include",
"values": ["GetBlob"]
}
],
"operator": "GreaterThan",
"threshold": "5",
"timeAggregation": "Total"
}
},
"criterion2": {
"value":{
"name": "2nd criterion",
"metricName": "SuccessE2ELatency",
"dimensions": [
{
"name":"ApiName",
"operator": "Include",
"values": ["GetBlob"]
}
],
"operator": "GreaterThan",
"threshold": "250",
"timeAggregation": "Average"
}
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-actiongroup-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
az login
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Resource ID of the resource emitting the metric that will be used for the
comparison."
}
},
"criterion":{
"type": "object",
"metadata": {
"description": "Criterion includes metric name, dimension values, threshold and an operator.
The alert rule fires when ALL criteria are met"
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H",
"PT6H",
"PT12H",
"PT24H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between one minute and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": {
"criteria": "[array(parameters('criterion'))]"
},
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria",
"allOf": "[variables('criteria')]"
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template along with the parameter file provided below.
Save and modify the json below as multidimensionalstaticmetricalert.parameters.json for the purpose of this
walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "New multi-dimensional metric alert rule (replace with your alert name)"
},
"alertDescription": {
"value": "New multi-dimensional metric alert rule created via template (replace with your alert
description)"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resourcegroup-
name/providers/Microsoft.Storage/storageAccounts/replace-with-storage-account"
},
"criterion": {
"value": {
"name": "Criterion",
"metricName": "Transactions",
"dimensions": [
{
"name":"ResponseType",
"operator": "Include",
"values": ["*"]
},
{
"name":"ApiName",
"operator": "Include",
"values": ["GetBlob", "PutBlob"]
}
],
"operator": "GreaterThan",
"threshold": "5",
"timeAggregation": "Total"
}
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-actiongroup-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Resource ID of the resource emitting the metric that will be used for the
comparison."
}
},
"criterion":{
"type": "object",
"metadata": {
"description": "Criterion includes metric name, dimension values, threshold and an operator."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one hour. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": {
"criteria": "[array(parameters('criterion'))]"
},
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": "[variables('criteria')]"
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template along with the parameter file provided below.
Save and modify the json below as advanceddynamicmetricalert.parameters.json for the purpose of this
walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "New Multi-dimensional Metric Alert with Dynamic Thresholds (Replace with your alert
name)"
},
"alertDescription": {
"value": "New multi-dimensional metric alert with Dynamic Thresholds created via template (Replace
with your alert description)"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resourcegroup-
name/providers/Microsoft.Storage/storageAccounts/replace-with-storage-account"
},
"criterion": {
"value": {
"criterionType": "DynamicThresholdCriterion",
"name": "1st criterion",
"metricName": "Transactions",
"dimensions": [
{
"name":"ResponseType",
"operator": "Include",
"values": ["*"]
},
{
"name":"ApiName",
"operator": "Include",
"values": ["GetBlob", "PutBlob"]
}
],
"operator": "GreaterOrLessThan",
"alertSensitivity": "Medium",
"failingPeriods": {
"numberOfEvaluationPeriods": "4",
"minFailingPeriodsToAlert": "3"
},
"timeAggregation": "Total"
}
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-actiongroup-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
az login
NOTE
Multiple criteria are not currently supported for metric alert rules that use Dynamic Thresholds.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"resourceId": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Full Resource ID of the resource emitting the metric that will be used for the
comparison. For example /subscriptions/00000000-0000-0000-0000-0000-
00000000/resourceGroups/ResourceGroupName/providers/Microsoft.compute/virtualMachines/VM_xyz"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"metricNamespace": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Namespace of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"Equals",
"NotEquals",
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "0",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H",
"PT6H",
"PT12H",
"PT24H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between one minute and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "How often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('resourceId')]"],
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.SingleResourceMultipleMetricCriteria",
"allOf": [
{
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"metricNamespace": "[parameters('metricNamespace')]",
"dimensions":[],
"dimensions":[],
"operator": "[parameters('operator')]",
"threshold" : "[parameters('threshold')]",
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template along with the parameter file provided below.
Save and modify the json below as customstaticmetricalert.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "New alert rule on a custom metric"
},
"alertDescription": {
"value": "New alert rule on a custom metric created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"resourceId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resourceGroup-
name/providers/microsoft.insights/components/replace-with-application-insights-resource-name"
},
"metricName": {
"value": "The custom metric name"
},
"metricNamespace": {
"value": "Azure.ApplicationInsights"
},
"operator": {
"value": "GreaterThan"
},
"threshold": {
"value": "80"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
az login
NOTE
You can find the metric namespace of a specific custom metric by browsing your custom metrics via the Azure portal
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"targetResourceGroup":{
"type": "array",
"minLength": 1,
"metadata": {
"description": "Full path of the resource group(s) where target resources to be monitored are
in. For example - /subscriptions/00000000-0000-0000-0000-0000-00000000/resourceGroups/ResourceGroupName"
}
},
"targetResourceRegion":{
"type": "string",
"allowedValues": [
"EastUS",
"EastUS2",
"CentralUS",
"NorthCentralUS",
"SouthCentralUS",
"WestCentralUS",
"WestUS",
"WestUS2",
"CanadaEast",
"CanadaCentral",
"BrazilSouth",
"NorthEurope",
"WestEurope",
"FranceCentral",
"FranceSouth",
"UKWest",
"UKSouth",
"GermanyCentral",
"GermanyNortheast",
"GermanyNorth",
"GermanyWestCentral",
"SwitzerlandNorth",
"SwitzerlandWest",
"NorwayEast",
"NorwayWest",
"SoutheastAsia",
"EastAsia",
"AustraliaEast",
"AustraliaSoutheast",
"AustraliaCentral",
"AustraliaCentral2",
"AustraliaCentral2",
"ChinaEast",
"ChinaNorth",
"ChinaEast2",
"ChinaNorth2",
"CentralIndia",
"WestIndia",
"SouthIndia",
"JapanEast",
"JapanWest",
"KoreaCentral",
"KoreaSouth",
"SouthAfricaWest",
"SouthAfricaNorth",
"UAECentral",
"UAENorth"
],
"metadata": {
"description": "Azure region in which target resources to be monitored are in (without
spaces). For example: EastUS"
}
},
"targetResourceType": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Resource type of target resources to be monitored. Currently only supported
resource type is Microsoft.Compute/virtualMachines"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"Equals",
"NotEquals",
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "0",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H",
"PT6H",
"PT12H",
"PT24H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between one minute and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": "[parameters('targetResourceGroup')]",
"targetResourceType": "[parameters('targetResourceType')]",
"targetResourceRegion": "[parameters('targetResourceRegion')]",
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"operator": "[parameters('operator')]",
"threshold" : "[parameters('threshold')]",
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template with the parameter file below. Save and modify the json below as all-vms-in-
resource-group-static.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "Multi-resource metric alert via Azure Resource Manager template"
},
"alertDescription": {
"value": "New Multi-resource metric alert created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"targetResourceGroup":{
"value": [
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name1",
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-name2"
]
},
"targetResourceRegion":{
"value": "SouthCentralUS"
},
"targetResourceType":{
"value": "Microsoft.Compute/virtualMachines"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterThan"
},
"threshold": {
"value": "0"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group-name"
}
}
}
You can create the static metric alert using the template and parameters file using PowerShell or Azure CLI from
your current working directory.
Using Azure PowerShell
Connect-AzAccount
Dynamic Thresholds alert on all virtual machines in one or more resource groups
This template will create a Dynamic Thresholds metric alert rule that monitors Percentage CPU for all virtual
machines (in one Azure region) in one or more resource groups.
Save the json below as all-vms-in-resource-group-dynamic.json for the purpose of this walk-through.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"targetResourceGroup":{
"type": "array",
"minLength": 1,
"metadata": {
"description": "Full path of the resource group(s) where target resources to be monitored are
in. For example - /subscriptions/00000000-0000-0000-0000-0000-00000000/resourceGroups/ResourceGroupName"
}
},
"targetResourceRegion":{
"type": "string",
"allowedValues": [
"EastUS",
"EastUS2",
"CentralUS",
"NorthCentralUS",
"SouthCentralUS",
"WestCentralUS",
"WestUS",
"WestUS2",
"CanadaEast",
"CanadaCentral",
"BrazilSouth",
"NorthEurope",
"WestEurope",
"FranceCentral",
"FranceSouth",
"UKWest",
"UKSouth",
"GermanyCentral",
"GermanyNortheast",
"GermanyNorth",
"GermanyWestCentral",
"SwitzerlandNorth",
"SwitzerlandWest",
"NorwayEast",
"NorwayWest",
"SoutheastAsia",
"EastAsia",
"AustraliaEast",
"AustraliaSoutheast",
"AustraliaCentral",
"AustraliaCentral2",
"ChinaEast",
"ChinaNorth",
"ChinaEast2",
"ChinaNorth2",
"CentralIndia",
"WestIndia",
"SouthIndia",
"JapanEast",
"JapanWest",
"KoreaCentral",
"KoreaSouth",
"SouthAfricaWest",
"SouthAfricaNorth",
"UAECentral",
"UAENorth"
],
"metadata": {
"description": "Azure region in which target resources to be monitored are in (without
spaces). For example: EastUS"
}
},
"targetResourceType": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Resource type of target resources to be monitored. Currently only supported
resource type is Microsoft.Compute/virtualMachines"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterOrLessThan",
"defaultValue": "GreaterOrLessThan",
"allowedValues": [
"GreaterThan",
"LessThan",
"GreaterOrLessThan"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"alertSensitivity": {
"type": "string",
"defaultValue": "Medium",
"allowedValues": [
"High",
"Medium",
"Low"
],
"metadata": {
"description": "Tunes how 'noisy' the Dynamic Thresholds alerts will be: 'High' will result in
more alerts while 'Low' will result in fewer alerts."
}
},
"numberOfEvaluationPeriods": {
"type": "string",
"defaultValue": "4",
"metadata": {
"description": "The number of periods to check in the alert evaluation."
}
},
"minFailingPeriodsToAlert": {
"type": "string",
"defaultValue": "3",
"metadata": {
"description": "The number of unhealthy periods to alert on (must be lower or equal to
numberOfEvaluationPeriods)."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one hour. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": "[parameters('targetResourceGroup')]",
"targetResourceType": "[parameters('targetResourceType')]",
"targetResourceRegion": "[parameters('targetResourceRegion')]",
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"criterionType": "DynamicThresholdCriterion",
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"alertSensitivity": "[parameters('alertSensitivity')]",
"failingPeriods": {
"numberOfEvaluationPeriods": "[parameters('numberOfEvaluationPeriods')]",
"minFailingPeriodsToAlert": "[parameters('minFailingPeriodsToAlert')]"
},
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template with the parameter file below. Save and modify the json below as all-vms-in-
resource-group-dynamic.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "Multi-resource metric alert with Dynamic Thresholds via Azure Resource Manager template"
},
"alertDescription": {
"value": "New Multi-resource metric alert with Dynamic Thresholds created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"targetResourceGroup":{
"value": [
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name1",
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-name2"
]
},
"targetResourceRegion":{
"value": "SouthCentralUS"
},
"targetResourceType":{
"value": "Microsoft.Compute/virtualMachines"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterOrLessThan"
},
"alertSensitivity": {
"value": "Medium"
},
"numberOfEvaluationPeriods": {
"value": "4"
},
"minFailingPeriodsToAlert": {
"value": "3"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
az login
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
},
"targetSubscription":{
"type": "string",
"minLength": 1,
"metadata": {
"description": "Azure Resource Manager path up to subscription ID. For example -
/subscriptions/00000000-0000-0000-0000-0000-00000000"
}
},
"targetResourceRegion":{
"type": "string",
"allowedValues": [
"EastUS",
"EastUS2",
"CentralUS",
"NorthCentralUS",
"SouthCentralUS",
"WestCentralUS",
"WestUS",
"WestUS2",
"CanadaEast",
"CanadaCentral",
"BrazilSouth",
"NorthEurope",
"WestEurope",
"FranceCentral",
"FranceSouth",
"UKWest",
"UKSouth",
"GermanyCentral",
"GermanyNortheast",
"GermanyNorth",
"GermanyWestCentral",
"SwitzerlandNorth",
"SwitzerlandWest",
"NorwayEast",
"NorwayWest",
"SoutheastAsia",
"EastAsia",
"AustraliaEast",
"AustraliaSoutheast",
"AustraliaCentral",
"AustraliaCentral2",
"ChinaEast",
"ChinaNorth",
"ChinaEast2",
"ChinaNorth2",
"CentralIndia",
"WestIndia",
"SouthIndia",
"JapanEast",
"JapanWest",
"KoreaCentral",
"KoreaSouth",
"SouthAfricaWest",
"SouthAfricaNorth",
"UAECentral",
"UAENorth"
],
"metadata": {
"description": "Azure region in which target resources to be monitored are in (without
spaces). For example: EastUS"
}
},
"targetResourceType": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Resource type of target resources to be monitored. Currently only supported
resource type is Microsoft.Compute/virtualMachines"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"Equals",
"NotEquals",
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "0",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H",
"PT6H",
"PT12H",
"PT24H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between one minute and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"allowedValues": [
"PT1M",
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('targetSubscription')]"],
"targetResourceType": "[parameters('targetResourceType')]",
"targetResourceRegion": "[parameters('targetResourceRegion')]",
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"threshold" : "[parameters('threshold')]",
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template with the parameter file below. Save and modify the json below as all-vms-in-
subscription-static.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "Multi-resource sub level metric alert via Azure Resource Manager template"
},
"alertDescription": {
"value": "New Multi-resource sub level metric alert created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"targetSubscription":{
"value": "/subscriptions/replace-with-subscription-id"
},
"targetResourceRegion":{
"value": "SouthCentralUS"
},
"targetResourceType":{
"value": "Microsoft.Compute/virtualMachines"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterThan"
},
"threshold": {
"value": "0"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"targetSubscription":{
"type": "string",
"minLength": 1,
"metadata": {
"description": "Azure Resource Manager path up to subscription ID. For example -
/subscriptions/00000000-0000-0000-0000-0000-00000000"
}
},
"targetResourceRegion":{
"type": "string",
"allowedValues": [
"EastUS",
"EastUS2",
"CentralUS",
"NorthCentralUS",
"SouthCentralUS",
"WestCentralUS",
"WestUS",
"WestUS2",
"CanadaEast",
"CanadaCentral",
"BrazilSouth",
"NorthEurope",
"WestEurope",
"FranceCentral",
"FranceSouth",
"UKWest",
"UKSouth",
"GermanyCentral",
"GermanyNortheast",
"GermanyNorth",
"GermanyWestCentral",
"SwitzerlandNorth",
"SwitzerlandWest",
"NorwayEast",
"NorwayWest",
"SoutheastAsia",
"EastAsia",
"AustraliaEast",
"AustraliaSoutheast",
"AustraliaCentral",
"AustraliaCentral2",
"ChinaEast",
"ChinaNorth",
"ChinaEast2",
"ChinaNorth2",
"CentralIndia",
"WestIndia",
"SouthIndia",
"JapanEast",
"JapanWest",
"KoreaCentral",
"KoreaSouth",
"SouthAfricaWest",
"SouthAfricaNorth",
"UAECentral",
"UAENorth"
],
"metadata": {
"description": "Azure region in which target resources to be monitored are in (without
spaces). For example: EastUS"
}
},
"targetResourceType": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Resource type of target resources to be monitored. Currently only supported
resource type is Microsoft.Compute/virtualMachines"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterOrLessThan",
"defaultValue": "GreaterOrLessThan",
"allowedValues": [
"GreaterThan",
"LessThan",
"GreaterOrLessThan"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"alertSensitivity": {
"type": "string",
"defaultValue": "Medium",
"allowedValues": [
"High",
"Medium",
"Low"
],
"metadata": {
"description": "Tunes how 'noisy' the Dynamic Thresholds alerts will be: 'High' will result in
more alerts while 'Low' will result in fewer alerts."
}
},
"numberOfEvaluationPeriods": {
"type": "string",
"defaultValue": "4",
"metadata": {
"description": "The number of periods to check in the alert evaluation."
}
},
"minFailingPeriodsToAlert": {
"type": "string",
"defaultValue": "3",
"metadata": {
"description": "The number of unhealthy periods to alert on (must be lower or equal to
numberOfEvaluationPeriods)."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one hour. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": ["[parameters('targetSubscription')]"],
"targetResourceType": "[parameters('targetResourceType')]",
"targetResourceRegion": "[parameters('targetResourceRegion')]",
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"criterionType": "DynamicThresholdCriterion",
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"alertSensitivity": "[parameters('alertSensitivity')]",
"failingPeriods": {
"numberOfEvaluationPeriods": "[parameters('numberOfEvaluationPeriods')]",
"minFailingPeriodsToAlert": "[parameters('minFailingPeriodsToAlert')]"
},
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template with the parameter file below. Save and modify the json below as all-vms-in-
subscription-dynamic.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "Multi-resource sub level metric alert with Dynamic Thresholds via Azure Resource Manager
template"
},
"alertDescription": {
"value": "New Multi-resource sub level metric alert with Dynamic Thresholds created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"targetSubscription":{
"value": "/subscriptions/replace-with-subscription-id"
},
"targetResourceRegion":{
"value": "SouthCentralUS"
},
"targetResourceType":{
"value": "Microsoft.Compute/virtualMachines"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterOrLessThan"
},
"alertSensitivity": {
"value": "Medium"
},
"numberOfEvaluationPeriods": {
"value": "4"
},
"minFailingPeriodsToAlert": {
"value": "3"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
az login
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"targetResourceId":{
"type": "array",
"minLength": 1,
"metadata": {
"description": "array of Azure resource Ids. For example - /subscriptions/00000000-0000-0000-
0000-0000-00000000/resourceGroup/resource-group-name/Microsoft.compute/virtualMachines/vm-name"
}
},
"targetResourceRegion":{
"type": "string",
"allowedValues": [
"EastUS",
"EastUS2",
"CentralUS",
"NorthCentralUS",
"SouthCentralUS",
"WestCentralUS",
"WestUS",
"WestUS2",
"CanadaEast",
"CanadaCentral",
"BrazilSouth",
"NorthEurope",
"WestEurope",
"FranceCentral",
"FranceSouth",
"UKWest",
"UKSouth",
"GermanyCentral",
"GermanyNortheast",
"GermanyNorth",
"GermanyWestCentral",
"SwitzerlandNorth",
"SwitzerlandWest",
"NorwayEast",
"NorwayWest",
"SoutheastAsia",
"EastAsia",
"AustraliaEast",
"AustraliaSoutheast",
"AustraliaCentral",
"AustraliaCentral2",
"ChinaEast",
"ChinaNorth",
"ChinaEast2",
"ChinaNorth2",
"CentralIndia",
"WestIndia",
"SouthIndia",
"JapanEast",
"JapanWest",
"KoreaCentral",
"KoreaSouth",
"SouthAfricaWest",
"SouthAfricaNorth",
"UAECentral",
"UAENorth"
],
"metadata": {
"description": "Azure region in which target resources to be monitored are in (without
spaces). For example: EastUS"
}
},
"targetResourceType": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Resource type of target resources to be monitored. Currently only supported
resource type is Microsoft.Compute/virtualMachines"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"Equals",
"NotEquals",
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "0",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H",
"PT6H",
"PT12H",
"PT24H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between one minute and one day. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT1M",
"allowedValues": [
"PT1M",
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": "[parameters('targetResourceId')]",
"targetResourceType": "[parameters('targetResourceType')]",
"targetResourceRegion": "[parameters('targetResourceRegion')]",
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"threshold" : "[parameters('threshold')]",
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template with the parameter file below. Save and modify the json below as list-of-vms-
static.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "Multi-resource metric alert by list via Azure Resource Manager template"
},
"alertDescription": {
"value": "New Multi-resource metric alert by list created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"targetResourceId":{
"value": [
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name1/Microsoft.Compute/virtualMachines/replace-with-vm-name1",
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name2/Microsoft.Compute/virtualMachines/replace-with-vm-name2"
]
},
"targetResourceRegion":{
"value": "SouthCentralUS"
},
"targetResourceType":{
"value": "Microsoft.Compute/virtualMachines"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterThan"
},
"threshold": {
"value": "0"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "This is a metric alert",
"metadata": {
"description": "Description of alert"
}
},
"alertSeverity": {
"type": "int",
"defaultValue": 3,
"allowedValues": [
0,
1,
2,
3,
4
],
"metadata": {
"description": "Severity of alert {0,1,2,3,4}"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether the alert is enabled"
}
},
"targetResourceId":{
"type": "array",
"minLength": 1,
"metadata": {
"description": "array of Azure resource Ids. For example - /subscriptions/00000000-0000-0000-
0000-0000-00000000/resourceGroup/resource-group-name/Microsoft.compute/virtualMachines/vm-name"
}
},
"targetResourceRegion":{
"type": "string",
"allowedValues": [
"EastUS",
"EastUS2",
"CentralUS",
"NorthCentralUS",
"SouthCentralUS",
"WestCentralUS",
"WestUS",
"WestUS2",
"CanadaEast",
"CanadaCentral",
"BrazilSouth",
"NorthEurope",
"WestEurope",
"FranceCentral",
"FranceSouth",
"UKWest",
"UKSouth",
"GermanyCentral",
"GermanyNortheast",
"GermanyNorth",
"GermanyWestCentral",
"SwitzerlandNorth",
"SwitzerlandWest",
"NorwayEast",
"NorwayWest",
"SoutheastAsia",
"EastAsia",
"AustraliaEast",
"AustraliaSoutheast",
"AustraliaCentral",
"AustraliaCentral2",
"ChinaEast",
"ChinaNorth",
"ChinaEast2",
"ChinaNorth2",
"CentralIndia",
"WestIndia",
"SouthIndia",
"JapanEast",
"JapanWest",
"KoreaCentral",
"KoreaSouth",
"SouthAfricaWest",
"SouthAfricaNorth",
"UAECentral",
"UAENorth"
],
"metadata": {
"description": "Azure region in which target resources to be monitored are in (without
spaces). For example: EastUS"
}
},
"targetResourceType": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Resource type of target resources to be monitored. Currently only supported
resource type is Microsoft.Compute/virtualMachines"
}
},
"metricName": {
"type": "string",
"minLength": 1,
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterOrLessThan",
"defaultValue": "GreaterOrLessThan",
"allowedValues": [
"GreaterThan",
"LessThan",
"GreaterOrLessThan"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"alertSensitivity": {
"type": "string",
"defaultValue": "Medium",
"allowedValues": [
"High",
"Medium",
"Low"
],
"metadata": {
"description": "Tunes how 'noisy' the Dynamic Thresholds alerts will be: 'High' will result in
more alerts while 'Low' will result in fewer alerts."
}
},
"numberOfEvaluationPeriods": {
"type": "string",
"defaultValue": "4",
"metadata": {
"description": "The number of periods to check in the alert evaluation."
}
},
"minFailingPeriodsToAlert": {
"type": "string",
"defaultValue": "3",
"metadata": {
"description": "The number of unhealthy periods to alert on (must be lower or equal to
numberOfEvaluationPeriods)."
}
},
"timeAggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Minimum",
"Maximum",
"Total",
"Count"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one hour. ISO 8601 duration format."
}
},
"evaluationFrequency": {
"type": "string",
"defaultValue": "PT5M",
"allowedValues": [
"allowedValues": [
"PT5M",
"PT15M",
"PT30M",
"PT1H"
],
"metadata": {
"description": "how often the metric alert is evaluated represented in ISO 8601 duration
format"
}
},
"actionGroupId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The ID of the action group that is triggered when the alert is activated or
deactivated"
}
}
},
"variables": { },
"resources": [
{
"name": "[parameters('alertName')]",
"type": "Microsoft.Insights/metricAlerts",
"location": "global",
"apiVersion": "2018-03-01",
"tags": {},
"properties": {
"description": "[parameters('alertDescription')]",
"severity": "[parameters('alertSeverity')]",
"enabled": "[parameters('isEnabled')]",
"scopes": "[parameters('targetResourceId')]",
"targetResourceType": "[parameters('targetResourceType')]",
"targetResourceRegion": "[parameters('targetResourceRegion')]",
"evaluationFrequency":"[parameters('evaluationFrequency')]",
"windowSize": "[parameters('windowSize')]",
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.MultipleResourceMultipleMetricCriteria",
"allOf": [
{
"criterionType": "DynamicThresholdCriterion",
"name" : "1st criterion",
"metricName": "[parameters('metricName')]",
"dimensions":[],
"operator": "[parameters('operator')]",
"alertSensitivity": "[parameters('alertSensitivity')]",
"failingPeriods": {
"numberOfEvaluationPeriods": "[parameters('numberOfEvaluationPeriods')]",
"minFailingPeriodsToAlert": "[parameters('minFailingPeriodsToAlert')]"
},
"timeAggregation": "[parameters('timeAggregation')]"
}
]
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can use the above template with the parameter file below. Save and modify the json below as list-of-vms-
dynamic.parameters.json for the purpose of this walkthrough.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"value": "Multi-resource metric alert with Dynamic Thresholds by list via Azure Resource Manager
template"
},
"alertDescription": {
"value": "New Multi-resource metric alert with Dynamic Thresholds by list created via template"
},
"alertSeverity": {
"value":3
},
"isEnabled": {
"value": true
},
"targetResourceId":{
"value": [
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name1/Microsoft.Compute/virtualMachines/replace-with-vm-name1",
"/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name2/Microsoft.Compute/virtualMachines/replace-with-vm-name2"
]
},
"targetResourceRegion":{
"value": "SouthCentralUS"
},
"targetResourceType":{
"value": "Microsoft.Compute/virtualMachines"
},
"metricName": {
"value": "Percentage CPU"
},
"operator": {
"value": "GreaterOrLessThan"
},
"alertSensitivity": {
"value": "Medium"
},
"numberOfEvaluationPeriods": {
"value": "4"
},
"minFailingPeriodsToAlert": {
"value": "3"
},
"timeAggregation": {
"value": "Average"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resource-group-
name/providers/Microsoft.Insights/actionGroups/replace-with-action-group-name"
}
}
}
You can create the metric alert using the template and parameters file using PowerShell or Azure CLI from your
current working directory.
Using Azure PowerShell
Connect-AzAccount
az login
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"appName": {
"type": "string"
},
"pingURL": {
"type": "string"
},
"pingText": {
"type": "string",
"defaultValue": ""
},
"actionGroupId": {
"type": "string"
},
"location": {
"type": "string"
}
},
"variables": {
"pingTestName": "[concat('PingTest-', toLower(parameters('appName')))]",
"pingAlertRuleName": "[concat('PingAlert-', toLower(parameters('appName')), '-',
subscription().subscriptionId)]"
},
"resources": [
{
"name": "[variables('pingTestName')]",
"type": "Microsoft.Insights/webtests",
"apiVersion": "2014-04-01",
"location": "[parameters('location')]",
"tags": {
"[concat('hidden-link:', resourceId('Microsoft.Insights/components', parameters('appName')))]":
"[concat('hidden-link:', resourceId('Microsoft.Insights/components', parameters('appName')))]":
"Resource"
},
"properties": {
"Name": "[variables('pingTestName')]",
"Description": "Basic ping test",
"Enabled": true,
"Frequency": 300,
"Timeout": 120,
"Kind": "ping",
"RetryEnabled": true,
"Locations": [
{
"Id": "us-va-ash-azr"
},
{
"Id": "emea-nl-ams-azr"
},
{
"Id": "apac-jp-kaw-edge"
}
],
"Configuration": {
"WebTest": "[concat('<WebTest Name=\"', variables('pingTestName'), '\" Enabled=\"True\"
CssProjectStructure=\"\" CssIteration=\"\" Timeout=\"120\" WorkItemIds=\"\"
xmlns=\"http://microsoft.com/schemas/VisualStudio/TeamTest/2010\" Description=\"\"
CredentialUserName=\"\" CredentialPassword=\"\" PreAuthenticate=\"True\" Proxy=\"default\"
StopOnError=\"False\" RecordedResultFile=\"\" ResultsLocale=\"\"> <Items> <Request Method=\"GET\"
Version=\"1.1\" Url=\"', parameters('pingURL'), '\" ThinkTime=\"0\" Timeout=\"300\"
ParseDependentRequests=\"True\" FollowRedirects=\"True\" RecordResult=\"True\" Cache=\"False\"
ResponseTimeGoal=\"0\" Encoding=\"utf-8\" ExpectedHttpStatusCode=\"200\" ExpectedResponseUrl=\"\"
ReportingName=\"\" IgnoreHttpStatusCode=\"False\" /> </Items> <ValidationRules> <ValidationRule
Classname=\"Microsoft.VisualStudio.TestTools.WebTesting.Rules.ValidationRuleFindText,
Microsoft.VisualStudio.QualityTools.WebTestFramework, Version=10.0.0.0, Culture=neutral,
PublicKeyToken=b03f5f7f11d50a3a\" DisplayName=\"Find Text\" Description=\"Verifies the existence of
the specified text in the response.\" Level=\"High\" ExecutionOrder=\"BeforeDependents\">
<RuleParameters> <RuleParameter Name=\"FindText\" Value=\"', parameters('pingText'), '\" />
<RuleParameter Name=\"IgnoreCase\" Value=\"False\" /> <RuleParameter Name=\"UseRegularExpression\"
Value=\"False\" /> <RuleParameter Name=\"PassIfTextFound\" Value=\"True\" /> </RuleParameters>
</ValidationRule> </ValidationRules> </WebTest>')]"
},
"SyntheticMonitorId": "[variables('pingTestName')]"
}
},
{
"name": "[variables('pingAlertRuleName')]",
"type": "Microsoft.Insights/metricAlerts",
"apiVersion": "2018-03-01",
"location": "global",
"dependsOn": [
"[resourceId('Microsoft.Insights/webtests', variables('pingTestName'))]"
],
"tags": {
"[concat('hidden-link:', resourceId('Microsoft.Insights/components', parameters('appName')))]":
"Resource",
"[concat('hidden-link:', resourceId('Microsoft.Insights/webtests', variables('pingTestName')))]":
"Resource"
},
"properties": {
"description": "Alert for web test",
"severity": 1,
"enabled": true,
"scopes": [
"[resourceId('Microsoft.Insights/webtests',variables('pingTestName'))]",
"[resourceId('Microsoft.Insights/components',parameters('appName'))]"
],
"evaluationFrequency": "PT1M",
"windowSize": "PT5M",
"templateType": 0,
"criteria": {
"criteria": {
"odata.type": "Microsoft.Azure.Monitor.WebtestLocationAvailabilityCriteria",
"webTestId": "[resourceId('Microsoft.Insights/webtests', variables('pingTestName'))]",
"componentId": "[resourceId('Microsoft.Insights/components', parameters('appName'))]",
"failedLocationCount": 2
},
"actions": [
{
"actionGroupId": "[parameters('actionGroupId')]"
}
]
}
}
]
}
You can set the values for the parameters either on the command line or through a parameter file. A sample
parameter file is provided below.
NOTE
& ; is the HTML entity reference for &. URL parameters are still separated by a single &, but if you mention the URL in
HTML, you need to encode it. So, if you have any "&" in your pingURL parameter value, you have to escape it with " & ;"
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"appName": {
"value": "Replace with your Application Insights resource name"
},
"pingURL": {
"value": "https://www.yoursite.com"
},
"actionGroupId": {
"value": "/subscriptions/replace-with-subscription-id/resourceGroups/replace-with-resourceGroup-
name/providers/microsoft.insights/actiongroups/replace-with-action-group-name"
},
"location": {
"value": "Replace with the location of your Application Insights resource"
}
}
}
You can create the availability test and associated alert using the template and parameters file using PowerShell or
Azure CLI.
Using Azure PowerShell
Connect-AzAccount
Next steps
Read more about alerts in Azure
Learn how to create an action group with Resource Manager templates
For the JSON syntax and properties, see Microsoft.Insights/metricAlerts template reference.
Create, view, and manage log alerts using Azure
Monitor
1/20/2020 • 11 minutes to read • Edit Online
Overview
This article shows you how to set up log alerts using the alerts interface inside Azure portal. Definition of an
alert rule is in three parts:
Target: Specific Azure resource, which is to be monitored
Criteria: Specific condition or logic that when seen in Signal, should trigger action
Action: Specific call sent to a receiver of a notification - email, SMS, webhook etc.
The term Log Alerts to describe alerts where signal is log query in a Log Analytics workspace or Application
Insights. Learn more about functionality, terminology, and types from Log alerts - Overview.
NOTE
Popular log data from a Log Analytics workspace is now also available on the metric platform in Azure Monitor. For
details view, Metric Alert for Logs
2. Select the New Alert Rule button to create a new alert in Azure.
3. The Create Alert section is shown with the three parts consisting of: Define alert condition, Define alert
details, and Define action group.
4. Define the alert condition by using the Select Resource link and specifying the target by selecting a
resource. Filter by choosing the Subscription, Resource Type, and required Resource.
NOTE
For creating a log alert - verify the log signal is available for the selected resource before you proceed.
5. Log Alerts: Ensure Resource Type is an analytics source like Log Analytics or Application Insights and
signal type as Log, then once appropriate resource is chosen, click Done. Next use the Add criteria
button to view list of signal options available for the resource and from the signal list Custom log
search option for chosen log monitor service like Log Analytics or Application Insights.
NOTE
Alerts lists can import analytics query as signal type - Log (Saved Query), as seen in above illustration. So users
can perfect your query in Analytics and then save them for future use in alerts - more details on using saving
query available at using log query in Azure Monitor or shared query in application insights analytics.
6. Log Alerts: Once selected, query for alerting can be stated in Search Query field; if the query syntax is
incorrect the field displays error in RED. If the query syntax is correct - For reference historic data of the
stated query is shown as a graph with option to tweak the time window from last six hours to last week.
NOTE
Historical data visualization can only be shown if the query results have time details. If your query results in
summarized data or specific column values - same is shown as a singular plot. For Metric Measurement type of
Log Alerts using Application Insights or switched to new API, you can specify which specific variable to group the
data by using the Aggregate on option; as illustrated in below:
7. Log Alerts: With the visualization in place, Alert Logic can be selected from shown options of
Condition, Aggregation and finally Threshold. Finally specify in the logic, the time to assess for the
specified condition, using Period option. Along with how often Alert should run by selecting
Frequency. Log Alerts can be based on:
Number of Records: An alert is created if the count of records returned by the query is either greater
than or less than the value provided.
Metric Measurement: An alert is created if each aggregate value in the results exceeds the threshold
value provided and it is grouped by chosen value. The number of breaches for an alert is the number
of times the threshold is exceeded in the chosen time period. You can specify Total breaches for any
combination of breaches across the results set or Consecutive breaches to require that the breaches
must occur in consecutive samples.
8. As the second step, define a name for your alert in the Alert rule name field along with a Description
detailing specifics for the alert and Severity value from the options provided. These details are reused
in all alert emails, notifications, or push done by Azure Monitor. Additionally, user can choose to
immediately activate the alert rule on creation by appropriately toggling Enable rule upon creation
option.
For Log Alerts only, some additional functionality is available in Alert details:
Suppress Alerts: When you turn on suppression for the alert rule, actions for the rule are
disabled for a defined length of time after creating a new alert. The rule is still running and
creates alert records provided the criteria is met. Allowing you time to correct the problem
without running duplicate actions.
TIP
Specify an suppress alert value greater than frequency of alert to ensure notifications are stopped
without overlap
9. As the third and final step, specify if any Action Group needs to be triggered for the alert rule when
alert condition is met. You can choose any existing Action Group with alert or create a new Action
Group. According to selected Action Group, when alert is trigger Azure will: send email(s), send SMS (s),
call Webhook(s), remediate using Azure Runbooks, push to your ITSM tool, etc. Learn more about
Action Groups.
NOTE
Refer to the Azure subscription service limits for limits on Runbook payloads triggered for log alerts via Azure
action groups
For Log Alerts some additional functionality is available to override the default Actions:
Email Notification: Overrides e-mail subject in the email, sent via Action Group; if one or more
email actions exist in the said Action Group. You cannot modify the body of the mail and this field
is not for email address.
Include custom Json payload: Overrides the webhook JSON used by Action Groups; if one or
more webhook actions exist in the said Action Group. User can specify format of JSON to be
used for all webhooks configured in associated Action Group; for more information on webhook
formats, see webhook action for Log Alerts. View Webhook option is provided to check format
using sample JSON data.
10. If all fields are valid and with green tick the create alert rule button can be clicked and an alert is
created in Azure Monitor - Alerts. All alerts can be viewed from the alerts Dashboard.
Within a few minutes, the alert is active and triggers as previously described.
Users can also finalize their analytics query in log analytics and then push it to create an alert via 'Set Alert'
button - then following instructions from Step 6 onwards in the above tutorial.
NOTE
Log alert rules comprise of custom query-based logic provided by users and hence without a resolved state. Due
to which every time the conditions specified in the log alert rule are met, it is fired.
3. Select the Manage rules button on the top bar, to navigate to the rule management section - where all
alert rules created are listed; including alerts that have been disabled.
NOTE
Log alerts for Log Analytics can also be managed using legacy Log Analytics Alert API and legacy templates of Log
Analytics saved searches and alerts as well. For more information on using the new ScheduledQueryRules API detailed
here by default, see Switch to new API for Log Analytics Alerts.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"alertLocation": "southcentralus",
"alertName": "samplelogalert",
"alertDescription": "Sample log search alert",
"alertStatus": "true",
"alertSource":{
"Query":"requests",
"SourceId": "/subscriptions/a123d7efg-123c-1234-5678-
a12bc3defgh4/resourceGroups/myRG/providers/microsoft.insights/components/sampleAIapplication",
"Type":"ResultCount"
},
"alertSchedule":{
"Frequency": 15,
"Time": 60
},
"alertActions":{
"SeverityLevel": "4"
},
"alertTrigger":{
"Operator":"GreaterThan",
"Threshold":"1"
},
"actionGrp":{
"ActionGroup": "/subscriptions/a123d7efg-123c-1234-5678-
a12bc3defgh4/resourceGroups/myRG/providers/microsoft.insights/actiongroups/sampleAG",
"Subject": "Customized Email Header",
"Webhook": "{ \"alertname\":\"#alertrulename\", \"IncludeSearchResults\":true }"
}
},
"resources":[ {
"name":"[variables('alertName')]",
"type":"Microsoft.Insights/scheduledQueryRules",
"apiVersion": "2018-04-16",
"location": "[variables('alertLocation')]",
"properties":{
"description": "[variables('alertDescription')]",
"enabled": "[variables('alertStatus')]",
"source": {
"query": "[variables('alertSource').Query]",
"dataSourceId": "[variables('alertSource').SourceId]",
"queryType":"[variables('alertSource').Type]"
},
"schedule":{
"frequencyInMinutes": "[variables('alertSchedule').Frequency]",
"timeWindowInMinutes": "[variables('alertSchedule').Time]"
},
"action":{
"odata.type":
"Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.Microsoft.AppInsights.Nexus.DataContracts.Resou
rces.ScheduledQueryRules.AlertingAction",
"severity":"[variables('alertActions').SeverityLevel]",
"aznsAction":{
"actionGroup":"[array(variables('actionGrp').ActionGroup)]",
"emailSubject":"[variables('actionGrp').Subject]",
"customWebhookPayload":"[variables('actionGrp').Webhook]"
},
"trigger":{
"thresholdOperator":"[variables('alertTrigger').Operator]",
"threshold":"[variables('alertTrigger').Threshold]"
}
}
}
} ]
}
The sample json above can be saved as (say) sampleScheduledQueryRule.json for the purpose of this walk
through and can be deployed using Azure Resource Manager in Azure portal.
Log alert with cross-resource query using Azure Resource Template
The following is the structure for Scheduled Query Rules creation based resource template using cross-
resource log search query of metric measurement type log alert, with sample data set as variables.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"alertLocation": "Region Name for your Application Insights App or Log Analytics Workspace",
"alertName": "sample log alert",
"alertDescr": "Sample log search alert",
"alertStatus": "true",
"alertSource":{
"Query":"union workspace(\"servicews\").Update, app('serviceapp').requests | summarize
AggregatedValue = count() by bin(TimeGenerated,1h), Classification",
"Resource1": "/subscriptions/a123d7efg-123c-1234-5678-
a12bc3defgh4/resourceGroups/contosoRG/providers/microsoft.OperationalInsights/workspaces/servicews",
"Resource2": "/subscriptions/a123d7efg-123c-1234-5678-
a12bc3defgh4/resourceGroups/contosoRG/providers/microsoft.insights/components/serviceapp",
"SourceId": "/subscriptions/a123d7efg-123c-1234-5678-
"SourceId": "/subscriptions/a123d7efg-123c-1234-5678-
a12bc3defgh4/resourceGroups/contosoRG/providers/microsoft.OperationalInsights/workspaces/servicews",
"Type":"ResultCount"
},
"alertSchedule":{
"Frequency": 15,
"Time": 60
},
"alertActions":{
"SeverityLevel": "4",
"SuppressTimeinMin": 20
},
"alertTrigger":{
"Operator":"GreaterThan",
"Threshold":"1"
},
"metricMeasurement": {
"thresholdOperator": "Equal",
"threshold": "1",
"metricTriggerType": "Consecutive",
"metricColumn": "Classification"
},
"actionGrp":{
"ActionGroup": "/subscriptions/a123d7efg-123c-1234-5678-
a12bc3defgh4/resourceGroups/contosoRG/providers/microsoft.insights/actiongroups/sampleAG",
"Subject": "Customized Email Header",
"Webhook": "{ \"alertname\":\"#alertrulename\", \"IncludeSearchResults\":true }"
}
},
"resources":[ {
"name":"[variables('alertName')]",
"type":"Microsoft.Insights/scheduledQueryRules",
"apiVersion": "2018-04-16",
"location": "[variables('alertLocation')]",
"properties":{
"description": "[variables('alertDescr')]",
"enabled": "[variables('alertStatus')]",
"source": {
"query": "[variables('alertSource').Query]",
"authorizedResources": "[concat(array(variables('alertSource').Resource1),
array(variables('alertSource').Resource2))]",
"dataSourceId": "[variables('alertSource').SourceId]",
"queryType":"[variables('alertSource').Type]"
},
"schedule":{
"frequencyInMinutes": "[variables('alertSchedule').Frequency]",
"timeWindowInMinutes": "[variables('alertSchedule').Time]"
},
"action":{
"odata.type":
"Microsoft.WindowsAzure.Management.Monitoring.Alerts.Models.Microsoft.AppInsights.Nexus.DataContracts.Resou
rces.ScheduledQueryRules.AlertingAction",
"severity":"[variables('alertActions').SeverityLevel]",
"throttlingInMin": "[variables('alertActions').SuppressTimeinMin]",
"aznsAction":{
"actionGroup": "[array(variables('actionGrp').ActionGroup)]",
"emailSubject":"[variables('actionGrp').Subject]",
"customWebhookPayload":"[variables('actionGrp').Webhook]"
},
"trigger":{
"thresholdOperator":"[variables('alertTrigger').Operator]",
"threshold":"[variables('alertTrigger').Threshold]",
"metricTrigger":{
"thresholdOperator": "[variables('metricMeasurement').thresholdOperator]",
"threshold": "[variables('metricMeasurement').threshold]",
"metricColumn": "[variables('metricMeasurement').metricColumn]",
"metricTriggerType": "[variables('metricMeasurement').metricTriggerType]"
}
}
}
}
}
} ]
}
IMPORTANT
When using cross-resource query in log alert, the usage of authorizedResources is mandatory and user must have
access to the list of resources stated
The sample json above can be saved as (say) sampleScheduledQueryRule.json for the purpose of this walk
through and can be deployed using Azure Resource Manager in Azure portal.
Azure Monitor - Scheduled Query Rules API is a REST API and fully compatible with Azure Resource
Manager REST API. And PowerShell cmdlets listed below are available to leverage the Scheduled Query Rules
API.
1. New -AzScheduledQueryRule : Powershell cmdlet to create a new log alert rule.
2. Set-AzScheduledQueryRule : Powershell cmdlet to update an existing log alert rule.
3. New -AzScheduledQueryRuleSource : Powershell cmdlet to create or update object specifying source
parameters for a log alert. Used as input by New -AzScheduledQueryRule and Set-AzScheduledQueryRule
cmdlet.
4. New -AzScheduledQueryRuleSchedule: Powershell cmdlet to create or update object specifying schedule
parameters for a log alert. Used as input by New -AzScheduledQueryRule and Set-AzScheduledQueryRule
cmdlet.
5. New -AzScheduledQueryRuleAlertingAction : Powershell cmdlet to create or update object specifying action
parameters for a log alert. Used as input by New -AzScheduledQueryRule and Set-AzScheduledQueryRule
cmdlet.
6. New -AzScheduledQueryRuleAznsActionGroup : Powershell cmdlet to create or update object specifying
action groups parameters for a log alert. Used as input by New -AzScheduledQueryRuleAlertingAction
cmdlet.
7. New -AzScheduledQueryRuleTriggerCondition : Powershell cmdlet to create or update object specifying
trigger condition parameters for log alert. Used as input by New -AzScheduledQueryRuleAlertingAction
cmdlet.
8. New -AzScheduledQueryRuleLogMetricTrigger : Powershell cmdlet to create or update object specifying
metric trigger condition parameters for metric measurement type log alert. Used as input by New -
AzScheduledQueryRuleTriggerCondition cmdlet.
9. Get-AzScheduledQueryRule : Powershell cmdlet to list existing log alert rules or a specific log alert rule
10. Update-AzScheduledQueryRule : Powershell cmdlet to enable or disable log alert rule
11. Remove-AzScheduledQueryRule: Powershell cmdlet to delete an existing log alert rule
NOTE
ScheduledQueryRules PowerShell cmdlets can only manage rules created cmdlet itself or using Azure Monitor -
Scheduled Query Rules API. Log alert rules created using legacy Log Analytics Alert API and legacy templates of Log
Analytics saved searches and alerts can be managed using ScheduledQueryRules PowerShell cmdlets only after user
switches API preference for Log Analytics Alerts.
Illustrated next are the steps for creation of a sample log alert rule using the scheduledQueryRules PowerShell
cmdlets.
NOTE
Log alerts for Log Analytics can also be managed using legacy Log Analytics Alert API and legacy templates of Log
Analytics saved searches and alerts as well. For more information on using the new ScheduledQueryRules API detailed
here by default, see Switch to new API for Log Analytics Alerts.
Log alerts currently do not have dedicated CLI commands currently; but as illustrated below can be used via
Azure Resource Manager CLI command for sample Resource Template shown earlier
(sampleScheduledQueryRule.json) in the Resource Template section:
On successful operation, 201 will be returned to state new alert rule creation or 200 will be returned if an
existing alert rule was modified.
Next steps
Learn about Log Alerts in Azure Alerts
Understand Webhook actions for log alerts
Learn more about Application Insights
Learn more about log queries.
Log alert queries in Azure Monitor
4/4/2019 • 6 minutes to read • Edit Online
Alert rules based on Azure Monitor logs run at regular intervals, so you should ensure that they are written to
minimize overhead and latency. This article provides recommendations on writing efficient queries for log alerts
and a process for converting existing queries.
Queries that start with search or union allow you to search across multiple columns in a table or even multiple
tables. The following examples show multiple methods for searching the term Memory:
search "Memory"
search * | where == "Memory"
search ObjectName: "Memory"
search ObjectName == "Memory"
union * | where ObjectName == "Memory"
Although search and union are useful during data exploration, searching terms over the entire data model, they
are less efficient than using a table since they must scan across multiple tables. Since queries in alert rules are run
at regular intervals, this can result in excessive overhead adding latency to the alert. Because of this overhead,
queries for log alert rules in Azure should always start with a table to define a clear scope, which improves both
query performance and the relevance of the results.
Unsupported queries
Starting January 11,2019, creating or modifyinglog alert rules that use search , or union operators will not be
supported the inAzure portal. Using these operators in an alert rule will return an error message. Existing alert
rules and alert rules created and edited with the Log Analytics API are not affected by this change. You should still
consider changing any alert rules that use these types of queries though to improve their efficiency.
Log alert rules using cross-resource queries are not affected by this change since cross-resource queriesuse union
, which limits the query scope to specific resources. This is not equivalent of union * which cannot be used. The
following example would be valid in a log alert rule:
union
app('Contoso-app1').requests,
app('Contoso-app2').requests,
workspace('Contoso-workspace1').Perf
NOTE
Cross-resource query in log alerts is supported in the new scheduledQueryRules API. By default, Azure Monitor uses the
legacy Log Analytics Alert API for creating new log alert rules from Azure portal, unless you switch from legacy Log Alerts
API. After the switch, the new API becomes the default for new alert rules in Azure portal and it lets you create cross-
resource query log alerts rules. You can create cross-resource query log alert rules without making the switch by using the
ARM template for scheduledQueryRules API – but this alert rule is manageable though scheduledQueryRules API and not
from Azure portal.
Examples
The following examples include log queries that use search and union and provide steps you can use to modify
these queries for use with alert rules.
Example 1
You want to create a log alert rule using the following query which retrieves performance information using
search :
To modify this query, start by using the following query to identify the table that the properties belong to:
The result of this query would show that the CounterName property came from the Perf table.
You can use this result to create the following query which you would use for the alert rule:
Perf
| where CounterName == '% Free Space'
| where CounterValue < 30
| summarize count()
Example 2
You want to create a log alert rule using the following query which retrieves performance information using
search :
To modify this query, start by using the following query to identify the table that the properties belong to:
The result of this query would show that the ObjectName and CounterName property came from the Perf table.
You can use this result to create the following query which you would use for the alert rule:
Perf
| where ObjectName =="Memory" and CounterName=="% Committed Bytes In Use"
| summarize Avg_Memory_Usage=avg(CounterValue) by Computer
| where Avg_Memory_Usage between(90 .. 95)
| count
Example 3
You want to create a log alert rule using the following query which uses both search and union to retrieve
performance information:
search (ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total")
| where Computer !in ((union * | where CounterName == "% Processor Utility" | summarize by Computer))
| summarize Avg_Idle_Time = avg(CounterValue) by Computer| count
To modify this query, start by using the following query to identify the table that the properties in the first part of
the query belong to:
search (ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total")
| summarize by $table
The result of this query would show that all these properties came from the Perf table.
Now use union with withsource command to identify which source table has contributed each row.
The result of this query would show that these properties also came from the Perf table.
You can use these results to create the following query which you would use for the alert rule:
Perf
| where ObjectName == "Processor" and CounterName == "% Idle Time" and InstanceName == "_Total"
| where Computer !in (
(Perf
| where CounterName == "% Processor Utility"
| summarize by Computer))
| summarize Avg_Idle_Time = avg(CounterValue) by Computer
| count
Example 4
You want to create a log alert rule using the following query which joins the results of two search queries:
To modify the query, start by using the following query to identify the table that contains the properties in the left
side of the join:
search Type == 'SecurityEvent' and EventID == '4625'
| summarize by $table
The result indicates that the properties in the left side of the join belong to SecurityEvent table.
Now use the following query to identify the table that contains the properties in the right side of the join:
The result indicates that the properties in the right side of the join belong to Heartbeat table.
You can use these results to create the following query which you would use for the alert rule:
SecurityEvent
| where EventID == '4625'
| summarize by Computer, Hour = bin(TimeGenerated, 1h)
| join kind = leftouter (
Heartbeat
| where OSType == 'Windows'
| summarize arg_max(TimeGenerated, Computer) by Computer , Hour = bin(TimeGenerated, 1h)
| project Hour , Computer
)
on Hour
| count
Next steps
Learn about log alerts in Azure Monitor.
Learn about log queries.
Webhook actions for log alert rules
10/3/2019 • 5 minutes to read • Edit Online
When a log alert is created in Azure, you have the option of configuring it by using action groups to perform one
or more actions. This article describes the different webhook actions that are available and shows how to
configure a custom JSON -based webhook.
NOTE
You also can use the common alert schema for your webhook integrations. The common alert schema provides the
advantage of having a single extensible and unified alert payload across all the alert services in Azure Monitor.Please note
that the common alert schema does not honour the custom JSON option for log alerts. It defers to the common alert
schema payload if that is selected irrespective of the customization you might have done at the alert rule level. Learn about
the common alert schema definitions.
Webhook actions
With webhook actions, you can invoke an external process through a single HTTP POST request. The service
that's called should support webhooks and determine how to use any payload it receives.
Webhook actions require the properties in the following table.
PROPERTY DESCRIPTION
Custom JSON payload The custom payload to send with the webhook when this
option is chosen during alert creation. For more information,
see Manage log alerts.
NOTE
The View Webhook button alongside the Include custom JSON payload for webhook option for the log alert displays
the sample webhook payload for the customization that was provided. It doesn't contain actual data but is representative of
the JSON schema that's used for log alerts.
Webhooks include a URL and a payload formatted in JSON that the data sent to the external service. By default,
the payload includes the values in the following table. You can choose to replace this payload with a custom one of
your own. In that case, use the variables in the table for each of the parameters to include their values in your
custom payload.
Search Interval End time #searchintervalendtimeutc End time for the query in UTC, with the
format mm/dd/yyyy HH:mm:ss AM/PM.
Search Interval #searchinterval Time window for the alert rule, with the
format HH:mm:ss.
Search Interval StartTime #searchintervalstarttimeutc Start time for the query in UTC, with
the format mm/dd/yyyy HH:mm:ss
AM/PM.
NOTE
LinkToSearchResults passes parameters like SearchQuery, Search Interval StartTime, and Search Interval End time in the
URL to the Azure portal for viewing in the Analytics section. The Azure portal has a URI size limit of approximately 2,000
characters. The portal will not open links provided in alerts if the parameter values exceed the limit. You can manually input
details to view results in the Analytics portal. Or, you can use the Application Insights Analytics REST API or the Log
Analytics REST API to retrieve results programmatically.
For example, you might specify the following custom payload that includes a single parameter called text. The
service that this webhook calls expects this parameter.
{
"text":"#alertrulename fired with #searchresultcount over threshold of #thresholdvalue."
}
This example payload resolves to something like the following when it's sent to the webhook:
{
"text":"My Alert Rule fired with 18 records over threshold of 10 ."
}
Because all variables in a custom webhook must be specified within a JSON enclosure, like "#searchinterval," the
resultant webhook also has variable data inside enclosures, like "00:05:00."
To include search results in a custom payload, ensure that IncludeSearchResults is set as a top-level property in
the JSON payload.
Sample payloads
This section shows sample payloads for webhooks for log alerts. The sample payloads include examples when the
payload is standard and when it's custom.
Standard webhook for log alerts
Both of these examples have a dummy payload with only two columns and two rows.
Log alert for Log Analytics
The following sample payload is for a standard webhook action without a custom JSON option that's used for
alerts based on Log Analytics:
{
"SubscriptionId":"12345a-1234b-123c-123d-12345678e",
"AlertRuleName":"AcmeRule",
"SearchQuery":"Perf | where ObjectName == \"Processor\" and CounterName == \"% Processor Time\" |
summarize AggregatedValue = avg(CounterValue) by bin(TimeGenerated, 5m), Computer",
"SearchIntervalStartTimeUtc": "2018-03-26T08:10:40Z",
"SearchIntervalEndtimeUtc": "2018-03-26T09:10:40Z",
"AlertThresholdOperator": "Greater Than",
"AlertThresholdValue": 0,
"ResultCount": 2,
"SearchIntervalInSeconds": 3600,
"LinkToSearchResults": "https://portal.azure.com/#Analyticsblade/search/index?
_timeInterval.intervalEnd=2018-03-26T09%3a10%3a40.0000000Z&_timeInterval.intervalDuration=3600&q=Usage",
"Description": "log alert rule",
"Severity": "Warning",
"SearchResult":
{
"tables":[
{"name":"PrimaryResult","columns":
[
{"name":"$table","type":"string"},
{"name":"Id","type":"string"},
{"name":"TimeGenerated","type":"datetime"}
],
"rows":
[
["Fabrikam","33446677a","2018-02-02T15:03:12.18Z"],
["Contoso","33445566b","2018-02-02T15:16:53.932Z"]
]
}
]
},
"WorkspaceId":"12345a-1234b-123c-123d-12345678e",
"AlertType": "Metric measurement"
}
NOTE
The "Severity" field value might change if you've switched your API preference for log alerts on Log Analytics.
{
"schemaId":"Microsoft.Insights/LogAlert","data":
{
"SubscriptionId":"12345a-1234b-123c-123d-12345678e",
"AlertRuleName":"AcmeRule",
"SearchQuery":"requests | where resultCode == \"500\"",
"SearchIntervalStartTimeUtc": "2018-03-26T08:10:40Z",
"SearchIntervalEndtimeUtc": "2018-03-26T09:10:40Z",
"AlertThresholdOperator": "Greater Than",
"AlertThresholdValue": 0,
"ResultCount": 2,
"SearchIntervalInSeconds": 3600,
"LinkToSearchResults": "https://portal.azure.com/AnalyticsBlade/subscriptions/12345a-1234b-123c-123d-
12345678e/?query=search+*+&timeInterval.intervalEnd=2018-03-
26T09%3a10%3a40.0000000Z&_timeInterval.intervalDuration=3600&q=Usage",
"Description": null,
"Severity": "3",
"SearchResult":
{
"tables":[
{"name":"PrimaryResult","columns":
[
{"name":"$table","type":"string"},
{"name":"Id","type":"string"},
{"name":"TimeGenerated","type":"datetime"}
],
"rows":
[
["Fabrikam","33446677a","2018-02-02T15:03:12.18Z"],
["Contoso","33445566b","2018-02-02T15:16:53.932Z"]
]
}
]
},
"ApplicationId": "123123f0-01d3-12ab-123f-abc1ab01c0a1",
"AlertType": "Number of results"
}
}
{
"alertname":"#alertrulename",
"IncludeSearchResults":true
}
The following sample payload is for a custom webhook action for any log alert:
{
"alertname":"AcmeRule","IncludeSearchResults":true,
"SearchResults":
{
"tables":[
{"name":"PrimaryResult","columns":
[
{"name":"$table","type":"string"},
{"name":"Id","type":"string"},
{"name":"TimeGenerated","type":"datetime"}
],
"rows":
[
["Fabrikam","33446677a","2018-02-02T15:03:12.18Z"],
["Contoso","33445566b","2018-02-02T15:16:53.932Z"]
]
}
]
}
}
Next steps
Learn about log alerts in Azure alerts .
Understand how to manage log alerts in Azure.
Create and manage action groups in Azure.
Learn more about Application Insights.
Learn more about log queries.
Troubleshoot log alerts in Azure Monitor
1/14/2020 • 9 minutes to read • Edit Online
This article shows you how to resolve common issues that might happen when you're setting up log alerts in Azure
Monitor. It also provides solutions to common problems with functionality or configuration of log alerts.
The term log alerts describe rules that fire based on a log query in an Azure Log Analytics workspace or in Azure
Application Insights. Learn more about functionality, terminology, and types in Log alerts in Azure Monitor.
NOTE
This article doesn't consider cases where the Azure portal shows an alert rule triggered and a notification is not performed by
an associated action group. For such cases, see the details in Create and manage action groups in the Azure portal.
Because Aggregate Upon is defined on timestamp, the data is sorted on the timestamp column (indicated in
red). Then we group by timestamp. For example, values for 2018-10-17T06:00:00Z will be considered as one
plot/entity (indicated in orange). In this value plot/entity, the alert service will find no consecutive breaches
(because each timestamp value has only one entry). So the alert is never triggered. In such a case, the user must
either:
Add a dummy variable or an existing variable (like $table) to correctly sort by using the Aggregate Upon
field.
Reconfigure the alert rule to use alert logic based on total breach instead.
The Query to be executed box is what the log alert service runs. If you want to understand what the alert query
output might be before you create the alert, you can run the stated query and the timespan via the Analytics portal
or the Analytics API.
Next steps
Learn about log alerts in Azure.
Learn more about Application Insights.
Learn more about log queries.
Create, view, and manage activity log alerts by using
Azure Monitor
1/14/2020 • 7 minutes to read • Edit Online
Overview
Activity log alerts are the alerts that get activated when a new activity log event occurs that matches the
conditions specified in the alert.
These alerts are for Azure resources and can be created by using an Azure Resource Manager template. They also
can be created, updated, or deleted in the Azure portal. Typically, you create activity log alerts to receive
notifications when specific changes occur to resources in your Azure subscription. Alerts are often scoped to
particular resource groups or resources. For example, you might want to be notified when any virtual machine in
the sample resource group myProductionResourceGroup is deleted. Or, you might want to get notified if any
new roles are assigned to a user in your subscription.
IMPORTANT
Alerts on service health notification can't be created via the interface for activity log alert creation. To learn more about how
to create and use service health notifications, see Receive activity log alerts on service health notifications .
Azure portal
You can use the Azure portal to create and modify activity log alert rules. The experience is integrated with an
Azure activity log to ensure seamless alert creation for specific events of interest.
Create with the Azure portal
Use the following procedure.
1. In the Azure portal, select Monitor > Alerts.
2. Select New alert rule in the upper-left corner of the Alerts window.
NOTE
You can select only Azure Resource Manager tracked resource, resource group, or an entire subscription for
an activity log signal.
You can use the available filters, Subscription, Resource group, Resource, Signal type, or Status, to find the
activity rule that you want to edit.
NOTE
You can edit only Description, Target criteria, and Action groups.
3. Select the rule, and double-click to edit the rule options. Make the required changes, and then select Save.
4. You can enable, disable, or delete a rule. Select the appropriate option at the top of the window after you
select the rule as described in step 2.
The previous sample JSON can be saved as, for example, sampleActivityLogAlert.json for the purpose of this
walk-through and can be deployed by using Azure Resource Manager in the Azure portal.
NOTE
It might take up to 5 minutes for the new activity log alert rule to become active.
REST API
The Azure Monitor Activity Log Alerts API is a REST API. It's fully compatible with the Azure Resource Manager
REST API. It can be used via PowerShell by using the Resource Manager cmdlet or the Azure CLI.
PowerShell
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
where the sampleActivityLogAlert.parameters.json contains the values provided for the parameters needed for
alert rule creation.
Use activity log PowerShell cmdlets
Activity log alerts have dedicated PowerShell cmdlets available:
Set-AzActivityLogAlert: Creates a new activity log alert or updates an existing activity log alert.
Get-AzActivityLogAlert: Gets one or more activity log alert resources.
Enable-AzActivityLogAlert: Enables an existing activity log alert and sets its tags.
Disable-AzActivityLogAlert: Disables an existing activity log alert and sets its tags.
Remove-AzActivityLogAlert: Removes an activity log alert.
Azure CLI
Dedicated Azure CLI commands under the set az monitor activity-log alert are available for managing activity log
alert rules.
To create a new activity log alert rule, use the following commands in this order:
1. az monitor activity-log alert create: Create a new activity log alert rule resource.
2. az monitor activity-log alert scope: Add scope for the created activity log alert rule.
3. az monitor activity-log alert action-group: Add an action group to the activity log alert rule.
To retrieve one activity log alert rule resource, use the Azure CLI command az monitor activity-log alert show. To
view all activity log alert rule resources in a resource group, use az monitor activity-log alert list. Activity log alert
rule resources can be removed by using the Azure CLI command az monitor activity-log alert delete.
Next steps
Learn about webhook schema for activity logs.
Read an overview of activity logs.
Learn more about action groups.
Learn about service health notifications.
2 minutes to read
2 minutes to read
2 minutes to read
Action rules (preview)
12/3/2019 • 9 minutes to read • Edit Online
Action rules help you define or suppress actions at any Azure Resource Manager scope (Azure subscription,
resource group, or target resource). They have various filters that help you narrow down the specific subset of
alert instances that you want to act on.
NOTE
Action rules currently don't apply to Azure Service Health alerts.
Alternatively, you can create an action rule while you're configuring an alert rule.
You should now see the flow page for creating action rules. Configure the following elements:
Scope
First choose the scope (Azure subscription, resource group, or target resource). You can also multiple-select a
combination of scopes within a single subscription.
Filter criteria
You can additionally define filters to narrow them down to a specific subset of the alerts.
The available filters are:
Severity: The option to select one or more alert severities. Severity = Sev1 means that the action rule is
applicable for all alerts set to Sev1.
Monitor Service: A filter based on the originating monitoring service. This filter is also multiple-select. For
example, Monitor Service = “Application Insights” means that the action rule is applicable for all
Application Insights-based alerts.
Resource Type: A filter based on a specific resource type. This filter is also multiple-select. For example,
Resource Type = “Virtual Machines” means that the action rule is applicable for all virtual machines.
Alert Rule ID: An option to filter for specific alert rules by using the Resource Manager ID of the alert rule.
Monitor Condition: A filter for alert instances with either Fired or Resolved as the monitor condition.
Description: A regex (regular expression) match that defines a string match against the description, defined as
part of the alert rule. For example, Description contains 'prod' will match all alerts that contain the string
"prod" in their descriptions.
Alert Context (payload): A regex match that defines a string match against the alert context fields of an
alert's payload. For example, Alert context (payload) contains 'Computer-01' will match all alerts whose
payloads contain the string "Computer-01."
These filters are applied in conjunction with one another. For example, if you set Resource type' = Virtual
Machines and Severity' = Sev0, then you've filtered for all Sev0 alerts on only your VMs.
Suppression or action group configuration
Next, configure the action rule for either alert suppression or action group support. You can't choose both. The
configuration acts on all alert instances that match the previously defined scope and filters.
Suppression
If you select suppression, configure the duration for the suppression of actions and notifications. Choose one of
the following options:
From now (Always): Suppresses all notifications indefinitely.
At a scheduled time: Suppresses notifications within a bounded duration.
With a recurrence: Suppresses notifications on a recurring daily, weekly, or monthly schedule.
Action group
If you select Action group in the toggle, either add an existing action group or create a new one.
NOTE
You can associate only one action group with an action rule.
Example scenarios
Scenario 1: Suppression of alerts based on severity
Contoso wants to suppress notifications for all Sev4 alerts on all VMs within the subscription ContosoSub every
weekend.
Solution: Create an action rule with:
Scope = ContosoSub
Filters
Severity = Sev4
Resource Type = Virtual Machines
Suppression with recurrence set to weekly, and Saturday and Sunday checked
Scenario 2: Suppression of alerts based on alert context (payload)
Contoso wants to suppress notifications for all log alerts generated for Computer-01 in ContosoSub indefinitely
as it's going through maintenance.
Solution: Create an action rule with:
Scope = ContosoSub
Filters
Monitor Service = Log Analytics
Alert Context (payload) contains Computer-01
Suppression set to From now (Always)
Scenario 3: Action group defined at a resource group
Contoso has defined a metric alert at a subscription level. But it wants to define the actions that trigger specifically
for alerts generated from the resource group ContosoRG.
Solution: Create an action rule with:
Scope = ContosoRG
No filters
Action group set to ContosoActionGroup
NOTE
Action groups defined within action rules and alert rules operate independently, with no deduplication. In the scenario
described earlier, if an action group is defined for the alert rule, it triggers in conjunction with the action group defined in
the action rule.
Best practices
Log alerts that you create with the number of results option generate a single alert instance by using the whole
search result (which might span across multiple computers). In this scenario, if an action rule uses the Alert
Context (payload) filter, it acts on the alert instance as long as there's a match. In Scenario 2, described
previously, if the search results for the generated log alert contain both Computer-01 and Computer-02, the
entire notification is suppressed. There's no notification generated for Computer-02 at all.
To best use log alerts with action rules, create log alerts with the metric measurement option. Separate alert
instances are generated by this option, based on its defined group field. Then, in Scenario 2, separate alert
instances are generated for Computer-01 and Computer-02. Due to the action rule described in the scenario,
only the notification for Computer-01 is suppressed. The notification for Computer-02 continues to fire as
normal.
FAQ
While I'm configuring an action rule, I'd like to see all the possible overlapping action rules, so that I avoid
duplicate notifications. Is it possible to do that?
After you define a scope as you configure an action rule, you can see a list of action rules that overlap on the same
scope (if any). This overlap can be one of the following options:
An exact match: For example, the action rule you're defining and the overlapping action rule are on the same
subscription.
A subset: For example, the action rule you're defining is on a subscription, and the overlapping action rule is on
a resource group within the subscription.
A superset: For example, the action rule you're defining is on a resource group, and the overlapping action rule
is on the subscription that contains the resource group.
An intersection: For example, the action rule you're defining is on VM1 and VM2, and the overlapping action
rule is on VM2 and VM3.
While I'm configuring an alert rule, is it possible to know if there are already action rules defined that might act
on the alert rule I'm defining?
After you define the target resource for your alert rule, you can see the list of action rules that act on the same
scope (if any) by selecting View configured actions under the Actions section. This list is populated based on
the following scenarios for the scope:
An exact match: For example, the alert rule you're defining and the action rule are on the same subscription.
A subset: For example, the alert rule you're defining is on a subscription, and the action rule is on a resource
group within the subscription.
A superset: For example, the alert rule you're defining is on a resource group, and the action rule is on the
subscription that contains the resource group.
An intersection: For example, the alert rule you're defining is on VM1 and VM2, and the action rule is on VM2
and VM3.
Can I see the alerts that have been suppressed by an action rule?
In the alerts list page, you can choose an additional column called Suppression Status. If the notification for an
alert instance was suppressed, it would show that status in the list.
If there's an action rule with an action group and another with suppression active on the same scope, what
happens?
Suppression always takes precedence on the same scope.
What happens if I have a resource that's monitored in two separate action rules? Do I get one or two
notifications? For example, VM2 in the following scenario:
action rule AR1 defined for VM1 and VM2 with action group AG1
action rule AR2 defined for VM2 and VM3 with action group AG1
For every alert on VM1 and VM3, action group AG1 would be triggered once. For every alert on VM2, action
group AG1 would be triggered twice, because action rules don't deduplicate actions.
What happens if I have a resource monitored in two separate action rules and one calls for action while another
for suppression? For example, VM2 in the following scenario:
action rule AR1 defined for VM1 and VM2 with action group AG1
action rule AR2 defined for VM2 and VM3 with suppression
For every alert on VM1, action group AG1 would be triggered once. Actions and notifications for every alert on
VM2 and VM3 will be suppressed.
What happens if I have an alert rule and an action rule defined for the same resource calling different action
groups? For example, VM1 in the following scenario:
alert rule rule1 on VM1 with action group AG2
action rule AR1 defined for VM1 with action group AG1
For every alert on VM1, action group AG1 would be triggered once. Whenever alert rule "rule1" is triggered, it
will also trigger AG2 additionally. Action groups defined within action rules and alert rules operate independently,
with no deduplication.
Next steps
Learn more about alerts in Azure
Create and manage action groups in the Azure
portal
1/3/2020 • 6 minutes to read • Edit Online
An action group is a collection of notification preferences defined by the owner of an Azure subscription.
Azure Monitor and Service Health alerts use action groups to notify users that an alert has been
triggered. Various alerts may use the same action group or different action groups depending on the
user's requirements. You may configure up to 2,000 action groups in a subscription.
You configure an action to notify a person by email or SMS, they receive a confirmation indicating they
have been added to the action group.
This article shows you how to create and manage action groups in the Azure portal.
Each action is made up of the following properties:
Name: A unique identifier within the action group.
Action type: The action performed. Examples include sending a voice call, SMS, email; or triggering
various types of automated actions. See types later in this article.
Details: The corresponding details that vary by action type.
For information on how to use Azure Resource Manager templates to configure action groups, see
Action group Resource Manager templates.
4. Enter a name in the Action group name box, and enter a name in the Short name box. The
short name is used in place of a full action group name when notifications are sent using this
group.
5. The Subscription box autofills with your current subscription. This subscription is the one in
which the action group is saved.
6. Select the Resource group in which the action group is saved.
7. Define a list of actions. Provide the following for each action:
a. Name: Enter a unique identifier for this action.
b. Action Type: Select Email/SMS/Push/Voice, Logic App, Webhook, ITSM, or Automation
Runbook.
c. Details: Based on the action type, enter a phone number, email address, webhook URI,
Azure app, ITSM connection, or Automation runbook. For ITSM Action, additionally
specify Work Item and other fields your ITSM tool requires.
d. Common alert schema: You can choose to enable the common alert schema, which
provides the advantage of having a single extensible and unified alert payload across all
the alert services in Azure Monitor.
8. Select OK to create the action group.
Modify the PowerShell script's Connect-AzureAD call to use your Azure AD Tenant ID.
Modify the PowerShell script's variable $myAzureADApplicationObjectId to use the Object ID
of your Azure AD Application
Run the modified script.
3. Configure the Action Group Secure Webhook action.
Copy the value $myApp.ObjectId from the script and enter it in the Application Object ID field
in the Webhook action definition.
# This is the name of the new role we will add to your Azure AD Application
$actionGroupRoleName = "ActionGroupsSecureWebhook"
SMS
See the rate limiting information and SMS alert behavior for additional important information.
You may have a limited number of SMS actions in an Action Group.
Voice
See the rate limiting information article.
You may have a limited number of Voice actions in an Action Group.
Webhook
Webhooks are retried using the following rules. The webhook call is retried a maximum of 2 times when
the following HTTP status codes are returned: 408, 429, 503, 504 or the HTTP endpoint does not
respond. The first retry happens after 10 seconds. The second retry happens after 100 seconds. After
two failures, no action group will call the endpoint for 30 minutes.
Source IP address ranges
13.72.19.232
13.106.57.181
13.106.54.3
13.106.54.19
13.106.38.142
13.106.38.148
13.106.57.196
13.106.57.197
52.244.68.117
52.244.65.137
52.183.31.0
52.184.145.166
51.4.138.199
51.5.148.86
51.5.149.19
To receive updates about changes to these IP addresses, we recommend you configure a Service Health
alert, which monitors for Informational notifications about the Action Groups service.
You may have a limited number of Webhook actions in an Action Group.
Next steps
Learn more about SMS alert behavior.
Gain an understanding of the activity log alert webhook schema.
Learn more about ITSM Connector
Learn more about rate limiting on alerts.
Get an overview of activity log alerts, and learn how to receive alerts.
Learn how to configure alerts whenever a service health notification is posted.
Common alert schema
10/17/2019 • 3 minutes to read • Edit Online
This article describes what the common alert schema is, the benefits of using it and how to enable it.
ACTION ENHANCEMENTS
Webhook/Logic App/Azure Function/Automation Runbook A consistent JSON structure for all alert types, which allows
you to easily build integrations across the different alert types.
The new schema will also enable a richer alert consumption experience across both the Azure portal and the Azure
mobile app in the immediate future.
Learn more about the schema definitions for Webhooks/Logic Apps/Azure Functions/Automation Runbooks.
NOTE
The following actions do not support the common alert schema: ITSM Connector.
NOTE
1. The following alert types support the common schema by default (no opt in required):
Smart detection alerts
2. The following alert types currently do not support the common schema:
Alerts generated by Azure Monitor for VMs
Alerts generated by Azure Cost Management
{
"properties": {
"groupShortName": "sample",
"enabled": true,
"emailReceivers": [
{
"name": "John Doe's email",
"emailAddress": "johndoe@email.com",
"useCommonAlertSchema": true
},
{
"name": "Jane Smith's email",
"emailAddress": "janesmith@email.com",
"useCommonAlertSchema": false
}
],
"smsReceivers": [
{
"name": "John Doe's mobile",
"countryCode": "1",
"phoneNumber": "1234567890"
},
{
"name": "Jane Smith's mobile",
"countryCode": "1",
"phoneNumber": "0987654321"
}
],
"webhookReceivers": [
{
"name": "Sample webhook",
"serviceUri": "http://www.example.com/webhook",
"useCommonAlertSchema": true
}
]
},
"location": "Global",
"tags": {}
}
Next steps
Common alert schema definitions for Webhooks/Logic Apps/Azure Functions/Automation Runbooks.
Learn how to create a logic app that leverages the common alert schema to handle all your alerts.
Common alert schema definitions
10/17/2019 • 6 minutes to read • Edit Online
This article describes the common alert schema definitions for Azure Monitor, including those for webhooks, Azure
Logic Apps, Azure Functions, and Azure Automation runbooks.
Any alert instance describes the resource that was affected and the cause of the alert. These instances are described
in the common schema in the following sections:
Essentials: A set of standardized fields, common across all alert types, which describe what resource the alert is
on, along with additional common alert metadata (for example, severity or description).
Alert context: A set of fields that describes the cause of the alert, with fields that vary based on the alert type.
For example, a metric alert includes fields like the metric name and metric value in the alert context, whereas an
activity log alert has information about the event that generated the alert.
Sample alert payload
{
"schemaId": "azureMonitorCommonAlertSchema",
"data": {
"essentials": {
"alertId": "/subscriptions/<subscription ID>/providers/Microsoft.AlertsManagement/alerts/b9569717-bc32-
442f-add5-83a997729330",
"alertRule": "WCUS-R2-Gen2",
"severity": "Sev3",
"signalType": "Metric",
"monitorCondition": "Resolved",
"monitoringService": "Platform",
"alertTargetIDs": [
"/subscriptions/<subscription
ID>/resourcegroups/pipelinealertrg/providers/microsoft.compute/virtualmachines/wcus-r2-gen2"
],
"originAlertId": "3f2d4487-b0fc-4125-8bd5-
7ad17384221e_PipeLineAlertRG_microsoft.insights_metricAlerts_WCUS-R2-Gen2_-117781227",
"firedDateTime": "2019-03-22T13:58:24.3713213Z",
"resolvedDateTime": "2019-03-22T14:03:16.2246313Z",
"description": "",
"essentialsVersion": "1.0",
"alertContextVersion": "1.0"
},
"alertContext": {
"properties": null,
"conditionType": "SingleResourceMultipleMetricCriteria",
"condition": {
"windowSize": "PT5M",
"allOf": [
{
"metricName": "Percentage CPU",
"metricNamespace": "Microsoft.Compute/virtualMachines",
"operator": "GreaterThan",
"threshold": "25",
"timeAggregation": "Average",
"dimensions": [
{
"name": "ResourceId",
"value": "3efad9dc-3d50-4eac-9c87-8b3fd6f97e4e"
}
],
"metricValue": 7.727
}
]
}
}
}
}
Essentials
FIELD DESCRIPTION
alertRule The name of the alert rule that generated the alert instance.
Severity The severity of the alert. Possible values: Sev0, Sev1, Sev2,
Sev3, or Sev4.
signalType Identifies the signal on which the alert rule was defined.
Possible values: Metric, Log, or Activity Log.
FIELD DESCRIPTION
alertTargetIds The list of the Azure Resource Manager IDs that are affected
targets of an alert. For a log alert defined on a Log Analytics
workspace or Application Insights instance, it's the respective
workspace or application.
firedDateTime The date and time when the alert instance was fired in
Coordinated Universal Time (UTC).
resolvedDateTime The date and time when the monitor condition for the alert
instance is set to Resolved in UTC. Currently only applicable
for metric alerts.
Sample values
{
"essentials": {
"alertId": "/subscriptions/<subscription ID>/providers/Microsoft.AlertsManagement/alerts/b9569717-bc32-
442f-add5-83a997729330",
"alertRule": "Contoso IT Metric Alert",
"severity": "Sev3",
"signalType": "Metric",
"monitorCondition": "Fired",
"monitoringService": "Platform",
"alertTargetIDs": [
"/subscriptions/<subscription ID>/resourceGroups/aimon-rg/providers/Microsoft.Insights/components/ai-
orion-int-fe"
],
"originAlertId": "74ff8faa0c79db6084969cf7c72b0710e51aec70b4f332c719ab5307227a984f",
"firedDateTime": "2019-03-26T05:25:50.4994863Z",
"description": "Test Metric alert",
"essentialsVersion": "1.0",
"alertContextVersion": "1.0"
}
}
Alert context
Metric alerts
monitoringService = Platform
Sample values
{
"alertContext": {
"properties": null,
"conditionType": "SingleResourceMultipleMetricCriteria",
"condition": {
"windowSize": "PT5M",
"allOf": [
{
"metricName": "Percentage CPU",
"metricNamespace": "Microsoft.Compute/virtualMachines",
"operator": "GreaterThan",
"threshold": "25",
"timeAggregation": "Average",
"dimensions": [
{
"name": "ResourceId",
"value": "3efad9dc-3d50-4eac-9c87-8b3fd6f97e4e"
}
],
"metricValue": 31.1105
}
],
"windowStartTime": "2019-03-22T13:40:03.064Z",
"windowEndTime": "2019-03-22T13:45:03.064Z"
}
}
}
Log alerts
NOTE
For log alerts that have a custom JSON payload defined, enabling the common schema reverts the payload schema to the
one described as follows. Alerts with the common schema enabled have an upper size limit of 256 KB per alert. Search results
aren't embedded in the log alerts payload if they cause the alert size to cross this threshold. You can determine this by
checking the flag IncludedSearchResults . When the search results aren't included, you should use the search query in
conjunction with the Log Analytics API.
Sample values
{
"alertContext": {
"SearchQuery": "search * \n| where Type == \"Heartbeat\" \n| where Category == \"Direct Agent\" \n| where
TimeGenerated > ago(30m) ",
"SearchIntervalStartTimeUtc": "3/22/2019 1:36:31 PM",
"SearchIntervalEndtimeUtc": "3/22/2019 1:51:31 PM",
"ResultCount": 15,
"LinkToSearchResults": "https://portal.azure.com#@72f988bf-86f1-41af-91ab-
2d7cd011db47/blade/Microsoft_OperationsManagementSuite_Workspace/AnalyticsBlade/initiator/AnalyticsShareLinkToQ
uery/isQueryEditorVisible/true/scope/%7B%22resources%22%3A%5B%7B%22resourceId%22%3A%22%2Fsubscriptions%
<subscription
ID>%2FresourceGroups%2Fpipelinealertrg%2Fproviders%2FMicrosoft.OperationalInsights%2Fworkspaces%2FINC-
OmsAlertRunner%22%7D%5D%7D/query/search%20%2A%20%0A%7C%20where%20Type%20%3D%3D%20%22Heartbeat%22%20%0A%7C%20whe
re%20Category%20%3D%3D%20%22Direct%20Agent%22%20%0A%7C%20where%20TimeGenerated%20%3E%20%28datetime%282019-03-
22T13%3A51%3A31.0000000%29%20-%2030m%29%20%20/isQuerybase64Compressed/false/timespanInIsoFormat/2019-03-
22T13%3a36%3a31.0000000Z%2f2019-03-22T13%3a51%3a31.0000000Z",
"SeverityDescription": "Warning",
"WorkspaceId": "2a1f50a7-ef97-420c-9d14-938e77c2a929",
"SearchIntervalDurationMin": "15",
"AffectedConfigurationItems": [
"INC-Gen2Alert"
],
"SearchIntervalInMinutes": "15",
"Threshold": 10000,
"Operator": "Less Than",
"SearchResult": {
"tables": [
{
"name": "PrimaryResult",
"columns": [
{
"name": "$table",
"type": "string"
},
{
"name": "Id",
"type": "string"
},
{
"name": "TimeGenerated",
"type": "datetime"
}
],
"rows": [
[
"Fabrikam",
"33446677a",
"2018-02-02T15:03:12.18Z"
],
[
"Contoso",
"33445566b",
"2018-02-02T15:16:53.932Z"
]
]
}
],
"dataSources": [
{
"resourceId": "/subscriptions/a5ea27e2-7482-49ba-90b3-60e7496dd873/resourcegroups/nrt-tip-
kc/providers/microsoft.operationalinsights/workspaces/nrt-tip-kc",
"tables": [
"Heartbeat"
]
}
]
},
"IncludedSearchResults": "True",
"AlertType": "Number of results"
}
}
Sample values
{
"alertContext": {
"SearchQuery": "search *",
"SearchIntervalStartTimeUtc": "3/22/2019 1:36:33 PM",
"SearchIntervalEndtimeUtc": "3/22/2019 1:51:33 PM",
"ResultCount": 0,
"LinkToSearchResults": "https://portal.azure.com#@72f988bf-86f1-41af-91ab-
2d7cd011db47/blade/Microsoft_OperationsManagementSuite_Workspace/AnalyticsBlade/initiator/AnalyticsShareLinkToQ
uery/isQueryEditorVisible/true/scope/%7B%22resources%22%3A%5B%7B%22resourceId%22%3A%22%2Fsubscriptions%
<subscription ID>%2FresourceGroups%2FPipeLineAlertRG%2Fproviders%2Fmicrosoft.insights%2Fcomponents%2FWEU-
AIRunner%22%7D%5D%7D/query/search%20%2A/isQuerybase64Compressed/false/timespanInIsoFormat/2019-03-
22T13%3a36%3a33.0000000Z%2f2019-03-22T13%3a51%3a33.0000000Z",
"SearchIntervalDurationMin": "15",
"SearchIntervalInMinutes": "15",
"Threshold": 10000,
"Operator": "Less Than",
"ApplicationId": "8e20151d-75b2-4d66-b965-153fb69d65a6",
"SearchResult": {
"tables": [
{
"name": "PrimaryResult",
"columns": [
{
"name": "$table",
"type": "string"
},
{
"name": "Id",
"type": "string"
},
{
"name": "TimeGenerated",
"type": "datetime"
}
],
"rows": [
[
"Fabrikam",
"33446677a",
"2018-02-02T15:03:12.18Z"
],
[
"Contoso",
"33445566b",
"2018-02-02T15:16:53.932Z"
]
]
}
],
"dataSources": [
{
"resourceId": "/subscriptions/a5ea27e2-7482-49ba-90b3-60e7496dd873/resourcegroups/nrt-tip-
kc/providers/microsoft.operationalinsights/workspaces/nrt-tip-kc",
"tables": [
"Heartbeat"
]
}
]
},
"IncludedSearchResults": "True",
"AlertType": "Number of results"
}
}
{
"alertContext": {
"authorization": {
"action": "Microsoft.Compute/virtualMachines/restart/action",
"scope": "/subscriptions/<subscription
ID>/resourceGroups/PipeLineAlertRG/providers/Microsoft.Compute/virtualMachines/WCUS-R2-ActLog"
},
"channels": "Operation",
"claims": "{\"aud\":\"https://management.core.windows.net/\",\"iss\":\"https://sts.windows.net/72f988bf-
86f1-41af-91ab-
2d7cd011db47/\",\"iat\":\"1553260826\",\"nbf\":\"1553260826\",\"exp\":\"1553264726\",\"aio\":\"42JgYNjdt+rr+3j/
dx68v018XhuFAwA=\",\"appid\":\"e9a02282-074f-45cf-93b0-
50568e0e7e50\",\"appidacr\":\"2\",\"http://schemas.microsoft.com/identity/claims/identityprovider\":\"https://s
ts.windows.net/72f988bf-86f1-41af-91ab-
2d7cd011db47/\",\"http://schemas.microsoft.com/identity/claims/objectidentifier\":\"9778283b-b94c-4ac6-8a41-
d5b493d03aa3\",\"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier\":\"9778283b-b94c-4ac6-
8a41-d5b493d03aa3\",\"http://schemas.microsoft.com/identity/claims/tenantid\":\"72f988bf-86f1-41af-91ab-
2d7cd011db47\",\"uti\":\"v5wYC9t9ekuA2rkZSVZbAA\",\"ver\":\"1.0\"}",
"caller": "9778283b-b94c-4ac6-8a41-d5b493d03aa3",
"correlationId": "8ee9c32a-92a1-4a8f-989c-b0ba09292a91",
"eventSource": "Administrative",
"eventTimestamp": "2019-03-22T13:56:31.2917159+00:00",
"eventDataId": "161fda7e-1cb4-4bc5-9c90-857c55a8f57b",
"level": "Informational",
"operationName": "Microsoft.Compute/virtualMachines/restart/action",
"operationId": "310db69b-690f-436b-b740-6103ab6b0cba",
"status": "Succeeded",
"subStatus": "",
"submissionTimestamp": "2019-03-22T13:56:54.067593+00:00"
}
}
Sample values
{
"alertContext": {
"authorization": {
"action": "Microsoft.Resources/checkPolicyCompliance/read",
"scope": "/subscriptions/<GUID>"
},
"channels": "Operation",
"claims": "
{\"aud\":\"https://management.azure.com/\",\"iss\":\"https://sts.windows.net/<GUID>/\",\"iat\":\"1566711059\",\
"nbf\":\"1566711059\",\"exp\":\"1566740159\",\"aio\":\"42FgYOhynHNw0scy3T/bL71+xLyqEwA=\",\"appid\":\"
<GUID>\",\"appidacr\":\"2\",\"http://schemas.microsoft.com/identity/claims/identityprovider\":\"https://sts.win
dows.net/<GUID>/\",\"http://schemas.microsoft.com/identity/claims/objectidentifier\":\"
<GUID>\",\"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier\":\"
<GUID>\",\"http://schemas.microsoft.com/identity/claims/tenantid\":\"
<GUID>\",\"uti\":\"Miy1GzoAG0Scu_l3m1aIAA\",\"ver\":\"1.0\"}",
"caller": "<GUID>",
"correlationId": "<GUID>",
"eventSource": "Policy",
"eventTimestamp": "2019-08-25T11:11:34.2269098+00:00",
"eventDataId": "<GUID>",
"level": "Warning",
"operationName": "Microsoft.Authorization/policies/audit/action",
"operationId": "<GUID>",
"properties": {
"isComplianceCheck": "True",
"resourceLocation": "eastus2",
"ancestors": "<GUID>",
"policies": "
[{\"policyDefinitionId\":\"/providers/Microsoft.Authorization/policyDefinitions/<GUID>/\",\"policySetDefinition
Id\":\"/providers/Microsoft.Authorization/policySetDefinitions/<GUID>/\",\"policyDefinitionReferenceId\":\"vuln
erabilityAssessmentMonitoring\",\"policySetDefinitionName\":\"<GUID>\",\"policyDefinitionName\":\"
<GUID>\",\"policyDefinitionEffect\":\"AuditIfNotExists\",\"policyAssignmentId\":\"/subscriptions/<GUID>/provide
rs/Microsoft.Authorization/policyAssignments/SecurityCenterBuiltIn/\",\"policyAssignmentName\":\"SecurityCenter
BuiltIn\",\"policyAssignmentScope\":\"/subscriptions/<GUID>\",\"policyAssignmentSku\":
{\"name\":\"A1\",\"tier\":\"Standard\"},\"policyAssignmentParameters\":{}}]"
},
"status": "Succeeded",
"subStatus": "",
"submissionTimestamp": "2019-08-25T11:12:46.1557298+00:00"
}
}
Sample values
{
"alertContext": {
"channels": "Admin, Operation",
"claims": "
{\"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/spn\":\"Microsoft.Insights/autoscaleSettings\"}",
"caller": "Microsoft.Insights/autoscaleSettings",
"correlationId": "<GUID>",
"eventSource": "Autoscale",
"eventTimestamp": "2019-08-21T16:17:47.1551167+00:00",
"eventDataId": "<GUID>",
"level": "Informational",
"operationName": "Microsoft.Insights/AutoscaleSettings/Scaleup/Action",
"operationId": "<GUID>",
"properties": {
"description": "The autoscale engine attempting to scale resource
'/subscriptions/d<GUID>/resourceGroups/testRG/providers/Microsoft.Compute/virtualMachineScaleSets/testVMSS'
from 9 instances count to 10 instances count.",
"resourceName":
"/subscriptions/<GUID>/resourceGroups/voiceassistancedemo/providers/Microsoft.Compute/virtualMachineScaleSets/a
lexademo",
"oldInstancesCount": "9",
"newInstancesCount": "10",
"activeAutoscaleProfile": "{\r\n \"Name\": \"Auto created scale condition\",\r\n \"Capacity\": {\r\n
\"Minimum\": \"1\",\r\n \"Maximum\": \"10\",\r\n \"Default\": \"1\"\r\n },\r\n \"Rules\": [\r\n
{\r\n \"MetricTrigger\": {\r\n \"Name\": \"Percentage CPU\",\r\n \"Namespace\":
\"microsoft.compute/virtualmachinescalesets\",\r\n \"Resource\":
\"/subscriptions/<GUID>/resourceGroups/testRG/providers/Microsoft.Compute/virtualMachineScaleSets/testVMSS\",\r
\n \"ResourceLocation\": \"eastus\",\r\n \"TimeGrain\": \"PT1M\",\r\n \"Statistic\":
\"Average\",\r\n \"TimeWindow\": \"PT5M\",\r\n \"TimeAggregation\": \"Average\",\r\n
\"Operator\": \"GreaterThan\",\r\n \"Threshold\": 0.0,\r\n \"Source\":
\"/subscriptions/<GUID>/resourceGroups/testRG/providers/Microsoft.Compute/virtualMachineScaleSets/testVMSS\",\r
\n \"MetricType\": \"MDM\",\r\n \"Dimensions\": [],\r\n \"DividePerInstance\": false\r\n
},\r\n \"ScaleAction\": {\r\n \"Direction\": \"Increase\",\r\n \"Type\":
\"ChangeCount\",\r\n \"Value\": \"1\",\r\n \"Cooldown\": \"PT1M\"\r\n }\r\n }\r\n
]\r\n}",
"lastScaleActionTime": "Wed, 21 Aug 2019 16:17:47 GMT"
},
"status": "Succeeded",
"submissionTimestamp": "2019-08-21T16:17:47.2410185+00:00"
}
}
Sample values
{
"alertContext": {
"channels": "Operation",
"correlationId": "<GUID>",
"eventSource": "Security",
"eventTimestamp": "2019-08-26T08:34:14+00:00",
"eventDataId": "<GUID>",
"level": "Informational",
"operationName": "Microsoft.Security/locations/alerts/activate/action",
"operationId": "<GUID>",
"properties": {
"threatStatus": "Quarantined",
"category": "Virus",
"threatID": "2147519003",
"filePath": "C:\\AlertGeneration\\test.eicar",
"protectionType": "Windows Defender",
"actionTaken": "Blocked",
"resourceType": "Virtual Machine",
"severity": "Low",
"compromisedEntity": "testVM",
"remediationSteps": "[\"No user action is necessary\"]",
"attackedResourceType": "Virtual Machine"
},
"status": "Active",
"submissionTimestamp": "2019-08-26T09:28:58.3019107+00:00"
}
}
monitoringService = ServiceHealth
Sample values
{
"alertContext": {
"authorization": null,
"channels": 1,
"claims": null,
"caller": null,
"correlationId": "f3cf2430-1ee3-4158-8e35-7a1d615acfc7",
"eventSource": 2,
"eventTimestamp": "2019-06-24T11:31:19.0312699+00:00",
"httpRequest": null,
"eventDataId": "<GUID>",
"level": 3,
"operationName": "Microsoft.ServiceHealth/maintenance/action",
"operationId": "<GUID>",
"properties": {
"title": "Azure SQL DW Scheduled Maintenance Pending",
"service": "SQL Data Warehouse",
"region": "East US",
"communication": "<MESSAGE>",
"incidentType": "Maintenance",
"trackingId": "<GUID>",
"impactStartTime": "2019-06-26T04:00:00Z",
"impactMitigationTime": "2019-06-26T12:00:00Z",
"impactedServices": "[{\"ImpactedRegions\":[{\"RegionName\":\"East US\"}],\"ServiceName\":\"SQL Data
Warehouse\"}]",
"impactedServicesTableRows": "<tr>\r\n<td align='center' style='padding: 5px 10px; border-right:1px solid
black; border-bottom:1px solid black'>SQL Data Warehouse</td>\r\n<td align='center' style='padding: 5px 10px;
border-bottom:1px solid black'>East US<br></td>\r\n</tr>\r\n",
"defaultLanguageTitle": "Azure SQL DW Scheduled Maintenance Pending",
"defaultLanguageContent": "<MESSAGE>",
"stage": "Planned",
"communicationId": "<GUID>",
"maintenanceId": "<GUID>",
"isHIR": "false",
"version": "0.1.1"
},
"status": "Active",
"subStatus": null,
"submissionTimestamp": "2019-06-24T11:31:31.7147357+00:00",
"ResourceType": null
}
}
Sample values
{
"alertContext": {
"channels": "Admin, Operation",
"correlationId": "<GUID>",
"eventSource": "ResourceHealth",
"eventTimestamp": "2019-06-24T15:42:54.074+00:00",
"eventDataId": "<GUID>",
"level": "Informational",
"operationName": "Microsoft.Resourcehealth/healthevent/Activated/action",
"operationId": "<GUID>",
"properties": {
"title": "This virtual machine is stopping and deallocating as requested by an authorized user or
process",
"details": null,
"currentHealthStatus": "Unavailable",
"previousHealthStatus": "Available",
"type": "Downtime",
"cause": "UserInitiated"
},
"status": "Active",
"submissionTimestamp": "2019-06-24T15:45:20.4488186+00:00"
}
}
Next steps
Learn more about the common alert schema.
Learn how to create a logic app that uses the common alert schema to handle all your alerts.
How to integrate the common alert schema with
Logic Apps
10/17/2019 • 2 minutes to read • Edit Online
This article shows you how to create a logic app that leverages the common alert schema to handle all your alerts.
Overview
The common alert schema provides a standardized and extensible JSON schema across all your different alert
types. The common alert schema is most useful when leveraged programmatically – through webhooks, runbooks,
and logic apps. In this article, we demonstrate how a single logic app can be authored to handle all your alerts. The
same principles can be applied to other programmatic methods. The logic app described in this article creates well-
defined variables for the 'essential' fields, and also describes how you can handle alert type specific logic.
Prerequisites
This article assumes that the reader is familiar with
Setting up alert rules ( metric, log, activity log)
Setting up action groups
Enabling the common alert schema from within action groups
Alternatively, you can author conditional logic based on the alert type using the 'Expression' option.
The 'monitoringService' field allows you to uniquely identify the alert type, based on which you can create
the conditional logic.
For example, the below snippet checks if the alert is a Application Insights based log alert, and if so prints
the search results. Else, it prints 'NA'.
if(equals(triggerBody()?['data']?['essentials']?['monitoringService'],'Application
Insights'),triggerBody()?['data']?['alertContext']?['SearchResults'],'NA')
Next steps
Learn more about action groups.
Learn more about the common alert schema.
Webhooks for Azure activity log alerts
1/8/2020 • 6 minutes to read • Edit Online
As part of the definition of an action group, you can configure webhook endpoints to receive activity log alert
notifications. With webhooks, you can route these notifications to other systems for post-processing or custom
actions. This article shows what the payload for the HTTP POST to a webhook looks like.
For more information on activity log alerts, see how to create Azure activity log alerts.
For information on action groups, see how to create action groups.
NOTE
You can also use the common alert schema, which provides the advantage of having a single extensible and unified alert
payload across all the alert services in Azure Monitor, for your webhook integrations. Learn about the common alert
schema definitions.
Payload schema
The JSON payload contained in the POST operation differs based on the payload's
data.context.activityLog.eventSource field.
Common
{
"schemaId": "Microsoft.Insights/activityLogs",
"data": {
"status": "Activated",
"context": {
"activityLog": {
"channels": "Operation",
"correlationId": "6ac88262-43be-4adf-a11c-bd2179852898",
"eventSource": "Administrative",
"eventTimestamp": "2017-03-29T15:43:08.0019532+00:00",
"eventDataId": "8195a56a-85de-4663-943e-1a2bf401ad94",
"level": "Informational",
"operationName": "Microsoft.Insights/actionGroups/write",
"operationId": "6ac88262-43be-4adf-a11c-bd2179852898",
"status": "Started",
"subStatus": "",
"subscriptionId": "52c65f65-0518-4d37-9719-7dbbfc68c57a",
"submissionTimestamp": "2017-03-29T15:43:20.3863637+00:00",
...
}
},
"properties": {}
}
}
Administrative
{
"schemaId": "Microsoft.Insights/activityLogs",
"data": {
"status": "Activated",
"context": {
"activityLog": {
"authorization": {
"action": "Microsoft.Insights/actionGroups/write",
"scope": "/subscriptions/52c65f65-0518-4d37-9719-7dbbfc68c57b/resourceGroups/CONTOSO-
TEST/providers/Microsoft.Insights/actionGroups/IncidentActions"
},
"claims": "{...}",
"caller": "me@contoso.com",
"description": "",
"httpRequest": "{...}",
"resourceId": "/subscriptions/52c65f65-0518-4d37-9719-7dbbfc68c57b/resourceGroups/CONTOSO-
TEST/providers/Microsoft.Insights/actionGroups/IncidentActions",
"resourceGroupName": "CONTOSO-TEST",
"resourceProviderName": "Microsoft.Insights",
"resourceType": "Microsoft.Insights/actionGroups"
}
},
"properties": {}
}
}
Security
{
"schemaId":"Microsoft.Insights/activityLogs",
"data":{"status":"Activated",
"context":{
"activityLog":{
"channels":"Operation",
"correlationId":"2518408115673929999",
"description":"Failed SSH brute force attack. Failed brute force attacks were detected from the following
attackers: [\"IP Address: 01.02.03.04\"]. Attackers were trying to access the host with the following user
names: [\"root\"].",
"eventSource":"Security",
"eventTimestamp":"2017-06-25T19:00:32.607+00:00",
"eventDataId":"Sec-07f2-4d74-aaf0-03d2f53d5a33",
"level":"Informational",
"operationName":"Microsoft.Security/locations/alerts/activate/action",
"operationId":"Sec-07f2-4d74-aaf0-03d2f53d5a33",
"properties":{
"attackers":"[\"IP Address: 01.02.03.04\"]",
"numberOfFailedAuthenticationAttemptsToHost":"456",
"accountsUsedOnFailedSignInToHostAttempts":"[\"root\"]",
"wasSSHSessionInitiated":"No","endTimeUTC":"06/25/2017 19:59:39",
"actionTaken":"Detected",
"resourceType":"Virtual Machine",
"severity":"Medium",
"compromisedEntity":"LinuxVM1",
"remediationSteps":"[In case this is an Azure virtual machine, add the source IP to NSG block list for 24
hours (see https://azure.microsoft.com/documentation/articles/virtual-networks-nsg/)]",
"attackedResourceType":"Virtual Machine"
},
"resourceId":"/subscriptions/12345-5645-123a-9867-
123b45a6789/resourceGroups/contoso/providers/Microsoft.Security/locations/centralus/alerts/Sec-07f2-4d74-aaf0-
03d2f53d5a33",
"resourceGroupName":"contoso",
"resourceProviderName":"Microsoft.Security",
"status":"Active",
"subscriptionId":"12345-5645-123a-9867-123b45a6789",
"submissionTimestamp":"2017-06-25T20:23:04.9743772+00:00",
"resourceType":"MICROSOFT.SECURITY/LOCATIONS/ALERTS"
}
},
"properties":{}
}
}
Recommendation
{
"schemaId":"Microsoft.Insights/activityLogs",
"data":{
"status":"Activated",
"context":{
"activityLog":{
"channels":"Operation",
"claims":"{\"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress\":\"Microsoft.Advisor\"}",
"caller":"Microsoft.Advisor",
"correlationId":"123b4c54-11bb-3d65-89f1-0678da7891bd",
"description":"A new recommendation is available.",
"eventSource":"Recommendation",
"eventTimestamp":"2017-06-29T13:52:33.2742943+00:00",
"httpRequest":"{\"clientIpAddress\":\"0.0.0.0\"}",
"eventDataId":"1bf234ef-e45f-4567-8bba-fb9b0ee1dbcb",
"level":"Informational",
"operationName":"Microsoft.Advisor/recommendations/available/action",
"properties":{
"recommendationSchemaVersion":"1.0",
"recommendationCategory":"HighAvailability",
"recommendationImpact":"Medium",
"recommendationName":"Enable Soft Delete to protect your blob data",
"recommendationResourceLink":"https://portal.azure.com/#blade/Microsoft_Azure_Expert/RecommendationListBlade/r
ecommendationTypeId/12dbf883-5e4b-4f56-7da8-123b45c4b6e6",
"recommendationType":"12dbf883-5e4b-4f56-7da8-123b45c4b6e6"
},
"resourceId":"/subscriptions/12345-5645-123a-9867-
123b45a6789/resourceGroups/contoso/providers/microsoft.storage/storageaccounts/contosoStore",
"resourceGroupName":"CONTOSO",
"resourceProviderName":"MICROSOFT.STORAGE",
"status":"Active",
"subStatus":"",
"subscriptionId":"12345-5645-123a-9867-123b45a6789",
"submissionTimestamp":"2017-06-29T13:52:33.2742943+00:00",
"resourceType":"MICROSOFT.STORAGE/STORAGEACCOUNTS"
}
},
"properties":{}
}
}
ServiceHealth
{
"schemaId": "Microsoft.Insights/activityLogs",
"data": {
"status": "Activated",
"context": {
"activityLog": {
"channels": "Admin",
"correlationId": "bbac944f-ddc0-4b4c-aa85-cc7dc5d5c1a6",
"description": "Active: Virtual Machines - Australia East",
"eventSource": "ServiceHealth",
"eventTimestamp": "2017-10-18T23:49:25.3736084+00:00",
"eventDataId": "6fa98c0f-334a-b066-1934-1a4b3d929856",
"level": "Informational",
"operationName": "Microsoft.ServiceHealth/incident/action",
"operationId": "bbac944f-ddc0-4b4c-aa85-cc7dc5d5c1a6",
"properties": {
"title": "Virtual Machines - Australia East",
"service": "Virtual Machines",
"region": "Australia East",
"communication": "Starting at 02:48 UTC on 18 Oct 2017 you have been identified as a customer
using Virtual Machines in Australia East who may receive errors starting Dv2 Promo and DSv2 Promo Virtual
Machines which are in a stopped "deallocated" or suspended state. Customers can still provision Dv1
and Dv2 series Virtual Machines or try deploying Virtual Machines in other regions, as a possible workaround.
Engineers have identified a possible fix for the underlying cause, and are exploring implementation options.
The next update will be provided as events warrant.",
"incidentType": "Incident",
"trackingId": "0NIH-U2O",
"impactStartTime": "2017-10-18T02:48:00.0000000Z",
"impactedServices": "[{\"ImpactedRegions\":[{\"RegionName\":\"Australia
East\"}],\"ServiceName\":\"Virtual Machines\"}]",
"defaultLanguageTitle": "Virtual Machines - Australia East",
"defaultLanguageContent": "Starting at 02:48 UTC on 18 Oct 2017 you have been identified as a
customer using Virtual Machines in Australia East who may receive errors starting Dv2 Promo and DSv2 Promo
Virtual Machines which are in a stopped "deallocated" or suspended state. Customers can still
provision Dv1 and Dv2 series Virtual Machines or try deploying Virtual Machines in other regions, as a
possible workaround. Engineers have identified a possible fix for the underlying cause, and are exploring
implementation options. The next update will be provided as events warrant.",
"stage": "Active",
"communicationId": "636439673646212912",
"version": "0.1.1"
},
"status": "Active",
"subscriptionId": "45529734-0ed9-4895-a0df-44b59a5a07f9",
"submissionTimestamp": "2017-10-18T23:49:28.7864349+00:00"
}
},
"properties": {}
}
}
For specific schema details on service health notification activity log alerts, see Service health notifications.
Additionally, learn how to configure service health webhook notifications with your existing problem
management solutions.
ResourceHealth
{
"schemaId": "Microsoft.Insights/activityLogs",
"data": {
"status": "Activated",
"context": {
"activityLog": {
"channels": "Admin, Operation",
"correlationId": "a1be61fd-37ur-ba05-b827-cb874708babf",
"eventSource": "ResourceHealth",
"eventTimestamp": "2018-09-04T23:09:03.343+00:00",
"eventDataId": "2b37e2d0-7bda-4de7-ur8c6-1447d02265b2",
"level": "Informational",
"operationName": "Microsoft.Resourcehealth/healthevent/Activated/action",
"operationId": "2b37e2d0-7bda-489f-81c6-1447d02265b2",
"properties": {
"title": "Virtual Machine health status changed to unavailable",
"details": "Virtual machine has experienced an unexpected event",
"currentHealthStatus": "Unavailable",
"previousHealthStatus": "Available",
"type": "Downtime",
"cause": "PlatformInitiated"
},
"resourceId": "/subscriptions/<subscription Id>/resourceGroups/<resource
group>/providers/Microsoft.Compute/virtualMachines/<resource name>",
"resourceGroupName": "<resource group>",
"resourceProviderName": "Microsoft.Resourcehealth/healthevent/action",
"status": "Active",
"subscriptionId": "<subscription Id>",
"submissionTimestamp": "2018-09-04T23:11:06.1607287+00:00",
"resourceType": "Microsoft.Compute/virtualMachines"
}
}
}
}
status Used for metric alerts. Always set to "activated" for activity
log alerts.
timestamp Time at which the event was generated by the Azure service
that processed the request.
caller Email address of the user who performed the operation, UPN
claim, or SPN claim based on availability. Can be null for
certain system calls.
For specific schema details on all other activity log alerts, see Overview of the Azure activity log.
Next steps
Learn more about the activity log.
Execute Azure automation scripts (Runbooks) on Azure alerts.
Use a logic app to send an SMS via Twilio from an Azure alert. This example is for metric alerts, but it can be
modified to work with an activity log alert.
Use a logic app to send a Slack message from an Azure alert. This example is for metric alerts, but it can be
modified to work with an activity log alert.
Use a logic app to send a message to an Azure queue from an Azure alert. This example is for metric alerts, but
it can be modified to work with an activity log alert.
SMS Alert Behavior in Action Groups
4/11/2019 • 2 minutes to read • Edit Online
Overview
Action groups enable you to configure a list of actions. These groups are used when defining alerts; ensuring that
a particular action group is notified when the alert is triggered. One of the actions supported is SMS; SMS
notifications support bi-directional communication. A user may respond to an SMS to:
Unsubscribe from alerts: A user may unsubscribe from all SMS alerts for all action groups, or a single action
group.
Resubscribe to alerts: A user may resubscribe to all SMS alerts for all action groups, or a single action group.
Request help: A user may ask for more information on the SMS. They are redirected to this article.
This article covers the behavior of the SMS alerts and the response actions the user can take based on the locale
of the user:
REPLY DESCRIPTION
DISABLE <Action Group Short name> Disables further SMS from the Action Group
ENABLE <Action Group Short name> Re-enables SMS from the Action Group
NOTE
If a user has unsubscribed from SMS alerts, but is then added to a new action group; they WILL receive SMS alerts for that
new action group, but remain unsubscribed from all previous action groups.
Next Steps
Get an overview of activity log alerts and learn how to get alerted
Learn more about SMS rate limiting
Learn more about action groups
Rate limiting for Voice, SMS, emails, Azure App push
notifications and webhook posts
3/15/2019 • 2 minutes to read • Edit Online
Rate limiting is a suspension of notifications that occurs when too many are sent to a particular phone number,
email address or device. Rate limiting ensures that alerts are manageable and actionable.
The rate limit thresholds are:
SMS: No more than 1 SMS every 5 minutes.
Voice: No more than 1 Voice call every 5 minutes.
Email: No more than 100 emails in an hour.
Other actions are not rate limited.
Next steps
Learn more about SMS alert behavior.
Get an overview of activity log alerts, and learn how to receive alerts.
Learn how to configure alerts whenever a service health notification is posted.
Create an action group with a Resource Manager
template
1/21/2019 • 2 minutes to read • Edit Online
This article shows you how to use an Azure Resource Manager template to configure action groups. By using
templates, you can automatically set up action groups that can be reused in certain types of alerts. These action
groups ensure that all the correct parties are notified when an alert is triggered.
The basic steps are:
1. Create a template as a JSON file that describes how to create the action group.
2. Deploy the template by using any deployment method.
First, we describe how to create a Resource Manager template for an action group where the action definitions are
hard-coded in the template. Second, we describe how to create a template that takes the webhook configuration
information as input parameters when the template is deployed.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"actionGroupName": {
"type": "string",
"metadata": {
"description": "Unique name (within the Resource Group) for the Action group."
}
},
"actionGroupShortName": {
"type": "string",
"metadata": {
"description": "Short name (maximum 12 characters) for the Action group."
}
}
},
"resources": [
{
"type": "Microsoft.Insights/actionGroups",
"apiVersion": "2018-03-01",
"name": "[parameters('actionGroupName')]",
"location": "Global",
"properties": {
"groupShortName": "[parameters('actionGroupShortName')]",
"enabled": true,
"smsReceivers": [
{
"name": "contosoSMS",
"countryCode": "1",
"phoneNumber": "5555551212"
},
{
"name": "contosoSMS2",
"name": "contosoSMS2",
"countryCode": "1",
"phoneNumber": "5555552121"
}
],
"emailReceivers": [
{
"name": "contosoEmail",
"emailAddress": "devops@contoso.com"
},
{
"name": "contosoEmail2",
"emailAddress": "devops2@contoso.com"
}
],
"webhookReceivers": [
{
"name": "contosoHook",
"serviceUri": "http://requestb.in/1bq62iu1"
},
{
"name": "contosoHook2",
"serviceUri": "http://requestb.in/1bq62iu2"
}
]
}
}
],
"outputs":{
"actionGroupId":{
"type":"string",
"value":"[resourceId('Microsoft.Insights/actionGroups',parameters('actionGroupName'))]"
}
}
}
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"actionGroupName": {
"type": "string",
"metadata": {
"description": "Unique name (within the Resource Group) for the Action group."
}
},
"actionGroupShortName": {
"type": "string",
"metadata": {
"description": "Short name (maximum 12 characters) for the Action group."
}
},
"webhookReceiverName": {
"type": "string",
"metadata": {
"description": "Webhook receiver service Name."
}
},
"webhookServiceUri": {
"type": "string",
"metadata": {
"description": "Webhook receiver service URI."
}
}
},
"resources": [
{
"type": "Microsoft.Insights/actionGroups",
"apiVersion": "2018-03-01",
"name": "[parameters('actionGroupName')]",
"location": "Global",
"properties": {
"groupShortName": "[parameters('actionGroupShortName')]",
"enabled": true,
"smsReceivers": [
],
"emailReceivers": [
],
"webhookReceivers": [
{
"name": "[parameters('webhookReceiverName')]",
"serviceUri": "[parameters('webhookServiceUri')]"
}
]
}
}
],
"outputs":{
"actionGroupResourceId":{
"type":"string",
"value":"[resourceId('Microsoft.Insights/actionGroups',parameters('actionGroupName'))]"
}
}
}
Next steps
Learn more about action groups.
Learn more about alerts.
Learn how to add alerts by using a Resource Manager template.
How to trigger complex actions with Azure Monitor
alerts
12/13/2019 • 6 minutes to read • Edit Online
This article shows you how to set up and trigger a logic app to create a conversation in Microsoft Teams when an
alert fires.
Overview
When an Azure Monitor alert triggers, it calls an action group. Action groups allow you to trigger one or more
actions to notify others about an alert and also remediate it.
The general process is:
Create the logic app for the respective alert type.
Import a sample payload for the respective alert type into the logic app.
Define the logic app behavior.
Copy the HTTP endpoint of the logic app into an Azure action group.
The process is similar if you want the logic app to perform a different action.
8. Copy and paste the following sample payload into the dialog box:
{
"schemaId": "Microsoft.Insights/activityLogs",
"data": {
"status": "Activated",
"context": {
"activityLog": {
"authorization": {
"action": "microsoft.insights/activityLogAlerts/write",
"scope": "/subscriptions/�"
},
"channels": "Operation",
"claims": "�",
"caller": "logicappdemo@contoso.com",
"correlationId": "91ad2bac-1afa-4932-a2ce-2f8efd6765a3",
"description": "",
"eventSource": "Administrative",
"eventTimestamp": "2018-04-03T22:33:11.762469+00:00",
"eventDataId": "ec74c4a2-d7ae-48c3-a4d0-2684a1611ca0",
"level": "Informational",
"operationName": "microsoft.insights/activityLogAlerts/write",
"operationId": "61f59fc8-1442-4c74-9f5f-937392a9723c",
"resourceId": "/subscriptions/�",
"resourceGroupName": "LOGICAPP-DEMO",
"resourceProviderName": "microsoft.insights",
"status": "Succeeded",
"subStatus": "",
"subscriptionId": "�",
"submissionTimestamp": "2018-04-03T22:33:36.1068742+00:00",
"resourceType": "microsoft.insights/activityLogAlerts"
}
},
"properties": {}
}
}
9. The Logic App Designer displays a pop-up window to remind you that the request sent to the logic app
must set the Content-Type header to application/json. Close the pop-up window. The Azure Monitor
alert sets the header.
11. Search for and select the Microsoft Teams connector. Choose the Microsoft Teams � Post message
action.
12. Configure the Microsoft Teams action. The Logic Apps Designer asks you to authenticate to your Office
365 account. Choose the Team ID and Channel ID to send the message to.
13. Configure the message by using a combination of static text and references to the <fields> in the dynamic
content. Copy and paste the following text into the Message field:
Then search for and replace the <fields> with dynamic content tags of the same name.
NOTE
There are two dynamic fields that are named status. Add both of these fields to the message. Use the field that's in
the activityLog property bag and delete the other field. Hover your cursor over the status field to see the fully
qualified field reference, as shown in the following screenshot:
14. At the top of the Logic Apps Designer, select Save to save your logic app.
15. Open your existing action group and add an action to reference the logic app. If you don�t have an existing
action group, see Create and manage action groups in the Azure portal to create one. Don�t forget to save
your changes.
The next time an alert calls your action group, your logic app is called.
<p>
<b>Alert Type:</b> <strong>[incidentType]</strong>
<strong>Tracking ID:</strong> [trackingId]
<strong>Title:</strong> [title]</p>
<p>
<a
href="https://ms.portal.azure.com/#blade/Microsoft_Azure_Health/AzureHealthBrowseBlade/serviceIss
ues">For details, log in to the Azure Service Health dashboard.</a>
</p>
<p>[communication]</p>
4. For the If false condition, provide a useful message:
{
"schemaId": "AzureMonitorMetricAlert",
"data": {
"version": "2.0",
"status": "Activated",
"context": {
"timestamp": "2018-04-09T19:00:07.7461615Z",
"id": "�",
"name": "TEST-VM CPU Utilization",
"description": "",
"conditionType": "SingleResourceMultipleMetricCriteria",
"condition": {
"windowSize": "PT15M",
"allOf": [
{
"metricName": "Percentage CPU",
"dimensions": [
{
"name": "ResourceId",
"value": "d92fc5cb-06cf-4309-8c9a-538eea6a17a6"
}
],
"operator": "GreaterThan",
"threshold": "5",
"timeAggregation": "PT15M",
"metricValue": 1.0
}
]
},
"subscriptionId": "�",
"resourceGroupName": "TEST",
"resourceName": "test-vm",
"resourceType": "Microsoft.Compute/virtualMachines",
"resourceId": "�",
"portalLink": "�"
},
"properties": {}
}
}
version == "2.0"
2. In the if true condition, add a For each loop and the Microsoft Teams action. Define the message by
using a combination of HTML and dynamic content.
3. In the If false condition, define a Microsoft Teams action to communicate that the metric alert
doesn�t match the expectations of the logic app. Include the JSON payload. Notice how to reference
the triggerBody dynamic content in the json() expression.
Step 15 is the same. Follow the instructions to save your logic app and update your action group.
Next steps
Get an overview of Azure activity log alerts and learn how to receive alerts.
Learn how to configure alerts when an Azure Service Health notification is posted.
Learn more about action groups.
Alert Management solution in Azure Log Analytics
10/25/2019 • 5 minutes to read • Edit Online
NOTE
Azure Monitor now supports enhanced capabilities for managing your alerts at scale, including those generated by
monitoring tools like System Center Operations Manager, Zabbix, or Nagios.
The Alert Management solution helps you analyze all of the alerts in your Log Analytics repository. These alerts
may have come from a variety of sources including those sources created by Log Analytics or imported from
Nagios or Zabbix. The solution also imports alerts from any connected System Center Operations Manager
management groups.
Prerequisites
The solution works with any records in the Log Analytics repository with a type of Alert, so you must perform
whatever configuration is required to collect these records.
For Log Analytics alerts, create alert rules to create alert records directly in the repository.
For Nagios and Zabbix alerts, configure those servers to send alerts to Log Analytics.
For System Center Operations Manager alerts, connect your Operations Manager management group to your
Log Analytics workspace. Any alerts created in System Center Operations Manager are imported into Log
Analytics.
Configuration
Add the Alert Management solution to your Log Analytics workspace using the process described in Add
solutions. There is no further configuration required.
Management packs
If your System Center Operations Manager management group is connected to your Log Analytics workspace,
then the following management packs are installed in System Center Operations Manager when you add this
solution. There is no configuration or maintenance of the management packs required.
Microsoft System Center Advisor Alert Management (Microsoft.IntelligencePacks.AlertManagement)
For more information on how solution management packs are updated, see Connect Operations Manager to Log
Analytics.
Data collection
Agents
The following table describes the connected sources that are supported by this solution.
CONNECTED SOURCE SUPPORT DESCRIPTION
Collection frequency
Alert records are available to the solution as soon as they are stored in the repository.
Alert data is sent from the Operations Manager management group to Log Analytics every three minutes.
Click on the Alert Management tile to open the Alert Management dashboard. The dashboard includes the
columns in the following table. Each column lists the top 10 alerts by count matching that column's criteria for the
specified scope and time range. You can run a log search that provides the entire list by clicking See all at the
bottom of the column or by clicking the column header.
COLUMN DESCRIPTION
Critical Alerts All alerts with a severity of Critical grouped by alert name.
Click on an alert name to run a log search returning all
records for that alert.
COLUMN DESCRIPTION
Warning Alerts All alerts with a severity of Warning grouped by alert name.
Click on an alert name to run a log search returning all
records for that alert.
Active System Center Operations Manager Alerts All alerts collected from Operations Manager with any state
other than Closed grouped by source that generated the
alert.
All Active Alerts All alerts with any severity grouped by alert name. Only
includes Operations Manager alerts with any state other than
Closed.
If you scroll to the right, the dashboard lists several common queries that you can click on to perform a log search
for alert data.
PROPERTY DESCRIPTION
Type Alert
SourceSystem OpsManager
AlertContext Details of the data item that caused the alert to be generated
in XML format.
RepeatCount Number of times the same alert was generated for the same
monitored object since being resolved.
ResolvedBy Name of the user who resolved the alert. Empty if the alert
has not yet been resolved.
SourceFullName Full name of the monitoring object that generated the alert.
TimeLastModified Date and time that the alert was last changed.
TimeResolved Date and time that the alert was resolved. Empty if the alert
has not yet been resolved.
QUERY DESCRIPTION
Alert | where SourceSystem == "OpsManager" and Critical alerts raised during the past 24 hours
AlertSeverity == "error" and TimeRaised > ago(24h)
Alert | where AlertSeverity == "warning" and TimeRaised > Warning alerts raised during the past 24 hours
ago(24h)
Alert | where SourceSystem == "OpsManager" and AlertState Sources with active alerts raised during the past 24 hours
!= "Closed" and TimeRaised > ago(24h) | summarize Count =
count() by SourceDisplayName
QUERY DESCRIPTION
Alert | where SourceSystem == "OpsManager" and Critical alerts raised during the past 24 hours that are still
AlertSeverity == "error" and TimeRaised > ago(24h) and active
AlertState != "Closed"
Alert | where SourceSystem == "OpsManager" and Alerts raised during the past 24 hours that are now closed
TimeRaised > ago(24h) and AlertState == "Closed"
Alert | where SourceSystem == "OpsManager" and Alerts raised during the past 1 day grouped by their severity
TimeRaised > ago(1d) | summarize Count = count() by
AlertSeverity
Alert | where SourceSystem == "OpsManager" and Alerts raised during the past 1 day sorted by their repeat
TimeRaised > ago(1d) | sort by RepeatCount desc count value
Next steps
Learn about Alerts in Log Analytics for details on generating alerts from Log Analytics.
Smart groups
10/17/2019 • 2 minutes to read • Edit Online
A common challenge faced when dealing with alerts is sifting through the noise to find out what actually matters -
smart groups are intended to be the solution to that problem.
Smart groups are automatically created by using machine learning algorithms to combine related alerts that
represent a single issue. When an alert is created, the algorithm adds it to a new smart group or an existing smart
group based on information such as historical patterns, similar properties, and similar structure. For example, if %
CPU on several virtual machines in a subscription simultaneously spikes leading to many individual alerts, and if
such alerts have occurred together anytime in the past, these alerts will likely be grouped into a single Smart
Group, suggesting a potential common root cause. This means that for someone troubleshooting alerts, smart
groups not only allows them to reduce noise by managing related alerts as a single aggregated unit, it also guides
them towards possible common root causes for their alerts.
Currently, the algorithm only considers alerts from the same monitor service within a subscription. Smart groups
can reduce up to 99% of alert noise through this consolidation. You can view the reason that alerts were included
in a group in the smart group details page.
You can view the details of smart groups and set the state similarly to how you can with alerts. Each alert is a
member of one and only one smart group.
STATE DESCRIPTION
New The issue has just been detected and has not yet been
reviewed.
Closed The issue has been resolved. After a smart group has been
closed, you can reopen it by changing it to another state.
NOTE
Changing the state of a smart group does not change the state of the individual member alerts.
SECTION DESCRIPTION
Alerts Lists the individual alerts that are included in the smart group.
Select an alert to open its alert detail page.
History Lists each action taken by the smart group and any changes
that are made to it. This is currently limited to state changes
and alert membership changes.
Next steps
Manage smart groups
Change your alert and smart group state
Manage alert instances with unified alerts
10/17/2019 • 2 minutes to read • Edit Online
With the unified alerts experience in Azure Monitor, you can see all your different types of alerts across Azure. This
spans multiple subscriptions, in a single pane. This article shows how you can view your alert instances, and how
to find specific alert instances for troubleshooting.
NOTE
You can only access alerts generated in the last 30 days.
Use the context of a specific resource. Open a resource, go to the Monitoring section, and choose Alerts.
The landing page is pre-filtered for alerts on that specific resource.
Use the context of a specific resource group. Open a resource group, go to the Monitoring section, and
choose Alerts. The landing page is pre-filtered for alerts on that specific resource group.
Find alert instances
The Alerts Summary page gives you an overview of all your alert instances across Azure. You can modify the
summary view by selecting multiple subscriptions (up to a maximum of 5), or by filtering across resource
groups, specific resources, or time ranges. Select Total Alerts, or any of the severity bands, to go to the list view
for your alerts.
On the All Alerts page, all the alert instances across Azure are listed. If you’re coming to the portal from an alert
notification, you can use the filters available to narrow in on that specific alert instance.
NOTE
If you came to the page by selecting any of the severity bands, the list is pre-filtered for that severity.
Apart from the filters available on the previous page, you can also filter on the basis of monitor service (for
example, platform for metrics), monitor condition (fired or resolved), severity, alert state
(new/acknowledged/closed), or the smart group ID.
NOTE
If you came to the page by selecting any of the severity bands, the list is pre-filtered for that severity.
Selecting any alert instance opens the Alert Details page, allowing you to see more details about that specific
alert instance.
Manage smart groups
10/17/2019 • 2 minutes to read • Edit Online
Smart groups use machine learning algorithms to group together alerts on the basis of co-occurrence or similarity,
so that the user can now manage smart groups instead of having to manage each alert individually. This article will
walk you through how to access and use smart groups in Azure Monitor.
1. To see the Smart Groups created for your alert instances you can either
a. Click on Smart Groups from the Alerts Summary page
2. This takes you to the list view for all Smart Groups created over your alert instances. Instead of sifting through
multiple alerts, you can now deal with the smart groups instead.
3. Clicking on any Smart Group opens up the details page, where you can see the grouping reason, along with the
member alerts. This aggregation allows you to deal with a singular smart group, instead of sifting through
multiple alerts.
Manage alert and smart group states
10/17/2019 • 2 minutes to read • Edit Online
Alerts in Azure Monitor now have an alert state and a monitor condition and, similarly, Smart Groups have a
smart group state. Changes to the state are now captured in history associated with the respective alert or smart
group. This article walks you through the process of changing the state, for both an alert and a smart group.
In the Alert Details page for a particular alert instance, you can click change state
In the Alert Details page for a specific alert instance, in the Smart Group pane you can click the
checkbox next to the alerts you wish
In the Smart Group Details page, in the list of member alerts you can click the checkbox next to the
alerts you wish to change the state of and click Change Stateto change the state of and click Change
State.
2. On clicking Change State, a popup opens up allowing you to select the state (New/Acknowledged/Closed) and
enter a comment if necessary.
3. Once this is done, the state change is recorded in the history of the respective alert. This can be viewed by
opening the respective Details page, and checking the history section.
Change the state of a smart group
1. You can change the state of a smart group in the following different ways:
a. In the Smart Group list page, you can click the checkbox next to the smart groups you wish to change
the state of and click Change State
b. In the Smart Group Details page, you can click change state
2. On clicking Change State, a popup opens up allowing you to select the state (New/Acknowledged/Closed)
and enter a comment if necessary.
NOTE
Changing the state of a smart group does not change the state of the individual member alerts.
3. Once this is done, the state change is recorded in the history of the respective smart group. This can be
viewed by opening the respective Details page, and checking the history section.
Manage alerts from System Center Operations
Manager, Zabbix and Nagios in Azure Monitor
10/17/2019 • 2 minutes to read • Edit Online
You can now view your alerts from Nagios, Zabbix, and System Center Operations Manager in Azure Monitor.
These alerts come from integrations with Nagios/Zabbix servers or System Center Operations Manager into Log
Analytics.
Prerequisites
Any records in the Log Analytics repository with a type of Alert will get imported into Azure Monitor, so you must
perform the configuration that is required to collect these records.
1. For Nagios and Zabbix alerts, configure those servers to send alerts to Log Analytics.
2. For System Center Operations Manager alerts, connect your Operations Manager management group to
your Log Analytics workspace. Following this, deploy the Alert Management solution from the Azure solutions
marketplace. Once done, any alerts created in System Center Operations Manager are imported into Log
Analytics.
NOTE
1. This solution only allows you to view System Center Operations Manager/Zabbix/Nagios fired alert instances in Azure
Monitor. Alert rule configuration can only be viewed/edited in respective monitoring tools.
2. All fired alert instances will be available both in Azure Monitor and Azure Log Analytics. Currently, there is no way to
choose between the two or ingest only specific fired alerts.
3. All alerts from System Center Operations Manager, Zabbix and Nagios have the signal type "Unknown" since the
underlying telemetry type is not available.
4. Nagios alerts are not stateful – for example, the monitor condition of an alert will not go from "Fired" to "Resolved".
Instead, both the “Fired” and “Resolved” are displayed as separate alert instances.
What are classic alerts in Microsoft Azure?
1/8/2020 • 5 minutes to read • Edit Online
NOTE
This article describes how to create older classic metric alerts. Azure Monitor now supports newer near-real time metric
alerts and a new alerts experience. Classic alerts are retired, though still in limited use for resources that do not yet support
the new alerts.
Alerts allow you to configure conditions over data and become notified when the conditions match the latest
monitoring data.
The new alerts user experience has the following benefits over the classic alerts experience:
Better notification system - All newer alerts use action groups, which are named groups of notifications
and actions that can be reused in multiple alerts. Classic metric alerts and older Log Analytics alerts do not
use action groups.
A unified authoring experience - All alert creation for metrics, logs and activity log across Azure Monitor,
Log Analytics, and Application Insights is in one place.
View fired Log Analytics alerts in Azure portal - You can now also see fired Log Analytics alerts in your
subscription. Previously these were in a separate portal.
Separation of fired alerts and alert rules - Alert rules (the definition of condition that triggers an alert), and
Fired Alerts (an instance of the alert rule firing) are differentiated, so the operational and configuration views
are separated.
Better workflow - The new alerts authoring experience guides the user along the process of configuring an
alert rule, which makes it simpler to discover the right things to get alerted on.
Smart Alerts consolidation and setting alert state - Newer alerts include auto grouping functionality
showing similar alerts together to reduce overload in the user interface.
The newer metric alerts have the following benefits over the classic metric alerts:
Improved latency: Newer metric alerts can run as frequently as every one minute. Older metric alerts
always run at a frequency of 5 minutes. Newer alerts have increasing smaller delay from issue occurrence to
notification or action (3 to 5 minutes). Older alerts are 5 to 15 minutes depending on the type. Log alerts
typically have 10 to 15-minute delay due to the time it takes to ingest the logs, but newer processing methods
are reducing that time.
Support for multi-dimensional metrics: You can alert on dimensional metrics allowing you to monitor an
interesting segment of the metric.
More control over metric conditions: You can define richer alert rules. The newer alerts support
monitoring the maximum, minimum, average, and total values of metrics.
Combined monitoring of multiple metrics: You can monitor multiple metrics (currently, up to two metrics)
with a single rule. An alert is triggered if both metrics breach their respective thresholds for the specified time-
period.
Better notification system: All newer alerts use action groups, which are named groups of notifications and
actions that can be reused in multiple alerts. Classic metric alerts and older Log Analytics alerts do not use
action groups.
Metrics from Logs (public preview ): Log data going into Log Analytics can now be extracted and converted
into Azure Monitor metrics and then alerted on just like other metrics. See Alerts (classic) for the terminology
specific to classic alerts.
Next steps
Get information about alert rules and configuring them by using:
Learn more about Metrics
Configure classic Metric Alerts via Azure portal
Configure classic Metric Alerts PowerShell
Configure classic Metric Alerts Command-line interface (CLI)
Configure classic Metric Alerts Azure Monitor REST API
Learn more about Activity Log
Configure Activity Log Alerts via Azure portal
Configure Activity Log Alerts via Resource Manager
Review the activity log alert webhook schema
Learn more about Action groups
Configure newer Alerts
Prepare your logic apps and runbooks for migration
of classic alert rules
1/23/2020 • 4 minutes to read • Edit Online
As previously announced, classic alerts in Azure Monitor are being retired in September 2019 (was originally July
2019). A migration tool is available in the Azure portal to customers who use classic alert rules and who want to
trigger migration themselves.
NOTE
Due to delay in roll-out of migration tool, the retirement date for classic alerts migration has been extended to August 31st,
2019 from the originally announced date of June 30th, 2019.
If you choose to voluntarily migrate your classic alert rules to new alert rules, be aware that there are some
differences between the two systems. This article explains those differences and how you can prepare for the
change.
API changes
The APIs that create and manage classic alert rules ( microsoft.insights/alertrules ) are different from the APIs
that create and manage new metric alerts ( microsoft.insights/metricalerts ). If you programmatically create and
manage classic alert rules today, update your deployment scripts to work with the new APIs.
The following table is a reference to the programmatic interfaces for both classic and new alerts:
Azure Resource Manager template For classic alerts For new metric alerts
The payloads are similar, as you can see. The following section offers:
Details about modifying logic apps to work with the new format.
A runbook example that parses the notification payload for new alerts.
[OutputType("PSAzureOperationResponse")]
param
(
[Parameter (Mandatory=$false)]
[object] $WebhookData
)
$ErrorActionPreference = "stop"
if ($WebhookData)
{
# Get the data object from WebhookData.
$WebhookBody = (ConvertFrom-Json -InputObject $WebhookData.RequestBody)
# Determine whether the alert triggering the runbook is a classic metric alert or a new metric alert
(depends on the payload schema).
$schemaId = $WebhookBody.schemaId
Write-Verbose "schemaId: $schemaId" -Verbose
if ($schemaId -eq "AzureMonitorMetricAlert") {
For a full example of a runbook that stops a virtual machine when an alert is triggered, see the Azure Automation
documentation.
Partner integration via webhooks
Most of our partners that integrate with classic alerts already support newer metric alerts through their
integrations. Known integrations that already work with new metric alerts are:
PagerDuty
OpsGenie
Signl4
If you're using a partner integration that's not listed here, confirm with the integration provider that the integration
works with new metric alerts.
Next steps
How to use the migration tool
Understand how the migration tool works
Understand how the migration tool works
1/6/2020 • 12 minutes to read • Edit Online
As previously announced, classic alerts in Azure Monitor are being retired by August 31, 2019 (was originally June
30, 2019). A migration tool is available in the Azure portal to customers who use classic alert rules and who want
to trigger migration themselves.
This article explains how the voluntary migration tool works. It also describes remedies for some common
problems.
NOTE
Due to delay in roll-out of migration tool, the retirement date for classic alerts migration has been extended to August 31st,
2019 from the originally announced date of June 30th, 2019.
Although the tool can migrate almost all classic alert rules, there are some exceptions. The following alert rules
won't be migrated by using the tool (or during the automatic migration starting September 2019):
Classic alert rules on virtual-machine guest metrics (both Windows and Linux). See the guidance for recreating
such alert rules in new metric alerts later in this article.
Classic alert rules on classic storage metrics. See the guidance for monitoring your classic storage accounts.
Classic alert rules on some storage account metrics. See details later in this article.
Classic alert rules on some Cosmos DB metrics. See details later in this article.
Classic alert rules on all classic virtual machines and cloud services metrics
(Microsoft.ClassicCompute/virtualMachines and Microsoft.ClassicCompute/domainNames/slots/roles). See
details later in this article.
If your subscription has any such classic rules, you must migrate them manually. Because we can't provide an
automatic migration, any existing, classic metric alerts of these types will continue to work until June 2020. This
extension gives you time to move over to new alerts. You can also continue to create new classic alerts on the
above listed exceptions till June 2020. However for everything else, no new classic alerts can be created after
August 2019.
NOTE
Besides the above listed exceptions, if your classic alert rules are invalid i.e. they are on deprecated metrics or resources that
have been deleted, they will not be migrated and will not be available after service is retired.
Microsoft.Network/publicIPAddresses defaultddostriggerrate
Microsoft.Web/hostingEnvironments/multirolepools averagememoryworkingset
How equivalent new alert rules and action groups are created
The migration tool converts your classic alert rules to equivalent new alert rules and action groups. For most
classic alert rules, equivalent new alert rules are on the same metric with the same properties such as windowSize
and aggregationType . However, there are some classic alert rules are on metrics that have a different, equivalent
metric in the new system. The following principles apply to the migration of classic alerts unless specified in the
section below:
Frequency: Defines how often a classic or new alert rule checks for the condition. The frequency in classic
alert rules was not configurable by the user and was always 5 mins for all resource types except Application
Insights components for which it was 1 min. So frequency of equivalent rules is also set to 5 min and 1 min
respectively.
Aggregation Type: Defines how the metric is aggregated over the window of interest. The aggregationType is
also the same between classic alerts and new alerts for most metrics. In some cases, since the metric is different
between classic alerts and new alerts, equivalent aggregationType or the primary Aggregation Type defined for
the metric is used.
Units: Property of the metric on which alert is created. Some equivalent metrics have different units. The
threshold is adjusted appropriately as needed. For example, if the original metric has seconds as units but
equivalent new metric has milliSeconds as units, the original threshold is multiplied by 1000 to ensure same
behavior.
Window Size: Defines the window over which metric data is aggregated to compare against the threshold. For
standard windowSize values like 5mins, 15mins, 30mins, 1hour, 3hours, 6 hours, 12 hours, 1 day, there is no
change made for equivalent new alert rule. For other values, the closest windowSize is chosen to be used. For
most customers, there is no impact with this change. For a small percentage of customers, there might be a
need to tweak the threshold to get exact same behavior.
In the following sections, we detail the metrics that have a different, equivalent metric in the new system. Any
metric that remains the same for classic and new alert rules is not listed. You can find a list of metrics supported in
the new system here.
Microsoft.StorageAccounts/services
For Storage account services like blob, table, file and queue, the following metrics are mapped to equivalent
metrics as shown below:
AverageE2ELatency SuccessE2ELatency
AverageServerLatency SuccessServerLatency
METRIC IN CLASSIC ALERTS EQUIVALENT METRIC IN NEW ALERTS COMMENTS
TotalBillableRequests Transactions
TotalEgress Egress
TotalIngress Ingress
TotalRequests Transactions
Microsoft.insights/components
For Application Insights, equivalent metrics are as shown below:
availability.availabilityMetric.value availabilityResults/availabilityPercentage
performanceCounter.available_bytes.val performanceCounters/memoryAvailable
ue Bytes
performanceCounter.io_data_bytes_per performanceCounters/processIOBytesP
_sec.value erSecond
METRIC IN CLASSIC ALERTS EQUIVALENT METRIC IN NEW ALERTS COMMENTS
performanceCounter.number_of_exceps performanceCounters/exceptionsPerSec
_thrown_per_sec.value ond
performanceCounter.percentage_proces performanceCounters/processCpuPerce
sor_time_normalized.value ntage
performanceCounter.percentage_proces performanceCounters/processorCpuPer
sor_total.value centage
performanceCounter.process_private_b performanceCounters/processPrivateBy
ytes.value tes
performanceCounter.request_execution performanceCounters/requestExecution
_time.value Time
performanceCounter.requests_in_applic performanceCounters/requestsInQueue
ation_queue.value
performanceCounter.requests_per_sec.v performanceCounters/requestsPerSeco
alue nd
request.rate requests/rate
Microsoft.DocumentDB/databaseAccounts
For Cosmos DB, equivalent metrics are as shown below:
AvailableStorage AvailableStorage
TotalRequestUnits TotalRequestUnits
NOTE
Classic alerts sent localized emails based on the locale of classic administrator when used to notify classic administrator roles.
New alert emails are sent via Action Groups and are only in English.
Rollout phases
The migration tool is rolling out in phases to customers that use classic alert rules. Subscription owners will
receive an email when the subscription is ready to be migrated by using the tool.
NOTE
Because the tool is being rolled out in phases, you might see that some of your subscriptions are not yet ready to be
migrated during the early phases.
Most of the subscriptions are currently marked as ready for migration. Only subscriptions that have classic alerts
on following resource types are still not ready for migration.
Microsoft.classicCompute/domainNames/slots/roles
Microsoft.insights/components
NOTE
In addition to having above permissions, your subscription should additionally be registered with
Microsoft.AlertsManagement resource provider. This is required to successfully migrate Failure Anomaly alerts on Application
Insights.
As previously announced, classic alerts in Azure Monitor are being retired in September 2019 (was originally July
2019). A migration tool is available in the Azure portal to customers who use classic alert rules and who want to
trigger migration themselves. This article explains how to use the migration tool to voluntarily migrate your
classic alert rules before the automatic migration starts in September 2019.
NOTE
Due to delay in roll-out of migration tool, the retirement date for classic alerts migration has been extended to August 31st,
2019 from the originally announced date of June 30th, 2019.
NOTE
The migration process won't impact the evaluation of your classic alert rules. They'll continue to run and send alerts
until they're migrated and the new alert rules take effect.
All subscriptions that can be migrated by using the tool are marked as Ready to migrate.
NOTE
The migration tool is rolling out in phases to all the subscriptions that use classic alert rules. In the early phases of
the rollout, you might see some subscriptions marked as not ready for migration.
IMPORTANT
After you initiate migration for a subscription, you won't be able to edit or create classic alert rules for that
subscription. This restriction ensures that no changes to your classic alert rules are lost during migration to the new
rules. Although you won't be able to change your classic alert rules, they'll still continue to run and to provide alerts
until they've been migrated. After the migration is complete for your subscription, you can't use classic alert rules
anymore.
7. When migration is complete, or if action is required from you, you'll receive an email at the addresses that
you provided earlier. You can also periodically check the status at the migration landing page in the portal.
Frequently asked questions
Why is my subscription listed as not ready for migration?
The migration tool is rolling out to customers in phases. In the early phases, most or all of your subscriptions
might be marked as Not ready for migration.
When a subscription becomes ready for migration, the subscription owner will receive an email message stating
that the tool is available. Keep an eye out for this message.
Who can trigger the migration?
Users who have the Monitoring Contributor role assigned to them at the subscription level are able to trigger the
migration. Learn more about Role-Based Access Control for the migration process.
How long will the migration take?
Migration is completed for most subscriptions in under an hour. You can keep track of the migration progress on
the migration landing page. During the migration, be assured that your alerts are still running either in the classic
alerts system or in the new one.
What can I do if I run into a problem during migration?
See the troubleshooting guide for help with problems you might face during migration. If any action is needed
from you to complete the migration, you'll be notified at the email addresses you provided when you set up the
tool.
Next steps
Prepare for the migration
Understand how the migration tool works
Understand the automatic migration process for your
classic alert rules
10/27/2019 • 3 minutes to read • Edit Online
As previously announced, classic alerts in Azure Monitor are being retired in September 2019 (was originally July
2019). As part of the retirement process, a migration tool is available in the Azure portal for customers to trigger
migration themselves. If you haven't used the migration tool by August 31, 2019, Azure Monitor will start the
process of automatic migration of your classic alerts starting September 1, 2019. This article walks you through
the automatic migration process and help you resolve any issues you might run into.
NOTE
This article only applies to Azure public cloud. Retirement process for Azure Monitor classic alerts in Azure Government cloud
and Azure China 21Vianet will be announced at future date.
NOTE
If you don't want to wait for the automatic migration process to start, you can still trigger the migration voluntarily
using the migration tool.
NOTE
The migration process won't impact the evaluation of your classic alert rules. They'll continue to run and send alerts
until they're migrated and the new alert rules take effect.
NOTE
In case an action is needed from customer like temporarily disabling a resource lock or changing a policy assignment,
customers will need to resolve any issues by October 31, 2019. If the issues are not resolved by then, successful migration of
your classic alerts cannot be guaranteed.
Next steps
Prepare for the migration
Understand how the migration tool works
Create, view, and manage classic metric alerts using
Azure Monitor
1/23/2020 • 4 minutes to read • Edit Online
Classic metric alerts in Azure Monitor provide a way to get notified when one of your metrics cross a threshold.
Classic metric alerts is an older functionality that allows for alerting only on non-dimensional metrics. There is an
existing newer functionality called Metric alerts which has improved functionality over classic metric alerts. You
can learn more about the new metric alerts functionality in metric alerts overview. In this article, we will describe
how to create, view and manage classic metric alert rules through Azure portal, Azure CLI and Powershell.
3. Select the Add metric alert (classic) command, and then fill in the fields.
4. Name your alert rule. Then choose a Description, which also appears in notification emails.
5. Select the Metric that you want to monitor. Then choose a Condition and Threshold value for the metric.
Also choose the Period of time that the metric rule must be satisfied before the alert triggers. For example,
if you use the period "Over the last 5 minutes" and your alert looks for a CPU above 80%, the alert triggers
when the CPU has been consistently above 80% for 5 minutes. After the first trigger occurs, it triggers
again when the CPU stays below 80% for 5 minutes. The CPU metric measurement happens every minute.
6. Select Email owners... if you want administrators and co-administrators to receive email notifications
when the alert fires.
7. If you want to send notifications to additional email addresses when the alert fires, add them in the
Additional Administrator email(s) field. Separate multiple emails with semicolons, in the following
format: email@contoso.com;email2@contoso.com
8. Put in a valid URI in the Webhook field if you want it to be called when the alert fires.
9. If you use Azure Automation, you can select a runbook to be run when the alert fires.
10. Select OK to create the alert.
Within a few minutes, the alert is active and triggers as previously described.
After you create an alert, you can select it and do one of the following tasks:
View a graph that shows the metric threshold and the actual values from the previous day.
Edit or delete it.
Disable or Enable it if you want to temporarily stop or resume receiving notifications for that alert.
With PowerShell
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
This sections shows how to use PowerShell commands create, view and manage classic metric alerts.The
examples in the article illustrate how you can use Azure Monitor cmdlets for classic metric alerts.
1. If you haven't already, set up PowerShell to run on your computer. For more information, seeHow to
Install and Configure PowerShell. You can also review the entire list of Azure Monitor PowerShell cmdlets
at Azure Monitor (Insights) Cmdlets.
2. First, log in to your Azure subscription.
Connect-AzAccount
3. You'll see a sign in screen. Once you sign in your Account, TenantID, and default Subscription ID are
displayed. All the Azure cmdlets work in the context of your default subscription. To view the list of
subscriptions you have access to, use the following command:
Get-AzSubscription
4. To change your working context to a different subscription, use the following command:
5. You can retrieve all classic metric alert rules on a resource group:
7. You can retrieve all alert rules set for a target resource. For example, all alert rules set on a VM.
8. Classic alert rules can no longer be created via PowerShell. To create an alert rule you need to use the new
'Add-AzMetricAlertRule' command.
Next steps
Create a classic metric alert with a Resource Manager template.
Have a classic metric alert notify a non-Azure system using a webhook.
Create a classic metric alert with a Resource Manager
template
1/14/2020 • 5 minutes to read • Edit Online
This article shows how you can use an Azure Resource Manager template to configure Azure metric alerts. This
enables you to automatically set up alerts on your resources when they are created to ensure that all resources are
monitored correctly.
NOTE
This article describes creating classic metric alerts using Resource Manager templates. If you are looking for creating newer
metric alerts using templates, this article provides the details.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"alertName": {
"type": "string",
"metadata": {
"description": "Name of alert"
}
},
"alertDescription": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Description of alert"
}
},
"isEnabled": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether alerts are enabled"
}
},
"resourceId": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Resource ID of the resource emitting the metric that will be used for the
comparison."
comparison."
}
},
"metricName": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Name of the metric used in the comparison to activate the alert."
}
},
"operator": {
"type": "string",
"defaultValue": "GreaterThan",
"allowedValues": [
"GreaterThan",
"GreaterThanOrEqual",
"LessThan",
"LessThanOrEqual"
],
"metadata": {
"description": "Operator comparing the current value with the threshold value."
}
},
"threshold": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "The threshold value at which the alert is activated."
}
},
"aggregation": {
"type": "string",
"defaultValue": "Average",
"allowedValues": [
"Average",
"Last",
"Maximum",
"Minimum",
"Total"
],
"metadata": {
"description": "How the data that is collected should be combined over time."
}
},
"windowSize": {
"type": "string",
"defaultValue": "PT5M",
"metadata": {
"description": "Period of time used to monitor alert activity based on the threshold. Must be
between five minutes and one day. ISO 8601 duration format."
}
},
"sendToServiceOwners": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "Specifies whether alerts are sent to service owners"
}
},
"customEmailAddresses": {
"type": "string",
"defaultValue": "",
"metadata": {
"description": "Comma-delimited email addresses where the alerts are also sent"
}
},
"webhookUrl": {
"type": "string",
"defaultValue": "",
"metadata": {
"metadata": {
"description": "URL of a webhook that will receive an HTTP POST when the alert activates."
}
}
},
"variables": {
"customEmails": "[split(parameters('customEmailAddresses'), ',')]"
},
"resources": [
{
"type": "Microsoft.Insights/alertRules",
"name": "[parameters('alertName')]",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-01",
"properties": {
"name": "[parameters('alertName')]",
"description": "[parameters('alertDescription')]",
"isEnabled": "[parameters('isEnabled')]",
"condition": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.ThresholdRuleCondition",
"dataSource": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleMetricDataSource",
"resourceUri": "[parameters('resourceId')]",
"metricName": "[parameters('metricName')]"
},
"operator": "[parameters('operator')]",
"threshold": "[parameters('threshold')]",
"windowSize": "[parameters('windowSize')]",
"timeAggregation": "[parameters('aggregation')]"
},
"actions": [
{
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleEmailAction",
"sendToServiceOwners": "[parameters('sendToServiceOwners')]",
"customEmails": "[variables('customEmails')]"
},
{
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleWebhookAction",
"serviceUri": "[parameters('webhookUrl')]",
"properties": {}
}
]
}
}
]
}
An explanation of the schema and properties for an alert rule is available here.
{
"$schema": "https://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"newStorageAccountName": {
"type": "string",
"metadata": {
"Description": "The name of the storage account where the VM disk is stored."
}
},
"adminUsername": {
"type": "string",
"metadata": {
"Description": "The name of the administrator account on the VM."
}
},
"adminPassword": {
"type": "securestring",
"metadata": {
"Description": "The administrator account password on the VM."
}
},
"dnsNameForPublicIP": {
"type": "string",
"metadata": {
"Description": "The name of the public IP address used to access the VM."
}
}
},
"variables": {
"location": "Central US",
"imagePublisher": "MicrosoftWindowsServer",
"imageOffer": "WindowsServer",
"windowsOSVersion": "2012-R2-Datacenter",
"OSDiskName": "osdisk1",
"nicName": "nc1",
"addressPrefix": "10.0.0.0/16",
"subnetName": "sn1",
"subnetPrefix": "10.0.0.0/24",
"storageAccountType": "Standard_LRS",
"publicIPAddressName": "ip1",
"publicIPAddressType": "Dynamic",
"vmStorageAccountContainerName": "vhds",
"vmName": "vm1",
"vmSize": "Standard_A0",
"virtualNetworkName": "vn1",
"vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
"subnetRef": "[concat(variables('vnetID'),'/subnets/',variables('subnetName'))]",
"vmID":"[resourceId('Microsoft.Compute/virtualMachines',variables('vmName'))]",
"alertName": "highCPUOnVM",
"alertDescription":"CPU is over 80%",
"alertIsEnabled": true,
"resourceId": "",
"metricName": "Percentage CPU",
"operator": "GreaterThan",
"threshold": "80",
"windowSize": "PT5M",
"aggregation": "Average",
"customEmails": "",
"sendToServiceOwners": true,
"webhookUrl": "http://testwebhook.test"
},
"resources": [
{
"type": "Microsoft.Storage/storageAccounts",
"name": "[parameters('newStorageAccountName')]",
"apiVersion": "2015-06-15",
"location": "[variables('location')]",
"properties": {
"accountType": "[variables('storageAccountType')]"
}
},
{
"apiVersion": "2016-03-30",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('publicIPAddressName')]",
"location": "[variables('location')]",
"properties": {
"publicIPAllocationMethod": "[variables('publicIPAddressType')]",
"dnsSettings": {
"domainNameLabel": "[parameters('dnsNameForPublicIP')]"
}
}
},
{
"apiVersion": "2016-03-30",
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
"location": "[variables('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
}
]
}
},
{
"apiVersion": "2016-03-30",
"type": "Microsoft.Network/networkInterfaces",
"name": "[variables('nicName')]",
"location": "[variables('location')]",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]",
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
],
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "
[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]"
},
"subnet": {
"id": "[variables('subnetRef')]"
}
}
}
]
}
},
{
"apiVersion": "2016-03-30",
"type": "Microsoft.Compute/virtualMachines",
"name": "[variables('vmName')]",
"location": "[variables('location')]",
"dependsOn": [
"[concat('Microsoft.Storage/storageAccounts/', parameters('newStorageAccountName'))]",
"[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
],
"properties": {
"hardwareProfile": {
"vmSize": "[variables('vmSize')]"
},
"osProfile": {
"osProfile": {
"computername": "[variables('vmName')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "[variables('imagePublisher')]",
"offer": "[variables('imageOffer')]",
"sku": "[variables('windowsOSVersion')]",
"version": "latest"
},
"osDisk": {
"name": "osdisk",
"vhd": {
"uri": "
[concat('http://',parameters('newStorageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountCon
tainerName'),'/',variables('OSDiskName'),'.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
}
]
}
}
},
{
"type": "Microsoft.Insights/alertRules",
"name": "[variables('alertName')]",
"dependsOn": [
"[variables('vmID')]"
],
"location": "[variables('location')]",
"apiVersion": "2016-03-01",
"properties": {
"name": "[variables('alertName')]",
"description": "variables('alertDescription')",
"isEnabled": "[variables('alertIsEnabled')]",
"condition": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.ThresholdRuleCondition",
"dataSource": {
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleMetricDataSource",
"resourceUri": "[variables('vmID')]",
"metricName": "[variables('metricName')]"
},
"operator": "[variables('operator')]",
"threshold": "[variables('threshold')]",
"windowSize": "[variables('windowSize')]",
"timeAggregation": "[variables('aggregation')]"
},
"actions": [
{
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleEmailAction",
"sendToServiceOwners": "[variables('sendToServiceOwners')]",
"customEmails": "[variables('customEmails')]"
},
{
"odata.type": "Microsoft.Azure.Management.Insights.Models.RuleWebhookAction",
"serviceUri": "[variables('webhookUrl')]",
"properties": {}
}
]
}
}
}
]
}
Next Steps
Read more about Alerts
Add Diagnostic Settings to your Resource Manager template
For the JSON syntax and properties, see Microsoft.Insights/alertrules template reference.
Call a webhook with a classic metric alert in Azure
Monitor
1/23/2020 • 3 minutes to read • Edit Online
You can use webhooks to route an Azure alert notification to other systems for post-processing or custom
actions. You can use a webhook on an alert to route it to services that send SMS messages, to log bugs, to notify
a team via chat or messaging services, or for various other actions.
This article describes how to set a webhook on an Azure metric alert. It also shows you what the payload for the
HTTP POST to a webhook looks like. For information about the setup and schema for an Azure activity log alert
(alert on events), see Call a webhook on an Azure activity log alert.
Azure alerts use HTTP POST to send the alert contents in JSON format to a webhook URI that you provide
when you create the alert. The schema is defined later in this article. The URI must be a valid HTTP or HTTPS
endpoint. Azure posts one entry per request when an alert is activated.
You can also configure an alert to post to a webhook URI by using Azure PowerShell cmdlets, a cross-platform
CLI, or Azure Monitor REST APIs.
{
"status": "Activated",
"context": {
"timestamp": "2015-08-14T22:26:41.9975398Z",
"id": "/subscriptions/s1/resourceGroups/useast/providers/microsoft.insights/alertrules/ruleName1",
"name": "ruleName1",
"description": "some description",
"conditionType": "Metric",
"condition": {
"metricName": "Requests",
"metricUnit": "Count",
"metricValue": "10",
"threshold": "10",
"windowSize": "15",
"timeAggregation": "Average",
"operator": "GreaterThanOrEqual"
},
"subscriptionId": "s1",
"resourceGroupName": "useast",
"resourceName": "mysite1",
"resourceType": "microsoft.foo/sites",
"resourceId": "/subscriptions/s1/resourceGroups/useast/providers/microsoft.foo/sites/mysite1",
"resourceRegion": "centralus",
"portalLink":
"https://portal.azure.com/#resource/subscriptions/s1/resourceGroups/useast/providers/microsoft.foo/sites/mysi
te1"
},
"properties": {
"key1": "value1",
"key2": "value2"
}
}
metricUnit For metric alerts Bytes, BytesPerSecond, The unit allowed in the
Count, CountPerSecond, metric. See allowed values.
Percent, Seconds
timeAggregation For metric alerts Average, Last, Maximum, How the data that's
Minimum, None, Total collected should be
combined over time. The
default value is Average. See
allowed values.
NOTE
You can set the properties field only by using Azure Monitor REST APIs.
Next steps
Learn more about Azure alerts and webhooks in the video Integrate Azure alerts with PagerDuty.
Learn how to execute Azure Automation scripts (runbooks) on Azure alerts.
Learn how to use a logic app to send an SMS message via Twilio from an Azure alert.
Learn how to use a logic app to send a Slack message from an Azure alert.
Learn how to use a logic app to send a message to an Azure queue from an Azure alert.
Create and manage alert rules in Log Analytics with
REST API
1/6/2020 • 13 minutes to read • Edit Online
The Log Analytics Alert REST API allows you to create and manage alerts in Log Analytics. This article
provides details of the API and several examples for performing different operations.
IMPORTANT
As announced earlier, log analytics workspace(s) created after June 1, 2019 - will be able to manage alert rules using
only Azure scheduledQueryRules REST API, Azure Resource Mananger Template and PowerShell cmdlet. Customers can
easily switch their preferred means of alert rule management for older workspaces to leverage Azure Monitor
scheduledQueryRules as default and gain many new benefits like the ability to use native PowerShell cmdlets, increased
lookback time period in rules, creation of rules in separate resource group or subscription and much more.
The Log Analytics Search REST API is RESTful and can be accessed via the Azure Resource Manager REST
API. In this document, you will find examples where the API is accessed from a PowerShell command line
using ARMClient, an open-source command-line tool that simplifies invoking the Azure Resource Manager
API. The use of ARMClient and PowerShell is one of many options to access the Log Analytics Search API.
With these tools, you can utilize the RESTful Azure Resource Manager API to make calls to Log Analytics
workspaces and perform search commands within them. The API will output search results to you in JSON
format, allowing you to use the search results in many different ways programmatically.
Prerequisites
Currently, alerts can only be created with a saved search in Log Analytics. You can refer to the Log Search
REST API for more information.
Schedules
A saved search can have one or more schedules. The schedule defines how often the search is run and the time
interval over which the criteria is identified. Schedules have the properties in the following table.
PROPERTY DESCRIPTION
QueryTimeSpan The time interval over which the criteria is evaluated. Must
be equal to or greater than Interval. Measured in minutes.
Version The API version being used. Currently, this should always be
set to 1.
For example, consider an event query with an Interval of 15 minutes and a Timespan of 30 minutes. In this
case, the query would be run every 15 minutes, and an alert would be triggered if the criteria continued to
resolve to true over a 30-minute span.
Retrieving schedules
Use the Get method to retrieve all schedules for a saved search.
armclient get /subscriptions/{Subscription
ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace
Name}/savedSearches/{Search ID}/schedules?api-version=2015-03-20
Use the Get method with a schedule ID to retrieve a particular schedule for a saved search.
{
"value": [{
"id": "subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxxxxx/resourceGroups/sampleRG/providers/Microsoft.OperationalInsights/workspaces/MyWorkspace/savedSe
arches/0f0f4853-17f8-4ed1-9a03-8e888b0d16ec/schedules/a17b53ef-bd70-4ca4-9ead-83b00f2024a8",
"etag": "W/\"datetime'2016-02-25T20%3A54%3A49.8074679Z'\"",
"properties": {
"Interval": 15,
"QueryTimeSpan": 15,
"Enabled": true,
}
}]
}
Creating a schedule
Use the Put method with a unique schedule ID to create a new schedule. Two schedules cannot have the same
ID even if they are associated with different saved searches. When you create a schedule in the Log Analytics
console, a GUID is created for the schedule ID.
NOTE
The name for all saved searches, schedules, and actions created with the Log Analytics API must be in lowercase.
Editing a schedule
Use the Put method with an existing schedule ID for the same saved search to modify that schedule; in example
below the schedule is disabled. The body of the request must include the etag of the schedule.
Deleting schedules
Use the Delete method with a schedule ID to delete a schedule.
armclient delete /subscriptions/{Subscription
ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace
Name}/savedSearches/{Subscription ID}/schedules/{Schedule ID}?api-version=2015-03-20
Actions
A schedule can have multiple actions. An action may define one or more processes to perform such as sending
a mail or starting a runbook, or it may define a threshold that determines when the results of a search match
some criteria. Some actions will define both so that the processes are performed when the threshold is met.
All actions have the properties in the following table. Different types of alerts have different additional
properties, which are described below.
PROPERTY DESCRIPTION
Type Type of the action. Currently the possible values are Alert
and Webhook.
Version The API version being used. Currently, this should always be
set to 1.
Retrieving actions
Use the Get method to retrieve all actions for a schedule.
Use the Get method with the action ID to retrieve a particular action for a schedule.
NOTE
The name for all saved searches, schedules, and actions created with the Log Analytics API must be in lowercase.
Use the Put method with an existing action ID for the same saved search to modify that schedule. The body of
the request must include the etag of the schedule.
The request format for creating a new action varies by action type so these examples are provided in the
sections below.
Deleting actions
Use the Delete method with the action ID to delete an action.
armclient delete /subscriptions/{Subscription
ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace
Name}/savedSearches/{Subscription ID}/schedules/{Schedule ID}/Actions/{Action ID}?api-version=2015-03-20
Alert Actions
A Schedule should have one and only one Alert action. Alert actions have one or more of the sections in the
following table. Each is described in further detail below.
Threshold Criteria for when the action is run. Required for every alert, before or
after they are extended to Azure.
Severity Label used to classify alert when Required for every alert, before or
triggered. after they are extended to Azure.
Suppress Option to stop notifications from alert. Optional for every alert, before or
after they are extended to Azure.
Action Groups IDs of Azure ActionGroup where Required once alerts are extended to
actions required are specified, like - E- Azure
Mails, SMSs, Voice Calls, Webhooks,
Automation Runbooks, ITSM
Connectors, etc.
Customize Actions Modify the standard output for select Optional for every alert, can be used
actions from ActionGroup after alerts are extended to Azure.
Thresholds
An Alert action should have one and only one threshold. When the results of a saved search match the
threshold in an action associated with that search, then any other processes in that action are run. An action can
also contain only a threshold so that it can be used with actions of other types that don’t contain thresholds.
Thresholds have the properties in the following table.
PROPERTY DESCRIPTION
For example, consider an event query with an Interval of 15 minutes, a Timespan of 30 minutes, and a
Threshold of greater than 10. In this case, the query would be run every 15 minutes, and an alert would be
triggered if it returned 10 events that were created over a 30-minute span.
Following is a sample response for an action with only a threshold.
"etag": "W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"",
"properties": {
"Type": "Alert",
"Name": "My threshold action",
"Threshold": {
"Operator": "gt",
"Value": 10
},
"Version": 1
}
Use the Put method with a unique action ID to create a new threshold action for a schedule.
Use the Put method with an existing action ID to modify a threshold action for a schedule. The body of the
request must include the etag of the action.
Severity
Log Analytics allows you to classify your alerts into categories, to allow easier management and triage. The
Alert severity defined is: informational, warning, and critical. These are mapped to the normalized severity scale
of Azure Alerts as:
critical Sev 0
warning Sev 1
informational Sev 2
Following is a sample response for an action with only a threshold and severity.
"etag": "W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"",
"properties": {
"Type": "Alert",
"Name": "My threshold action",
"Threshold": {
"Operator": "gt",
"Value": 10
},
"Severity": "critical",
"Version": 1 }
Use the Put method with a unique action ID to create a new action for a schedule with severity.
$thresholdWithSevJson = "{'properties': { 'Name': 'My Threshold', 'Version':'1','Severity': 'critical',
'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 10 } } }"
armclient put /subscriptions/{Subscription
ID}/resourceGroups/{ResourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{Workspace
Name}/savedSearches/{Search ID}/schedules/{Schedule ID}/actions/mythreshold?api-version=2015-03-20
$thresholdWithSevJson
Use the Put method with an existing action ID to modify a severity action for a schedule. The body of the
request must include the etag of the action.
Suppress
Log Analytics based query alerts will fire every time threshold is met or exceeded. Based on the logic implied in
the query, this may result in alert getting fired for a series of intervals and hence notifications also being sent
constantly. To prevent such scenario, a user can set Suppress option instructing Log Analytics to wait for a
stipulated amount of time before notification is fired the second time for the alert rule. So if suppress is set for
30 minutes; then alert will fire the first time and send notifications configured. But then wait for 30 minutes,
before notification for the alert rule is again used. In the interim period, alert rule will continue to run - only
notification is suppressed by Log Analytics for specified time, regardless of how many times the alert rule fired
in this period.
Suppress property of Log Analytics alert rule is specified using the Throttling value and the suppression period
using DurationInMinutes value.
Following is a sample response for an action with only a threshold, severity, and suppress property
"etag": "W/\"datetime'2016-02-25T20%3A54%3A20.1302566Z'\"",
"properties": {
"Type": "Alert",
"Name": "My threshold action",
"Threshold": {
"Operator": "gt",
"Value": 10
},
"Throttling": {
"DurationInMinutes": 30
},
"Severity": "critical",
"Version": 1 }
Use the Put method with a unique action ID to create a new action for a schedule with severity.
Use the Put method with an existing action ID to modify a severity action for a schedule. The body of the
request must include the etag of the action.
Action Groups
All alerts in Azure, use Action Group as the default mechanism for handling actions. With Action Group, you
can specify your actions once and then associate the action group to multiple alerts - across Azure. Without the
need, to repeatedly declare the same actions over and over again. Action Groups support multiple actions -
including email, SMS, Voice Call, ITSM Connection, Automation Runbook, Webhook URI and more.
For users who have extended their alerts into Azure - a schedule should now have Action Group details passed
along with threshold, to be able to create an alert. E -mail details, Webhook URLs, Runbook Automation details,
and other Actions, need to be defined in side an Action Group first before creating an alert; one can create
Action Group from Azure Monitor in Portal or use Action Group API.
To add association of action group to an alert, specify the unique Azure Resource Manager ID of the action
group in the alert definition. A sample illustration is provided below:
"etag": "W/\"datetime'2017-12-13T10%3A52%3A21.1697364Z'\"",
"properties": {
"Type": "Alert",
"Name": "test-alert",
"Description": "I need to put a description here",
"Threshold": {
"Operator": "gt",
"Value": 12
},
"AzNsNotification": {
"GroupIds": [
"/subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-
group/providers/microsoft.insights/actiongroups/test-actiongroup"
]
},
"Severity": "critical",
"Version": 1
},
Use the Put method with a unique action ID to associate already existing Action Group for a schedule. The
following is a sample illustration of usage.
Use the Put method with an existing action ID to modify an Action Group associated for a schedule. The body
of the request must include the etag of the action.
$AzNsJson = "{'etag': 'datetime'2017-12-13T10%3A52%3A21.1697364Z'\"', 'properties': { 'Name': 'test-alert',
'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical',
'AzNsNotification': { 'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-
resource-group/providers/microsoft.insights/actiongroups/test-actiongroup'] } } }"
armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group
Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search
ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson
Customize Actions
By default actions, follow standard template and format for notifications. But user can customize some actions,
even if they are controlled by Action Groups. Currently, customization is possible for Email Subject and
Webhook Payload.
C u st o m i z e E- M a i l Su b j e c t fo r A c t i o n G r o u p
By default, the email subject for alerts is: Alert Notification <AlertName> for <WorkspaceName> . But this can be
customized, so that you can specific words or tags - to allow you to easily employ filter rules in your Inbox. The
customize email header details need to send along with ActionGroup details, as in sample below.
"etag": "W/\"datetime'2017-12-13T10%3A52%3A21.1697364Z'\"",
"properties": {
"Type": "Alert",
"Name": "test-alert",
"Description": "I need to put a description here",
"Threshold": {
"Operator": "gt",
"Value": 12
},
"AzNsNotification": {
"GroupIds": [
"/subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-
group/providers/microsoft.insights/actiongroups/test-actiongroup"
],
"CustomEmailSubject": "Azure Alert fired"
},
"Severity": "critical",
"Version": 1
},
Use the Put method with a unique action ID to associate already existing Action Group with customization for a
schedule. The following is a sample illustration of usage.
Use the Put method with an existing action ID to modify an Action Group associated for a schedule. The body
of the request must include the etag of the action.
$AzNsJson = "{'etag': 'datetime'2017-12-13T10%3A52%3A21.1697364Z'\"', 'properties': { 'Name': 'test-alert',
'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical',
'AzNsNotification': {'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-
resource-group/providers/microsoft.insights/actiongroups/test-actiongroup']}, 'CustomEmailSubject': 'Azure
Alert fired' } }"
armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group
Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search
ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson
C u st o m i z e W e b h o o k P a y l o a d fo r A c t i o n G r o u p
By default, the webhook sent via Action Group for log analytics has a fixed structure. But one can customize the
JSON payload by using specific variables supported, to meet requirements of the webhook endpoint. For more
information, see Webhook action for log alert rules.
The customize webhook details need to send along with ActionGroup details and will be applied to all
Webhook URI specified inside the action group; as in sample below.
"etag": "W/\"datetime'2017-12-13T10%3A52%3A21.1697364Z'\"",
"properties": {
"Type": "Alert",
"Name": "test-alert",
"Description": "I need to put a description here",
"Threshold": {
"Operator": "gt",
"Value": 12
},
"AzNsNotification": {
"GroupIds": [
"/subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-resource-
group/providers/microsoft.insights/actiongroups/test-actiongroup"
],
"CustomWebhookPayload": "{\"field1\":\"value1\",\"field2\":\"value2\"}",
"CustomEmailSubject": "Azure Alert fired"
},
"Severity": "critical",
"Version": 1
},
Use the Put method with a unique action ID to associate already existing Action Group with customization for a
schedule. The following is a sample illustration of usage.
Use the Put method with an existing action ID to modify an Action Group associated for a schedule. The body
of the request must include the etag of the action.
$AzNsJson = "{'etag': 'datetime'2017-12-13T10%3A52%3A21.1697364Z'\"', 'properties': { 'Name': 'test-alert',
'Version':'1', 'Type':'Alert', 'Threshold': { 'Operator': 'gt', 'Value': 12 },'Severity': 'critical',
'AzNsNotification': {'GroupIds': ['subscriptions/1234a45-123d-4321-12aa-123b12a5678/resourcegroups/my-
resource-group/providers/microsoft.insights/actiongroups/test-actiongroup']}, 'CustomEmailSubject': 'Azure
Alert fired','CustomWebhookPayload': '{\"field1\":\"value1\",\"field2\":\"value2\"}' } }"
armclient put /subscriptions/{Subscription ID}/resourceGroups/{Resource Group
Name}/Microsoft.OperationalInsights/workspaces/{Workspace Name}/savedSearches/{Search
ID}/schedules/{Schedule ID}/actions/myAzNsaction?api-version=2015-03-20 $AzNsJson
Next steps
Use the REST API to perform log searches in Log Analytics.
Learn about log alerts in Azure monitor
How to create, edit or manage log alert rules in Azure monitor
Connect Azure to ITSM tools using IT Service
Management Connector
1/21/2020 • 7 minutes to read • Edit Online
The IT Service Management Connector (ITSMC ) allows you to connect Azure and a supported IT Service
Management (ITSM ) product/service.
Azure services like Log Analytics and Azure Monitor provide tools to detect, analyze and troubleshoot issues with
your Azure and non-Azure resources. However, the work items related to an issue typically reside in an ITSM
product/service. The ITSM connector provides a bi-directional connection between Azure and ITSM tools to help
you resolve issues faster.
ITSMC supports connections with the following ITSM tools:
ServiceNow
System Center Service Manager
Provance
Cherwell
With ITSMC, you can:
Create work items in ITSM tool, based on your Azure alerts (metric alerts, Activity Log alerts and Log
Analytics alerts).
Optionally, you can sync your incident and change request data from your ITSM tool to an Azure Log Analytics
workspace.
Read more about the legal terms and privacy policy.
You can start using the ITSM Connector using the following steps:
1. Add the ITSM Connector Solution
2. Create an ITSM connection
3. Use the connection
3. In the OMS Workspace section, select the Azure Log Analytics workspace where you want to install the
solution.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite (OMS) to Azure Monitor, OMS
Workspaces are now referred to as Log Analytics workspaces.
The ITSM Connector can only be installed in Log Analytics workspaces in the following regions: East US, West
US2, South Central US, West Central US, Central Canada, West Europe, South UK, Southeast Asia, East Japan,
Central India, Southeast Australia.
4. In the OMS Workspace Settings section, select the ResourceGroup where you want to create the solution
resource.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite (OMS) to Azure Monitor, OMS
Workspaces are now referred to as Log Analytics workspaces.
5. Click Create.
When the solution resource is deployed, a notification appears at the top right- of the window.
4. Specify the connection settings as described in Configuring the ITSMC connection with your ITSM
products/services article.
NOTE
By default, ITSMC refreshes the connection's configuration data once in every 24 hours. To refresh your connection's
data instantly for any edits or template updates that you make, click the Sync button on your connection's blade.
Using the solution
By using the ITSM Connector solution, you can create work items from Azure alerts, Log Analytics alerts and Log
Analytics log records.
3. Provide Name and ShortName for your action group. Select the Resource Group and Subscription
where you want to create your action group.
4. In the Actions list, select ITSM from the drop-down menu for Action Type. Provide a Name for the action
and click Edit details.
5. Select the Subscription where your Log Analytics workspace is located. Select the Connection name
(your ITSM Connector name) followed by your Workspace name. For example,
"MyITSMMConnector(MyWorkspace)."
6. Select Work Item type from the drop-down menu. Choose to use an existing template or fill the fields
required by your ITSM product.
7. Click OK.
When creating/editing an Azure alert rule, use an Action group, which has an ITSM Action. When the alert
triggers, work item is created/updated in the ITSM tool.
NOTE
For information on pricing of ITSM Action, see the pricing page for Action Groups.
The dashboard also provides information on connector status which can be used as a starting point to analyze any
issues with the connections.
You can also visualize the incidents synced against the impacted computers, within the Service Map solution.
Service Map automatically discovers the application components on Windows and Linux systems and maps the
communication between services. It allows you to view your servers as you think of them – as interconnected
systems that deliver critical services. Service Map shows connections between servers, processes, and ports
across any TCP -connected architecture with no configuration required other than installation of an agent. Learn
more.
If you are using the Service Map solution, you can view the service desk items created in the ITSM solutions as
shown in the following example:
NOTE
Depending on the work item type imported into Log Analytics, ServiceDesk_CL contains the following fields:
ServiceDeskId_s Number
IncidentState_s State
Urgency_s Urgency
Impact_s Impact
Priority_s Priority
CreatedBy_s Opened by
ResolvedBy_s Resolved by
ClosedBy_s Closed by
AssignedTo_s Assigned to
Category_s Category
Description_s Notes
LOG ANALYTICS FIELD SERVICENOW FIELD
CreatedDate_t Opened
ClosedDate_t closed
ResolvedDate_t Resolved
ServiceDeskId_s Number
CreatedBy_s Requested by
ClosedBy_s Closed by
AssignedTo_s Assigned to
Type_s Type
Category_s Category
CRState_s State
Urgency_s Urgency
Priority_s Priority
Risk_s Risk
Impact_s Impact
Description_s Description
LOG ANALYTICS SERVICENOW FIELD
Contact us
For any queries or feedback on the IT Service Management Connector, contact us at
omsitsmfeedback@microsoft.com.
Next steps
Add ITSM products/services to IT Service Management Connector.
Connect ITSM products/services with IT Service
Management Connector
1/5/2020 • 13 minutes to read • Edit Online
This article provides information about how to configure the connection between your ITSM product/service and
the IT Service Management Connector (ITSMC ) in Log Analytics to centrally manage your work items. For more
information about ITSMC, see Overview.
The following ITSM products/services are supported. Select the product to view detailed information about how
to connect the product to ITSMC.
System Center Service Manager
ServiceNow
Provance
Cherwell
NOTE
ITSM Connector can only connect to cloud-based ServiceNow instances. On-premises ServiceNow instances are currently not
supported.
NOTE
All these parameters are mandatory.
FIELD DESCRIPTION
Connection Name Type a name for the System Center Service Manager instance
that you want to connect with ITSMC. You use this name later
when you configure work items in this instance/ view detailed
log analytics.
Server URL Type the URL of the Service Manager Web app. More
information about Service Manager Web app is here.
Client ID Type the client ID that you generated (using the automatic
script) for authenticating the Web app. More information
about the automated script is here.
Client Secret Type the client secret, generated for this ID.
Sync Data Select the Service Manager work items that you want to sync
through ITSMC. These work items are imported into Log
Analytics. Options: Incidents, Change Requests.
Data Sync Scope Type the number of past days that you want the data from.
Maximum limit: 120 days.
FIELD DESCRIPTION
Create new configuration item in ITSM solution Select this option if you want to create the configuration items
in the ITSM product. When selected, Log Analytics creates the
affected CIs as configuration items (in case of non-existing
CIs) in the supported ITSM system. Default: disabled.
5. In the Add Hybrid Connections blade, click Create new hybrid Connection.
6. Type the following values:
EndPoint Name: Specify a name for the new Hybrid connection.
EndPoint Host: FQDN of the Service Manager management server.
EndPoint Port: Type 5724
Servicebus namespace: Use an existing servicebus namespace or create a new one.
Location: select the location.
Name: Specify a name to the servicebus if you are creating it.
7. Click OK to close the Create hybrid connection blade and start creating the hybrid connection.
Once the Hybrid connection is created, it is displayed under the blade.
8. After the hybrid connection is created, select the connection and click Add selected hybrid connection.
Configure the listener setup
Use the following procedure to configure the listener setup for the hybrid connection.
1. In the Hybrid Connections blade, click Download the Connection Manager and install it on the
machine where System Center Service Manager instance is running.
Once the installation is complete, Hybrid Connection Manager UI option is available under Start menu.
2. Click Hybrid Connection Manager UI , you will be prompted for your Azure credentials.
3. Login with your Azure credentials and select your subscription where the Hybrid connection was created.
4. Click Save.
Your hybrid connection is successfully connected.
NOTE
After the hybrid connection is created, verify and test the connection by visiting the deployed Service Manager Web app.
Ensure the connection is successful before you try to connect to ITSMC in Azure.
NOTE
All these parameters are mandatory.
FIELD DESCRIPTION
Connection Name Type a name for the ServiceNow instance that you want to
connect with ITSMC. You use this name later in Log Analytics
when you configure work items in this ITSM/ view detailed log
analytics.
Username Type the integration user name that you created in the
ServiceNow app to support the connection to ITSMC. More
information: Create ServiceNow app user role.
FIELD DESCRIPTION
Password Type the password associated with this user name. Note: User
name and password are used for generating authentication
tokens only, and are not stored anywhere within the ITSMC
service.
Server URL Type the URL of the ServiceNow instance that you want to
connect to ITSMC.
Client ID Type the client ID that you want to use for OAuth2
Authentication, which you generated earlier. More information
on generating client ID and secret: OAuth Setup.
Client Secret Type the client secret, generated for this ID.
Data Sync Scope Select the ServiceNow work items that you want to sync to
Azure Log Analytics, through the ITSMC. The selected values
are imported into log analytics. Options: Incidents and
Change Requests.
Sync Data Type the number of past days that you want the data from.
Maximum limit: 120 days.
Create new configuration item in ITSM solution Select this option if you want to create the configuration items
in the ITSM product. When selected, ITSMC creates the
affected CIs as configuration items (in case of non-existing
CIs) in the supported ITSM system. Default: disabled.
When successfully connected, and synced:
Selected work items from ServiceNow instance are imported into Azure Log Analytics. You can view the
summary of these work items on the IT Service Management Connector tile.
You can create incidents from Log Analytics alerts or from log records, or from Azure alerts in this
ServiceNow instance.
Learn more: Create ITSM work items from Azure alerts.
Create integration user role in ServiceNow app
User the following procedure:
1. Visit the ServiceNow store and install the User App for ServiceNow and Microsoft OMS Integration
into your ServiceNow Instance.
NOTE
As part of the ongoing transition from Microsoft Operations Management Suite (OMS) to Azure Monitor, OMS is
now referred to as Log Analytics.
2. After installation, visit the left navigation bar of the ServiceNow instance, search, and select Microsoft OMS
integrator.
3. Click Installation Checklist.
The status is displayed as Not complete if the user role is yet to be created.
4. In the text boxes, next to Create integration user, enter the user name for the user that can connect to
ITSMC in Azure.
5. Enter the password for this user, and click OK.
NOTE
You use these credentials to make the ServiceNow connection in Azure.
The newly created user is displayed with the default roles assigned.
Default roles:
personalize_choices
import_transformer
x_mioms_microsoft.user
itil
template_editor
view_changer
Once the user is successfully created, the status of Check Installation Checklist moves to Completed, listing the
details of the user role created for the app.
NOTE
ITSM Connector can send incidents to ServiceNow without any other modules installed on your ServiceNow instance. If you
are using EventManagement module in your ServiceNow instance and wish to create Events or Alerts in ServiceNow using
the connector, add the following roles to the integration user:
evt_mgmt_integration
evt_mgmt_operator
NOTE
All these parameters are mandatory.
FIELD DESCRIPTION
Connection Name Type a name for the Provance instance that you want to
connect with ITSMC. You use this name later when you
configure work items in this ITSM/ view detailed log analytics.
Password Type the password associated with this user name. Note:
User name and password are used for generating
authentication tokens only, and are not stored anywhere
within the ITSMC service._
Server URL Type the URL of your Provance instance that you want to
connect to ITSMC.
FIELD DESCRIPTION
Data Sync Scope Select the Provance work items that you want to sync to
Azure Log Analytics, through ITSMC. These work items are
imported into log analytics. Options: Incidents, Change
Requests.
Sync Data Type the number of past days that you want the data from.
Maximum limit: 120 days.
Create new configuration item in ITSM solution Select this option if you want to create the configuration items
in the ITSM product. When selected, ITSMC creates the
affected CIs as configuration items (in case of non-existing
CIs) in the supported ITSM system. Default: disabled.
NOTE
All these parameters are mandatory.
FIELD DESCRIPTION
FIELD DESCRIPTION
Connection Name Type a name for the Cherwell instance that you want to
connect to ITSMC. You use this name later when you
configure work items in this ITSM/ view detailed log analytics.
Username Type the Cherwell user name that can connect to ITSMC.
Password Type the password associated with this user name. Note:
User name and password are used for generating
authentication tokens only, and are not stored anywhere
within the ITSMC service.
Server URL Type the URL of your Cherwell instance that you want to
connect to ITSMC.
Data Sync Scope Select the Cherwell work items that you want to sync through
ITSMC. These work items are imported into log analytics.
Options: Incidents, Change Requests.
Sync Data Type the number of past days that you want the data from.
Maximum limit: 120 days.
Create new configuration item in ITSM solution Select this option if you want to create the configuration items
in the ITSM product. When selected, ITSMC creates the
affected CIs as configuration items (in case of non-existing
CIs) in the supported ITSM system. Default: disabled.
When successfully connected, and synced:
Selected work items from this Cherwell instance are imported into Azure Log Analytics. You can view the
summary of these work items on the IT Service Management Connector tile.
You can create incidents from Log Analytics alerts or from log records, or from Azure alerts in this Cherwell
instance.
Learn more: Create ITSM work items from Azure alerts.
Generate client ID for Cherwell
To generate the client ID/key for Cherwell, use the following procedure:
1. Log in to your Cherwell instance as admin.
2. Click Security > Edit REST API client settings.
3. Select Create new client > client secret.
Next steps
Create ITSM work items from Azure alerts
Create Service Manager Web app using the
automated script
10/25/2019 • 4 minutes to read • Edit Online
Use the following script to create the Web app for your Service Manager instance. More information about
Service Manager connection is here: Service Manager Web app
Run the script by providing the following required details:
Azure subscription details
Resource group name
Location
Service Manager server details (server name, domain, username, and password)
Site name prefix for your Web app
ServiceBus Namespace.
The script will create the Web app using the name that you specified (along with few additional strings to make it
unique). It generates the Web app URL, client ID, and client secret.
Save these values, you will need these values when you create a connection with IT Service Management
Connector.
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Prerequisites
Windows Management Framework 5.0 or above. Windows 10 has 5.1 by default. You can download the
framework from here:
Use the following script:
####################################
# User Configuration Section Begins
####################################
# Resource group name for resource deployment. Could be an existing resource group or a new one to be created.
$resourceGroupName = ""
# Azure site Name Prefix. Default is "smoc". It can be configured to any desired value.
$siteNamePrefix = ""
# Service Bus namespace. Please provide an already existing service bus namespace.
# If it doesn't exist, a new one will be created with name $siteName + "sbn" which can also be later reused
for any other hybrid connections.
$serviceName = ""
##################################
# User Configuration Section Ends
##################################
################
# Installations
################
#############
# Parameters
#############
$errorActionPreference = "Stop"
$templateUri = "https://raw.githubusercontent.com/Azure/SMOMSConnector/master/azuredeploy.json"
if(!$siteNamePrefix)
{
$siteNamePrefix = "smoc"
}
Connect-AzAccount
}while($resource)
$azureSite = "https://"+$siteName+".azurewebsites.net"
##############
# MAIN Begins
##############
$tenant = $context.Tenant.Id
if(!$tenant)
{
#For backward compatibility with older versions
$tenant = $context.Tenant.TenantId
}
try
{
Get-AzResourceGroup -Name $resourceGroupName
}
catch
{
New-AzResourceGroup -Location $location -Name $resourceGroupName
}
# AAD Authentication
####################
$secret = [System.Web.Security.Membership]::GeneratePassword(30,2).ToString()
$clientSecret = $secret | ConvertTo-SecureString -AsPlainText -Force
try
{
exit
$clientId = $adApp.ApplicationId
$appSettingList = $webApp.SiteConfig.AppSettings
$appSettings = @{}
ForEach ($item in $appSettingList) {
$appSettings[$item.Name] = $item.Value
}
$appSettings['ida:Tenant'] = $tenant
$appSettings['ida:Audience'] = $azureSite
$appSettings['ida:ServerName'] = $serverName
$appSettings['ida:Domain'] = $domain
$appSettings['ida:Username'] = $userName
$appSettings['ida:WhitelistedClientId'] = $clientId
$connStrings = @{}
$kvp = @{"Type"="Custom"; "Value"=$password}
$connStrings['ida:Password'] = $kvp
}
catch
{
Write-Host "Web App configuration failed. Please ensure all values are provided in Service Manager
Authentication Settings in User Configuration Section"
exit
}
# Relay Namespace
###################
if(!$serviceName)
{
$serviceName = $siteName + "sbn"
}
if(!$resource)
{
$serviceName = $siteName + "sbn"
$properties = @{
"sku" = @{
"name"= "Standard"
"tier"= "Standard"
"capacity"= 1
}
}
try
{
Write-Host "Creating Service Bus namespace..."
New-AzResource -ResourceName $serviceName -Location $location -PropertyObject $properties -
ResourceGroupName $resourceGroupName -ResourceType Microsoft.Relay/namespaces -ApiVersion 2016-07-01 -Force
}
catch
{
$err = $TRUE
Write-Host "Creation of Service Bus Namespace failed...Please create it manually from Azure Portal.`n"
}
Write-Host "Note: Please Configure Hybrid connection in the Networking section of the web application in Azure
Portal to link to the on-premises system.`n"
Write-Host "App Details"
Write-Host "============"
Write-Host "App Name:" $siteName
Write-Host "Client Id:" $clientId
Write-Host "Client Secret:" $secret
Write-Host "URI:" $azureSite
if(!$err)
{
Write-Host "ServiceBus Namespace:" $serviceName
}```
## Next steps
[Configure the Hybrid connection](../../azure-monitor/platform/itsmc-connections.md#configure-the-hybrid-
connection).
connection).
Overview of autoscale in Microsoft Azure Virtual
Machines, Cloud Services, and Web Apps
12/23/2019 • 4 minutes to read • Edit Online
This article describes what Microsoft Azure autoscale is, its benefits, and how to get started using it.
Azure Monitor autoscale applies only to Virtual Machine Scale Sets, Cloud Services, App Service - Web Apps, and
API Management services.
NOTE
Azure has two autoscale methods. An older version of autoscale applies to Virtual Machines (availability sets). This feature
has limited support and we recommend migrating to virtual machine scale sets for faster and more reliable autoscale
support. A link on how to use the older technology is included in this article.
What is autoscale?
Autoscale allows you to have the right amount of resources running to handle the load on your application. It
allows you to add resources to handle increases in load and also save money by removing resources that are
sitting idle. You specify a minimum and maximum number of instances to run and add or remove VMs
automatically based on a set of rules. Having a minimum makes sure your application is always running even
under no load. Having a maximum limits your total possible hourly cost. You automatically scale between these
two extremes using rules you create.
When rule conditions are met, one or more autoscale actions are triggered. You can add and remove VMs, or
perform other actions. The following conceptual diagram shows this process.
The following explanation applies to the pieces of the previous diagram.
Resource Metrics
Resources emit metrics, these metrics are later processed by rules. Metrics come via different methods. Virtual
machine scale sets use telemetry data from Azure diagnostics agents whereas telemetry for Web apps and Cloud
services comes directly from the Azure Infrastructure. Some commonly used statistics include CPU Usage,
memory usage, thread counts, queue length, and disk usage. For a list of what telemetry data you can use, see
Autoscale Common Metrics.
Custom Metrics
You can also leverage your own custom metrics that your application(s) may be emitting. If you have configured
your application(s) to send metrics to Application Insights you can leverage those metrics to make decisions on
whether to scale or not.
Time
Schedule-based rules are based on UTC. You must set your time zone properly when setting up your rules.
Rules
The diagram shows only one autoscale rule, but you can have many of them. You can create complex overlapping
rules as needed for your situation. Rule types include
Metric-based - For example, do this action when CPU usage is above 50%.
Time-based - For example, trigger a webhook every 8am on Saturday in a given time zone.
Metric-based rules measure application load and add or remove VMs based on that load. Schedule-based rules
allow you to scale when you see time patterns in your load and want to scale before a possible load increase or
decrease occurs.
Autoscale Settings
Autoscale use the following terminology and structure.
An autoscale setting is read by the autoscale engine to determine whether to scale up or down. It
contains one or more profiles, information about the target resource, and notification settings.
An autoscale profile is a combination of a:
capacity setting, which indicates the minimum, maximum, and default values for number of
instances.
set of rules, each of which includes a trigger (time or metric) and a scale action (up or down).
recurrence, which indicates when autoscale should put this profile into effect.
You can have multiple profiles, which allow you to take care of different overlapping
requirements. You can have different autoscale profiles for different times of day or days of
the week, for example.
A notification setting defines what notifications should occur when an autoscale event occurs
based on satisfying the criteria of one of the autoscale setting’s profiles. Autoscale can notify one or
more email addresses or make calls to one or more webhooks.
The full list of configurable fields and descriptions is available in the Autoscale REST API.
For code examples, see
Advanced Autoscale configuration using Resource Manager templates for VM Scale Sets
Autoscale REST API
Virtual Machines: Windows Scale Sets Scaling virtual machine scale sets in Windows
Virtual Machines: Linux Scale Sets Scaling virtual machine scale sets in Linux
Virtual Machines: Windows Example Advanced Autoscale configuration using Resource Manager
templates for VM Scale Sets
Next steps
To learn more about autoscale, use the Autoscale Walkthroughs listed previously or refer to the following
resources:
Azure Monitor autoscale common metrics
Best practices for Azure Monitor autoscale
Use autoscale actions to send email and webhook alert notifications
Autoscale REST API
Troubleshooting Virtual Machine Scale Sets Autoscale
Get started with Autoscale in Azure
12/23/2019 • 3 minutes to read • Edit Online
This article describes how to set up your Autoscale settings for your resource in the Microsoft Azure portal.
Azure Monitor autoscale applies only to Virtual Machine Scale Sets, Cloud Services, App Service - Web Apps, and
API Management services.
3. Click Autoscale to view all the resources for which Autoscale is applicable, along with their current Autoscale
status.
You can use the filter pane at the top to scope down the list to select resources in a specific resource group, specific
resource types, or a specific resource.
For each resource, you will find the current instance count and the Autoscale status. The Autoscale status can be:
Not configured: You have not enabled Autoscale yet for this resource.
Enabled: You have enabled Autoscale for this resource.
Disabled: You have disabled Autoscale for this resource.
3. Provide a name for the scale setting, and then click Add a rule. Notice the scale rule options that open as a
context pane on the right side. By default, this sets the option to scale your instance count by 1 if the CPU
percentage of the resource exceeds 70 percent. Leave it at its default values and click Add.
4. You've now created your first scale rule. Note that the UX recommends best practices and states that "It is
recommended to have at least one scale in rule." To do so:
a. Click Add a rule.
b. Set Operator to Less than.
c. Set Threshold to 20.
d. Set Operation to Decrease count by.
You should now have a scale setting that scales out/scales in based on CPU usage.
5. Click Save.
Congratulations! You've now successfully created your first scale setting to autoscale your web app based on CPU
usage.
NOTE
The same steps are applicable to get started with a virtual machine scale set or cloud service role.
Other considerations
Scale based on a schedule
In addition to scale based on CPU, you can set your scale differently for specific days of the week.
1. Click Add a scale condition.
2. Setting the scale mode and the rules is the same as the default condition.
3. Select Repeat specific days for the schedule.
4. Select the days and the start/end time for when the scale condition should be applied.
If you want to view the complete scale history (for up to 90 days), select Click here to see more details. The
activity log opens, with Autoscale pre-selected for your resource and category.
View the scale definition of your resource
Autoscale is an Azure Resource Manager resource. You can view the scale definition in JSON by switching to the
JSON tab.
You can make changes in JSON directly, if required. These changes will be reflected after you save them.
Disable Autoscale and manually scale your instances
There might be times when you want to disable your current scale setting and manually scale your resource.
Click the Disable autoscale button at the top.
NOTE
This option disables your configuration. However, you can get back to it after you enable Autoscale again.
You can now set the number of instances that you want to scale to manually.
You can always return to Autoscale by clicking Enable autoscale and then Save.
Next steps
Create an Activity Log Alert to monitor all Autoscale engine operations on your subscription
Create an Activity Log Alert to monitor all failed Autoscale scale-in/scale-out operations on your subscription
Understand Autoscale settings
12/23/2019 • 10 minutes to read • Edit Online
Autoscale settings help ensure that you have the right amount of resources running to handle the fluctuating load
of your application. You can configure Autoscale settings to be triggered based on metrics that indicate load or
performance, or triggered at a scheduled date and time. This article takes a detailed look at the anatomy of an
Autoscale setting. The article begins with the schema and properties of a setting, and then walks through the
different profile types that can be configured. Finally, the article discusses how the Autoscale feature in Azure
evaluates which profile to execute at any given time.
NOTE
A setting can have multiple profiles. To learn more, see the profiles section. A profile can also have multiple scale-out rules
and scale-in rules defined. To see how they are evaluated, see the evaluation section.
{
"id": "/subscriptions/s1/resourceGroups/rg1/providers/microsoft.insights/autoscalesettings/setting1",
"name": "setting1",
"type": "Microsoft.Insights/autoscaleSettings",
"location": "East US",
"properties": {
"enabled": true,
"targetResourceUri":
"/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.Compute/virtualMachineScaleSets/vmss1",
"profiles": [
{
"name": "mainProfile",
"capacity": {
"minimum": "1",
"maximum": "4",
"default": "1"
},
"rules": [
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri":
"/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.Compute/virtualMachineScaleSets/vmss1",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT10M",
"timeAggregation": "Average",
"operator": "GreaterThan",
"threshold": 85
},
"scaleAction": {
"direction": "Increase",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri":
"/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.Compute/virtualMachineScaleSets/vmss1",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT10M",
"timeAggregation": "Average",
"operator": "LessThan",
"threshold": 60
},
"scaleAction": {
"direction": "Decrease",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
}
]
}
]
}
}
"profiles": [{
"name": " regularProfile",
"capacity": {
...
},
"rules": [{
...
},
{
...
}]
},
{
"name": "eventProfile",
"capacity": {
...
},
"rules": [{
...
}, {
...
}],
"fixedDate": {
"timeZone": "Pacific Standard Time",
"start": "2017-12-26T00:00:00",
"end": "2017-12-26T23:59:00"
}}
]
Recurrence profile: This type of profile enables you to ensure that this profile is always used on a
particular day of the week. Recurrence profiles only have a start time. They run until the next recurrence
profile or fixed date profile is set to start. An Autoscale setting with only one recurrence profile runs that
profile, even if there is a regular profile defined in the same setting. The following two examples illustrate
how this profile is used:
Example 1: Weekdays vs. weekends
Let’s say that on weekends, you want your maximum capacity to be 4. On weekdays, because you expect
more load, you want your maximum capacity to be 10. In this case, your setting would contain two
recurrence profiles, one to run on weekends and the other on weekdays. The setting looks like this:
"profiles": [
{
"name": "weekdayProfile",
"capacity": {
...
},
"rules": [{
...
}],
"recurrence": {
"frequency": "Week",
"schedule": {
"timeZone": "Pacific Standard Time",
"days": [
"Monday"
],
"hours": [
0
],
"minutes": [
0
]
}
}}
},
{
"name": "weekendProfile",
"capacity": {
...
},
"rules": [{
...
}]
"recurrence": {
"frequency": "Week",
"schedule": {
"timeZone": "Pacific Standard Time",
"days": [
"Saturday"
],
"hours": [
0
],
"minutes": [
0
]
}
}
}]
The preceding setting shows that each recurrence profile has a schedule. This schedule determines when the
profile starts running. The profile stops when it’s time to run another profile.
For example, in the preceding setting, “weekdayProfile” is set to start on Monday at 12:00 AM. That means
this profile starts running on Monday at 12:00 AM. It continues until Saturday at 12:00 AM, when
“weekendProfile” is scheduled to start running.
Example 2: Business hours
Let's say you want to have one metric threshold during business hours (9:00 AM to 5:00 PM ), and a
different one for all other times. The setting would look like this:
"profiles": [
{
"name": "businessHoursProfile",
"capacity": {
...
},
"rules": [{
...
}],
"recurrence": {
"frequency": "Week",
"schedule": {
"timeZone": "Pacific Standard Time",
"days": [
"Monday", “Tuesday”, “Wednesday”, “Thursday”, “Friday”
],
"hours": [
9
],
"minutes": [
0
]
}
}
},
{
"name": "nonBusinessHoursProfile",
"capacity": {
...
},
"rules": [{
...
}]
"recurrence": {
"frequency": "Week",
"schedule": {
"timeZone": "Pacific Standard Time",
"days": [
"Monday", “Tuesday”, “Wednesday”, “Thursday”, “Friday”
],
"hours": [
17
],
"minutes": [
0
]
}
}
}]
The preceding setting shows that “businessHoursProfile” begins running on Monday at 9:00 AM, and
continues to 5:00 PM. That’s when “nonBusinessHoursProfile” starts running. The
“nonBusinessHoursProfile” runs until 9:00 AM Tuesday, and then the “businessHoursProfile” takes over
again. This repeats until Friday at 5:00 PM. At that point, “nonBusinessHoursProfile” runs all the way to
Monday at 9:00 AM.
NOTE
The Autoscale user interface in the Azure portal enforces end times for recurrence profiles, and begins running the Autoscale
setting's default profile in between recurrence profiles.
Autoscale evaluation
Given that Autoscale settings can have multiple profiles, and each profile can have multiple metric rules, it is
important to understand how an Autoscale setting is evaluated. Each time the Autoscale job runs, it begins by
choosing the profile that is applicable. Then Autoscale evaluates the minimum and maximum values, and any
metric rules in the profile, and decides if a scale action is necessary.
Which profile will Autoscale pick?
Autoscale uses the following sequence to pick the profile:
1. It first looks for any fixed date profile that is configured to run now. If there is, Autoscale runs it. If there are
multiple fixed date profiles that are supposed to run, Autoscale selects the first one.
2. If there are no fixed date profiles, Autoscale looks at recurrence profiles. If a recurrence profile is found, it runs it.
3. If there are no fixed date or recurrence profiles, Autoscale runs the regular profile.
How does Autoscale evaluate multiple rules?
After Autoscale determines which profile to run, it evaluates all the scale-out rules in the profile (these are rules
with direction = “Increase”).
If one or more scale-out rules are triggered, Autoscale calculates the new capacity determined by the scaleAction
of each of those rules. Then it scales out to the maximum of those capacities, to ensure service availability.
For example, let's say there is a virtual machine scale set with a current capacity of 10. There are two scale-out
rules: one that increases capacity by 10 percent, and one that increases capacity by 3 counts. The first rule would
result in a new capacity of 11, and the second rule would result in a capacity of 13. To ensure service availability,
Autoscale chooses the action that results in the maximum capacity, so the second rule is chosen.
If no scale-out rules are triggered, Autoscale evaluates all the scale-in rules (rules with direction = “Decrease”).
Autoscale only takes a scale-in action if all of the scale-in rules are triggered.
Autoscale calculates the new capacity determined by the scaleAction of each of those rules. Then it chooses the
scale action that results in the maximum of those capacities to ensure service availability.
For example, let's say there is a virtual machine scale set with a current capacity of 10. There are two scale-in rules:
one that decreases capacity by 50 percent, and one that decreases capacity by 3 counts. The first rule would result
in a new capacity of 5, and the second rule would result in a capacity of 7. To ensure service availability, Autoscale
chooses the action that results in the maximum capacity, so the second rule is chosen.
Next steps
Learn more about Autoscale by referring to the following:
Overview of autoscale
Azure Monitor autoscale common metrics
Best practices for Azure Monitor autoscale
Use autoscale actions to send email and webhook alert notifications
Autoscale REST API
Best practices for Autoscale
12/23/2019 • 9 minutes to read • Edit Online
Azure Monitor autoscale applies only to Virtual Machine Scale Sets, Cloud Services, App Service - Web Apps, and
API Management services.
Autoscale concepts
A resource can have only one autoscale setting
An autoscale setting can have one or more profiles and each profile can have one or more autoscale rules.
An autoscale setting scales instances horizontally, which is out by increasing the instances and in by decreasing
the number of instances. An autoscale setting has a maximum, minimum, and default value of instances.
An autoscale job always reads the associated metric to scale by, checking if it has crossed the configured
threshold for scale-out or scale-in. You can view a list of metrics that autoscale can scale by at Azure Monitor
autoscaling common metrics.
All thresholds are calculated at an instance level. For example, "scale out by one instance when average CPU >
80% when instance count is 2", means scale-out when the average CPU across all instances is greater than
80%.
All autoscale failures are logged to the Activity Log. You can then configure an activity log alert so that you can
be notified via email, SMS, or webhooks whenever there is an autoscale failure.
Similarly, all successful scale actions are posted to the Activity Log. You can then configure an activity log alert
so that you can be notified via email, SMS, or webhooks whenever there is a successful autoscale action. You
can also configure email or webhook notifications to get notified for successful scale actions via the
notifications tab on the autoscale setting.
Next Steps
Create an Activity Log Alert to monitor all autoscale engine operations on your subscription.
Create an Activity Log Alert to monitor all failed autoscale scale in/scale out operations on your subscription
Azure Monitor autoscaling common metrics
12/23/2019 • 5 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Azure Monitor autoscaling allows you to scale the number of running instances up or down, based on telemetry
data (metrics). This document describes common metrics that you might want to use. In the Azure portal, you can
choose the metric of the resource to scale by. However, you can also choose any metric from a different resource
to scale by.
Azure Monitor autoscale applies only to Virtual Machine Scale Sets, Cloud Services, App Service - Web Apps, and
API Management services. Other Azure services use different scaling methods.
\System\Processes Count
\Memory\AvailableMemory Bytes
\Memory\PercentAvailableMemory Percent
\Memory\UsedMemory Bytes
\Memory\PercentUsedMemory Percent
\Memory\PercentUsedByCache Percent
\Memory\PagesPerSec CountPerSecond
\Memory\PagesReadPerSec CountPerSecond
\Memory\PagesWrittenPerSec CountPerSecond
\Memory\AvailableSwap Bytes
\Memory\PercentAvailableSwap Percent
\Memory\UsedSwap Bytes
\Memory\PercentUsedSwap Percent
\Processor\PercentIdleTime Percent
\Processor\PercentUserTime Percent
\Processor\PercentNiceTime Percent
\Processor\PercentPrivilegedTime Percent
METRIC NAME UNIT
\Processor\PercentInterruptTime Percent
\Processor\PercentDPCTime Percent
\Processor\PercentProcessorTime Percent
\Processor\PercentIOWaitTime Percent
\PhysicalDisk\BytesPerSecond BytesPerSecond
\PhysicalDisk\ReadBytesPerSecond BytesPerSecond
\PhysicalDisk\WriteBytesPerSecond BytesPerSecond
\PhysicalDisk\TransfersPerSecond CountPerSecond
\PhysicalDisk\ReadsPerSecond CountPerSecond
\PhysicalDisk\WritesPerSecond CountPerSecond
\PhysicalDisk\AverageReadTime Seconds
\PhysicalDisk\AverageWriteTime Seconds
\PhysicalDisk\AverageTransferTime Seconds
\PhysicalDisk\AverageDiskQueueLength Count
\NetworkInterface\BytesTransmitted Bytes
\NetworkInterface\BytesReceived Bytes
\NetworkInterface\PacketsTransmitted Count
\NetworkInterface\PacketsReceived Count
\NetworkInterface\BytesTotal Bytes
\NetworkInterface\TotalRxErrors Count
\NetworkInterface\TotalTxErrors Count
\NetworkInterface\TotalCollisions Count
CpuPercentage Percent
MemoryPercentage Percent
DiskQueueLength Count
HttpQueueLength Count
BytesReceived Bytes
BytesSent Bytes
"metricName": "ApproximateMessageCount",
"metricNamespace": "",
"metricResourceUri":
"/subscriptions/SUBSCRIPTION_ID/resourceGroups/RES_GROUP_NAME/providers/Microsoft.ClassicStorage/storageAccoun
ts/STORAGE_ACCOUNT_NAME/services/queue/queues/QUEUE_NAME"
"metricName": "ApproximateMessageCount",
"metricNamespace": "",
"metricResourceUri":
"/subscriptions/SUBSCRIPTION_ID/resourceGroups/RES_GROUP_NAME/providers/Microsoft.Storage/storageAccounts/STOR
AGE_ACCOUNT_NAME/services/queue/queues/QUEUE_NAME"
"metricName": "MessageCount",
"metricNamespace": "",
"metricResourceUri":
"/subscriptions/SUBSCRIPTION_ID/resourceGroups/RES_GROUP_NAME/providers/Microsoft.ServiceBus/namespaces/SB_NAM
ESPACE/queues/QUEUE_NAME"
NOTE
For Service Bus, the resource group concept does not exist but Azure Resource Manager creates a default resource group
per region. The resource group is usually in the 'Default-ServiceBus-[region]' format. For example, 'Default-ServiceBus-
EastUS', 'Default-ServiceBus-WestUS', 'Default-ServiceBus-AustraliaEast' etc.
Overview of common autoscale patterns
12/23/2019 • 2 minutes to read • Edit Online
This article describes some of the common patterns to scale your resource in Azure.
Azure Monitor autoscale applies only to Virtual Machine Scale Sets, Cloud Services, App Service - Web Apps, and
API Management services.
This article describes how to scale your resource by a custom metric in Azure portal.
Azure Monitor autoscale applies only to Virtual Machine Scale Sets, Cloud Services, App Service - Web Apps, and
API Management services.
Click on Autoscale setting to view all the resources for which auto scale is applicable, along with its current
autoscale status
Open 'Autoscale' blade in Azure Monitor and select a resource you want to scale
Note: The steps below use an app service plan associated with a web app that has app insights
configured.
In the scale setting blade for the resource, notice that the current instance count is 1. Click on 'Enable autoscale'.
Provide a name for the scale setting, and the click on "Add a rule". Notice the scale rule options that opens as a
context pane in the right hand side. By default, it sets the option to scale your instance count by 1 if the CPU
percentage of the resource exceeds 70%. Change the metric source at the top to "Application Insights", select
the app insights resource in the 'Resource' dropdown and then select the custom metric based on which you
want to scale.
Similar to the step above, add a scale rule that will scale in and decrease the scale count by 1 if the custom
metric is below a threshold.
Set the instance limits. For example, if you want to scale between 2-5 instances depending on the custom metric
fluctuations, set 'minimum' to '2', 'maximum' to '5' and 'default' to '2'
Note: In case there is a problem reading the resource metrics and the current capacity is below the
default capacity, then to ensure the availability of the resource, Autoscale will scale out to the default
value. If the current capacity is already higher than default capacity, Autoscale will not scale in.
Click on 'Save'
Congratulations. You now successfully created your scale setting to auto scale your web app based on a custom
metric.
Note: The same steps are applicable to get started with a VMSS or cloud service role.
2 minutes to read
Advanced autoscale configuration using Resource
Manager templates for VM Scale Sets
12/23/2019 • 5 minutes to read • Edit Online
You can scale-in and scale-out in Virtual Machine Scale Sets based on performance metric thresholds, by a
recurring schedule, or by a particular date. You can also configure email and webhook notifications for scale
actions. This walkthrough shows an example of configuring all these objects using a Resource Manager template
on a VM Scale Set.
NOTE
While this walkthrough explains the steps for VM Scale Sets, the same information applies to autoscaling Cloud Services, App
Service - Web Apps, and API Management services For a simple scale in/out setting on a VM Scale Set based on a simple
performance metric such as CPU, refer to the Linux and Windows documents
Walkthrough
In this walkthrough, we use Azure Resource Explorer to configure and update the autoscale setting for a scale set.
Azure Resource Explorer is an easy way to manage Azure resources via Resource Manager templates. If you are
new to Azure Resource Explorer tool, read this introduction.
1. Deploy a new scale set with a basic autoscale setting. This article uses the one from the Azure QuickStart
Gallery, which has a Windows scale set with a basic autoscale template. Linux scale sets work the same way.
2. After the scale set is created, navigate to the scale set resource from Azure Resource Explorer. You see the
following under Microsoft.Insights node.
The template execution has created a default autoscale setting with the name 'autoscalewad'. On the
right-hand side, you can view the full definition of this autoscale setting. In this case, the default autoscale
setting comes with a CPU% based scale-out and scale-in rule.
3. You can now add more profiles and rules based on the schedule or specific requirements. We create an
autoscale setting with three profiles. To understand profiles and rules in autoscale, review Autoscale Best
Practices.
{
"name": "Perf_Based_Scale",
"capacity": {
"minimum": "2",
"maximum": "12",
"default": "2"
},
"rules": [
{
"metricTrigger": {
"metricName": "MessageCount",
"metricNamespace": "",
"metricResourceUri":
"/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.ServiceBus/namespaces/mySB/queues/myqueue",
"timeGrain": "PT5M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "GreaterThan",
"threshold": 10
},
"scaleAction": {
"direction": "Increase",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
"metricTrigger": {
"metricName": "MessageCount",
"metricNamespace": "",
"metricResourceUri":
"/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.ServiceBus/namespaces/mySB/queues/myqueue",
"timeGrain": "PT5M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "LessThan",
"threshold": 3
},
"scaleAction": {
"direction": "Decrease",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricNamespace": "",
"metricResourceUri":
"/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.Compute/virtualMachineScaleSets/<this_vmss_na
me>",
"timeGrain": "PT5M",
"statistic": "Average",
"timeWindow": "PT30M",
"timeAggregation": "Average",
"operator": "GreaterThan",
"threshold": 85
},
"scaleAction": {
"direction": "Increase",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricNamespace": "",
"metricResourceUri":
"/subscriptions/s1/resourceGroups/rg1/providers/Microsoft.Compute/virtualMachineScaleSets/<this_vmss_na
me>",
"timeGrain": "PT5M",
"statistic": "Average",
"timeWindow": "PT30M",
"timeAggregation": "Average",
"operator": "LessThan",
"threshold": 60
},
"scaleAction": {
"direction": "Decrease",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
}
]
},
{
"name": "Weekday_Morning_Hours_Scale",
"capacity": {
"minimum": "4",
"maximum": "12",
"default": "4"
},
"rules": [],
"recurrence": {
"frequency": "Week",
"schedule": {
"timeZone": "Pacific Standard Time",
"days": [
"Monday",
"Tuesday",
"Wednesday",
"Thursday",
"Friday"
],
"hours": [
6
],
"minutes": [
0
]
}
}
}
},
{
"name": "Product_Launch_Day",
"capacity": {
"minimum": "6",
"maximum": "20",
"default": "6"
},
"rules": [],
"fixedDate": {
"timeZone": "Pacific Standard Time",
"start": "2016-06-20T00:06:00Z",
"end": "2016-06-21T23:59:00Z"
}
}
For supported fields and their values, see Autoscale REST API documentation. Now your autoscale setting
contains the three profiles explained previously.
7. Finally, look at the Autoscale notification section. Autoscale notifications allow you to do three things
when a scale-out or in action is successfully triggered.
Notify the admin and co-admins of your subscription
Email a set of users
Trigger a webhook call. When fired, this webhook sends metadata about the autoscaling condition and
the scale set resource. To learn more about the payload of autoscale webhook, see Configure Webhook
& Email Notifications for Autoscale.
Add the following to the Autoscale setting replacing your notification element whose value is null
"notifications": [
{
"operation": "Scale",
"email": {
"sendToSubscriptionAdministrator": true,
"sendToSubscriptionCoAdministrators": false,
"customEmails": [
"user1@mycompany.com",
"user2@mycompany.com"
]
},
"webhooks": [
{
"serviceUri": "https://foo.webhook.example.com?token=abcd1234",
"properties": {
"optional_key1": "optional_value1",
"optional_key2": "optional_value2"
}
}
]
}
]
Next Steps
Use these links to learn more about autoscaling.
TroubleShoot Autoscale with Virtual Machine Scale Sets
Common Metrics for Autoscale
Best Practices for Azure Autoscale
Manage Autoscale using PowerShell
Manage Autoscale using CLI
Configure Webhook & Email Notifications for Autoscale
Microsoft.Insights/autoscalesettings template reference
Use autoscale actions to send email and webhook
alert notifications in Azure Monitor
12/23/2019 • 3 minutes to read • Edit Online
This article shows you how set up triggers so that you can call specific web URLs or send emails based on
autoscale actions in Azure.
Webhooks
Webhooks allow you to route the Azure alert notifications to other systems for post-processing or custom
notifications. For example, routing the alert to services that can handle an incoming web request to send SMS, log
bugs, notify a team using chat or messaging services, etc. The webhook URI must be a valid HTTP or HTTPS
endpoint.
Email
Email can be sent to any valid email address. Administrators and co-administrators of the subscription where the
rule is running will also be notified.
Authentication in webhooks
The webhook can authenticate using token-based authentication, where you save the webhook URI with a token
ID as a query parameter. For example, https://mysamplealert/webcallback?
tokenid=sometokenid&someparameter=somevalue
Azure Monitor autoscale helps you to have the right amount of resources running to handle the load on your
application. It enables you to add resources to handle increases in load and also save money by removing
resources that are sitting idle. You can scale based on a schedule, fixed date-time, or resource metric you choose.
For more information, see Autoscale Overview.
The autoscale service provides you metrics and logs to understand what scale actions have occurred and the
evaluation of the conditions that led to those actions. You can find answers to questions such as:
Why did my service scale-out or in?
Why did my service not scale?
Why did an autoscale action fail?
Why is an autoscale action taking time to scale?
Autoscale metrics
Autoscale provides you with four metrics to understand its operation.
Observed Metric Value - The value of the metric you chose to take the scale action on, as seen or computed
by the autoscale engine. Because a single autoscale setting can have multiple rules and therefore multiple
metric sources, you can filter using "metric source" as a dimension.
Metric Threshold - The threshold you set to take the scale action. Because a single autoscale setting can have
multiple rules and therefore multiple metric sources, you can filter using "metric rule" as a dimension.
Observed Capacity - The active number of instances of the target resource as seen by Autoscale engine.
Scale Actions Initiated - The number of scale-out and scale-in actions initiated by the autoscale engine. You
can filter by scale-out vs. scale in actions.
You can use the Metrics Explorer to chart the above metrics all in one place. The chart should show:
the actual metric
the metric as seen/computed by autoscale engine
the threshold for a scale action
the change in capacity
NOTE
You will need to filter the Metric Threshold by the metric trigger rule dimension scale out (increase) rule to see the scale-out
threshold and by the scale in rule (decrease).
Example 2 - Advanced autoscaling for a virtual machine scale set
We have an autoscale setting that allows a virtual machine scale set resource to scale out based on its own metric
Outbound Flows. Notice that the divide metric by instance count option for the metric threshold is checked.
The scale action rule is:
If the value of Outbound Flow per instance is greater than 10, then autoscale service should scale out by 1
instance.
In this case, the autoscale engine’s observed metric value is calculated as the actual metric value divided by the
number of instances. If the observed metric value is less than the threshold, no scale-out action is initiated.
AutoscaleEvaluationsLog
| limit 50
Or try the following query to view the most recent scale action logs:
AutoscaleScaleActionsLog
| limit 50
Use the following sections to these questions.
AutoscaleScaleActionsLog
| take 1
Select the CorrelationId field from the scale actions log. Use the CorrelationId to find the right Evaluation log.
Executing the below query will display all the rules and conditions evaluated leading to that scale action.
AutoscaleEvaluationsLog
| where CorrelationId = "<correliationId>"
AutoscaleEvaluationsLog
| where CorrelationId = "<correliationId_Guid>"
| where ProfileSelected == true
| project ProfileEvaluationTime, Profile, ProfileSelected, EvaluationResult
The whole profile evaluation can also be understood better using the following query
AutoscaleEvaluationsLog
| where TimeGenerated > ago(2h)
| where OperationName contains == "profileEvaluation"
| project OperationName, Profile, ProfileEvaluationTime, ProfileSelected, EvaluationResult
AutoscaleEvaluationsLog
| where TimeGenerated > ago(2h)
| where OperationName == "MetricEvaluation" or OperationName == "ScaleRuleEvaluation"
| project OperationName, MetricData, ObservedValue, Threshold, EstimateScaleResult
Scale action failed
There may be a case where autoscale service took the scale action but the system decided not to scale or failed to
complete the scale action. Use this query to find the failed scale actions.
AutoscaleScaleActionsLog
| where ResultType == "Failed"
| project ResultDescription
Create alert rules to get notified of autoscale actions or failures. You can also create alert rules to get notified on
autoscale events.
Next steps
Read information on autoscale best practices.
Azure Monitor autoscale actions resource log schema
12/23/2019 • 2 minutes to read • Edit Online
Following are the general formats for autoscale resource logs with example data included. Not all examples below
are properly formed JSON because they may include multiple values that could be valid for a given field.
Use events of this type to troubleshoot problems you may be having with autoscale. For more information, see
Troubleshooting autoscale problems.
Profile evaluation
Recorded when autoscale first looks at an autoscale profile
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": ["FixedDateProfileEvaluation", "RecurrentProfileEvaluation", "DefaultProfileEvaluation"],
"category": "AutoscaleEvaluations",
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"profile": "defaultProfile",
"profileSelected": [true, false]
}
}
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": "ScaleRuleCooldownEvaluation",
"category": "AutoscaleEvaluations",
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"selectedProfile": "defaultProfile",
"scaleDirection": ["Increase", "Decrease"]
"lastScaleActionTime": "2018-09-10 18:08:00.6132593",
"cooldown": "00:30:00",
"evaluationTime": "2018-09-10 18:11:00.6132593",
"skipRuleEvaluationForCooldown": true
}
}
Rule evaluation
Recorded when autoscale first starts evaluating a particular scale rule.
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": "ScaleRuleEvaluation",
"category": "AutoscaleEvaluations",
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"metricName": "Percentage CPU",
"metricNamespace": "",
"timeGrain": "00:01:00",
"timeGrainStatistic": "Average",
"timeWindow": "00:10:00",
"timeAggregationType": "Average",
"operator": "GreaterThan",
"threshold": 70,
"observedValue": 25,
"estimateScaleResult": ["Triggered", "NotTriggered", "Unknown"]
}
}
Metric evaluation
Recorded when autoscale evaluated the metric being used to trigger a scale action.
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": "MetricEvaluation",
"category": "AutoscaleEvaluations",
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"metricName": "Percentage CPU",
"metricNamespace": "",
"timeGrain": "00:01:00",
"timeGrainStatistic": "Average",
"startTime": "2018-09-10 18:00:00.43833793",
"endTime": "2018-09-10 18:10:00.43833793",
"data": [0.33333333333333331,0.16666666666666666,1.0,0.33333333333333331,2.0,0.16666666666666666,9.5]
}
}
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": "ScaleActionOperationEvaluation",
"category": "AutoscaleEvaluations",
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"lastScaleActionOperationId": "378ejr-7yye-892d-17dd-92ndijfe1738",
"lastScaleActionOperationStatus": ["InProgress", "Timeout"]
"skipCurrentAutoscaleEvaluation": [true, false]
}
}
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": "InstanceUpdateEvaluation",
"category": "AutoscaleEvaluations",
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"currentInstanceCount": 20,
"newInstanceCount": 21,
"shouldUpdateInstance": [true, false],
"reason": ["Scale down action triggered", "Scale up to default instance count", ...]
}
}
Scale action
Recorded when autoscale initiates a scale action, either up or down.
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": "InstanceScaleAction",
"category": "AutoscaleScaleActions",
"resultType": ["Succeeded", "InProgress", "Failed"],
"resultDescription": ["Create async operation job failed", ...]
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"currentInstanceCount": 20,
"newInstanceCount": 21,
"scaleDirection": ["Increase", "Decrease"],
["createdAsyncScaleActionJob": [true, false],]
["createdAsyncScaleActionJobId": "378ejr-7yye-892d-17dd-92ndijfe1738",]
}
}
{
"time": "2018-09-10 18:12:00.6132593",
"resourceId": "/SUBSCRIPTIONS/BA13C41D-C957-4774-8A37-
092D62ACFC85/RESOURCEGROUPS/AUTOSCALETRACKING12042017/PROVIDERS/MICROSOFT.INSIGHTS/AUTOSCALESETTINGS/DEFAULTSE
TTING",
"operationName": "InstanceScaleAction",
"category": "AutoscaleScaleActions",
"correlationId": "e8f67045-f381-445d-bc2d-eeff81ec0d77",
"property": {
"targetResourceId": "/subscriptions/d45c994a-809b-4cb3-a952-
e75f8c488d23/resourceGroups/RingAhoy/providers/Microsoft.Web/serverfarms/ringahoy",
"scaleActionOperationId": "378ejr-7yye-892d-17dd-92ndijfe1738",
"scaleActionOperationStatus": ["InProgress", "Timeout", "Canceled", ...],
"scaleActionMessage": ["Scale action is inprogress", ...]
}
}
Next steps
Learn about autoscale
Azure Monitor PowerShell quick start samples
10/17/2019 • 9 minutes to read • Edit Online
This article shows you sample PowerShell commands to help you access Azure Monitor features.
NOTE
Azure Monitor is the new name for what was called "Azure Insights" until Sept 25th, 2016. However, the namespaces and
thus the following commands still contain the word "insights."
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
Set up PowerShell
If you haven't already, set up PowerShell to run on your computer. For more information, seeHow to Install and
Configure PowerShell.
Connect-AzAccount
You'll see a sign in screen. Once you sign in your Account, TenantID, and default Subscription ID are displayed. All
the Azure cmdlets work in the context of your default subscription. To view the list of subscriptions you have
access to, use the following command:
Get-AzSubscription
To see your working context (which subscription your commands are run against), use the following command:
Get-AzContext
To change your working context to a different subscription, use the following command:
Get-Date
Get log entries from a specific resource provider between a time/date range:
The following command retrieves the last 1000 events from the activity log:
Get-AzLog -MaxRecord 10
Get-AzLog supports many other parameters. See the Get-AzLog reference for more information.
NOTE
Get-AzLog only provides 15 days of history. Using the -MaxRecords parameter allows you to query the last N events,
beyond 15 days. To access events older than 15 days, use the REST API or SDK (C# sample using the SDK). If you do not
include StartTime, then the default value is EndTime minus one hour. If you do not include EndTime, then the default value
is current time. All times are in UTC.
To view the history for a specific alert rule, you can use the Get-AzAlertHistory cmdlet, passing in the resource ID
of the alert rule.
Get-AzAlertHistory -ResourceId
/subscriptions/s1/resourceGroups/rg1/providers/microsoft.insights/alertrules/myalert -StartTime 2016-03-1 -
Status Activated
The Get-AzAlertHistory cmdlet supports various parameters. More information, see Get-AlertHistory.
Retrieve all alert rules set for a target resource. For example, all alert rules set on a VM.
PARAMETER VALUE
Name simpletestdiskwrite
ResourceGroup montest
TargetResourceId /subscriptions/s1/resourceGroups/montest/providers/Microso
ft.Compute/virtualMachines/testconfig
operator GreaterThan
PARAMETER VALUE
The Add alert cmdlet also updates the rule if an alert rule already exists for the given properties. To disable an alert
rule, include the parameter -DisableRule.
The following example generates a table with the metric Name and the Unit for it.
The additional webhook properties are optional. You can get back the contents of an Activity Log Alert using
Get-AzActivityLogAlert .
Create the notification property for the autoscale setting, including email and the webhook that you created
previously.
Finally, create the autoscale setting to add the profile that you created previously.
Autoscale history
The following example shows you how you can view recent autoscale and alert events. Use the activity log search
to view the autoscale history.
Get-AzAutoScaleHistory -ResourceId
/subscriptions/s1/resourceGroups/myrg1/providers/microsoft.insights/autoscalesettings/myScaleSetting -
StartTime 2016-03-15 -DetailedOutput
The following example shows details about all autoscale settings in the resource group 'myrg1' and specifically the
autoscale setting named 'MyScaleVMSSSetting'.
Get-AzDiagnosticSetting -ResourceId
/subscriptions/s1/resourceGroups/myrg1/providers/Microsoft.Logic/workflows/andy0315logicapp
Set-AzDiagnosticSetting -ResourceId
/subscriptions/s1/resourceGroups/myrg1/providers/Microsoft.Logic/workflows/andy0315logicapp -StorageAccountId
/subscriptions/s1/resourceGroups/Default-Storage-
WestUS/providers/Microsoft.Storage/storageAccounts/mystorageaccount -Enable $false
Set-AzDiagnosticSetting -ResourceId
/subscriptions/s1/resourceGroups/myrg1/providers/Microsoft.Logic/workflows/andy0315logicapp -StorageAccountId
/subscriptions/s1/resourceGroups/Default-Storage-
WestUS/providers/Microsoft.Storage/storageAccounts/mystorageaccount -Enable $true
Set-AzDiagnosticSetting -ResourceId
/subscriptions/s1/resourceGroups/myrg1/providers/Microsoft.Logic/workflows/andy0315logicapp -StorageAccountId
/subscriptions/s1/resourceGroups/Default-Storage-
WestUS/providers/Microsoft.Storage/storageAccounts/mystorageaccount -Enable $true -RetentionEnabled $true -
RetentionInDays 90
Note that the WorkspaceId property takes the resource ID of the workspace. You can obtain the resource ID of
your Log Analytics workspace using the following command:
(Get-AzOperationalInsightsWorkspace).ResourceId
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install Azure
PowerShell.
NOTE
If you want to create resources and alerts at the same time, consider using an Azure Resource Manager template.
One-time setup
If you haven't used PowerShell with your Azure subscription before:
Install the Azure Powershell module on the machine where you want to run the scripts.
Install Microsoft Web Platform Installer (v5 or higher).
Use it to install Microsoft Azure Powershell
Connect to Azure
Start Azure PowerShell and connect to your subscription:
Add-AzAccount
Get alerts
Get-AzAlertRule -ResourceGroup "Fabrikam" [-Name "My rule"] [-DetailedOutput]
Add alert
Add-AzMetricAlertRule -Name "{ALERT NAME}" -Description "{TEXT}" `
-ResourceGroup "{GROUP NAME}" `
-ResourceId "/subscriptions/{SUBSCRIPTION ID}/resourcegroups/{GROUP
NAME}/providers/microsoft.insights/components/{APP RESOURCE NAME}" `
-MetricName "{METRIC NAME}" `
-Operator GreaterThan `
-Threshold {NUMBER} `
-WindowSize {HH:MM:SS} `
[-SendEmailToServiceOwners] `
[-CustomEmails "EMAIL1@X.COM","EMAIL2@Y.COM" ] `
-Location "East US" // must be East US at present
-RuleType Metric
Example 1
Email me if the server's response to HTTP requests, averaged over 5 minutes, is slower than 1 second. My
Application Insights resource is called IceCreamWebApp, and it is in resource group Fabrikam. I am the owner of
the Azure subscription.
The GUID is the subscription ID (not the instrumentation key of the application).
Example 2
I have an application in which I use TrackMetric() to report a metric named "salesPerHour." Send an email to my
colleagues if "salesPerHour" drops below 100, averaged over 24 hours.
The same rule can be used for the metric reported by using the measurement parameter of another tracking call
such as TrackEvent or trackPageView.
Metric names
METRIC NAME SCREEN NAME DESCRIPTION
clientPerformance.clientProcess.value Client processing time Time between receiving the last byte of
a document until the DOM is loaded.
Async requests may still be processing.
clientPerformance.total.value Browser page load time Time from user request until DOM,
stylesheets, scripts and images are
loaded.
Available memory
performanceCounter.available_bytes.value Physical memory immediately available
for a process or for system use.
Process IO
performanceCounter.io_data_bytes_per_sec.value Rate Total bytes per second read and written
to files, network and devices.
exception rate
performanceCounter.number_of_exceps_thrown_per_sec.value Exceptions thrown per second.
Process CPU
performanceCounter.percentage_processor_time.value The percentage of elapsed time of all
process threads used by the processor
to execution instructions for the
applications process.
Processor time
performanceCounter.percentage_processor_total.value The percentage of time that the
processor spends in non-Idle threads.
Dependency failures
remoteDependencyFailed.durationMetric.count Count of failed calls made by the server
application to external resources.
METRIC NAME SCREEN NAME DESCRIPTION
{your custom metric name} {Your metric name} Your metric value reported by
TrackMetric or in the measurements
parameter of a tracking call.
performanceCounter Performance
remoteDependencyFailed Dependency
Webhooks
You can automate your response to an alert. Azure will call a web address of your choice when an alert is raised.
See also
Script to configure Application Insights
Create Application Insights and web test resources from templates
Automate coupling Microsoft Azure Diagnostics to Application Insights
Automate your response to an alert
Manage Log Analytics workspace in Azure Monitor using
PowerShell
12/19/2019 • 7 minutes to read • Edit Online
You can use the Log Analytics PowerShell cmdlets to perform various functions on a Log Analytics workspace in Azure Monitor from a
command line or as part of a script. Examples of the tasks you can perform with PowerShell include:
Create a workspace
Add or remove a solution
Import and export saved searches
Create a computer group
Enable collection of IIS logs from computers with the Windows agent installed
Collect performance counters from Linux and Windows computers
Collect events from syslog on Linux computers
Collect events from Windows event logs
Collect custom event logs
Add the log analytics agent to an Azure virtual machine
Configure log analytics to index data collected using Azure diagnostics
This article provides two code samples that illustrate some of the functions that you can perform from PowerShell. You can refer to the
Log Analytics PowerShell cmdlet reference for other functions.
NOTE
Log Analytics was previously called Operational Insights, which is why it is the name used in the cmdlets.
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will continue to receive bug
fixes until at least December 2020. To learn more about the new Az module and AzureRM compatibility, see Introducing the new Azure PowerShell Az
module. For Az module installation instructions, see Install Azure PowerShell.
Prerequisites
These examples work with version 1.0.0 or later of the Az.OperationalInsights module.
$ResourceGroup = "oms-example"
$WorkspaceName = "log-analytics-" + (Get-Random -Maximum 99999) # workspace names need to be unique across all Azure subscriptions -
$WorkspaceName = "log-analytics-" + (Get-Random -Maximum 99999) # workspace names need to be unique across all Azure subscriptions -
Get-Random helps with this for the example code
$Location = "westeurope"
# Add solutions
foreach ($solution in $Solutions) {
Set-AzOperationalInsightsIntelligencePack -ResourceGroupName $ResourceGroup -WorkspaceName $WorkspaceName -IntelligencePackName
$solution -Enabled $true
}
# Linux Perf
New-AzOperationalInsightsLinuxPerformanceObjectDataSource -ResourceGroupName $ResourceGroup -WorkspaceName $WorkspaceName -ObjectName
"Logical Disk" -InstanceName "*" -CounterNames @("% Used Inodes", "Free Megabytes", "% Used Space", "Disk Transfers/sec", "Disk
Reads/sec", "Disk Reads/sec", "Disk Writes/sec") -IntervalSeconds 20 -Name "Example Linux Disk Performance Counters"
Enable-AzOperationalInsightsLinuxPerformanceCollection -ResourceGroupName $ResourceGroup -WorkspaceName $WorkspaceName
# Linux Syslog
New-AzOperationalInsightsLinuxSyslogDataSource -ResourceGroupName $ResourceGroup -WorkspaceName $WorkspaceName -Facility "kern" -
CollectEmergency -CollectAlert -CollectCritical -CollectError -CollectWarning -Name "Example kernel syslog collection"
Enable-AzOperationalInsightsLinuxSyslogCollection -ResourceGroupName $ResourceGroup -WorkspaceName $WorkspaceName
# Windows Event
New-AzOperationalInsightsWindowsEventDataSource -ResourceGroupName $ResourceGroup -WorkspaceName $WorkspaceName -EventLogName
"Application" -CollectErrors -CollectWarnings -Name "Example Application Event Log"
# Windows Perf
New-AzOperationalInsightsWindowsPerformanceCounterDataSource -ResourceGroupName $ResourceGroup -WorkspaceName $WorkspaceName -
ObjectName "Memory" -InstanceName "*" -CounterName "Available MBytes" -IntervalSeconds 20 -Name "Example Windows Performance Counter"
# Custom Logs
NOTE
The format for the CustomLogRawJson parameter which defines the configuration for a custom log can be complex. Use Get-
AzOperationalInsightsDataSource to retrieve the configuration for an existing Custom Log. The Properties property is the configuration required for
the CustomLogRawJson parameter.
In the above example regexDelimiter was defined as "\n" for newline. The log delimiter may also be a timestamp. These are the supported
formats:
yyyy-MM-ddTHH:mm:ss ((\\d{2})|(\\d{4}))-([0-
The T is a literal letter T 1]\\d)-(([0-3]\\d)|
(\\d))T((\\d)|([0-1]\\d)|
(2[0-4])):[0-5][0-9]:[0-5][0-
9]
For the details of the available metrics, refer to supported metrics with Azure Monitor.
For the details of the available logs, refer to supported services and schema for resource logs.
$workspaceId = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx/resourcegroups/oi-default-east-
us/providers/microsoft.operationalinsights/workspaces/rollingbaskets"
$resourceId = "/SUBSCRIPTIONS/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxx/RESOURCEGROUPS/DEMO/PROVIDERS/MICROSOFT.NETWORK/NETWORKSECURITYGROUPS/DEMO"
You can also use the preceding cmdlet to collect logs from resources that are in different subscriptions. The cmdlet is able to work across
subscriptions since you're providing the ID of both the resource creating logs and the workspace the logs are sent to.
# Update these two lines with the storage account resource ID and the storage account key for the storage account you want the
workspace to index
$storageId = "/subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-
xxxxxxxxx/resourceGroups/demo/providers/Microsoft.Storage/storageAccounts/wadv2storage"
$key = "abcd=="
You can also use the preceding script to collect logs from storage accounts in different subscriptions. The script is able to work across
subscriptions since you are providing the storage account resource ID and a corresponding access key. When you change the access key,
you need to update the storage insight to have the new key.
Next steps
Review Log Analytics PowerShell cmdlets for additional information on using PowerShell for configuration of Log Analytics.
Manage Application Insights resources using
PowerShell
1/14/2020 • 8 minutes to read • Edit Online
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which
will continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install
Azure PowerShell.
This article shows you how to automate the creation and update of Application Insights resources automatically
by using Azure Resource Management. You might, for example, do so as part of a build process. Along with the
basic Application Insights resource, you can create availability web tests, set up alerts, set the pricing scheme,
and create other Azure resources.
The key to creating these resources is JSON templates for Azure Resource Manager. The basic procedure is:
download the JSON definitions of existing resources; parameterize certain values such as names; and then run
the template whenever you want to create a new resource. You can package several resources together, to create
them all in one go - for example, an app monitor with availability tests, alerts, and storage for continuous export.
There are some subtleties to some of the parameterizations, which we'll explain here.
One-time setup
If you haven't used PowerShell with your Azure subscription before:
Install the Azure Powershell module on the machine where you want to run the scripts:
1. Install Microsoft Web Platform Installer (v5 or higher).
2. Use it to install Microsoft Azure Powershell.
In addition to using Resource Manager templates, there is a rich set of Application Insights PowerShell cmdlets,
which make it easy to configure Application Insights resources programatically. The capabilities enabled by the
cmdlets include:
Create and delete Application Insights resources
Get lists of Application Insights resources and their properties
Create and manage Continuous Export
Create and manage Application Keys
Set the Daily Cap
Set the Pricing Plan
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"appName": {
"type": "string",
"metadata": {
"description": "Enter the name of your Application Insights resource."
}
},
"appType": {
"type": "string",
"defaultValue": "web",
"allowedValues": [
"web",
"java",
"other"
],
"metadata": {
"description": "Enter the type of the monitored application."
}
},
"appLocation": {
"type": "string",
"defaultValue": "eastus",
"metadata": {
"description": "Enter the location of your Application Insights resource."
}
},
"retentionInDays": {
"type": "int",
"defaultValue": 90,
"allowedValues": [
30,
60,
90,
120,
180,
270,
365,
550,
730
],
"metadata": {
"description": "Data retention in days"
}
},
"ImmediatePurgeDataOn30Days": {
"type": "bool",
"defaultValue": false,
"metadata": {
"description": "If set to true when changing retention to 30 days, older data will be
immediately deleted. Use this with extreme caution. This only applies when retention is being set to 30
days."
}
},
"priceCode": {
"type": "int",
"defaultValue": 1,
"allowedValues": [
1,
2
],
"metadata": {
"description": "Pricing plan: 1 = Per GB (or legacy Basic plan), 2 = Per Node (legacy
Enterprise plan)"
}
},
"dailyQuota": {
"type": "int",
"defaultValue": 100,
"minValue": 1,
"metadata": {
"description": "Enter daily quota in GB."
}
},
"dailyQuotaResetTime": {
"type": "int",
"defaultValue": 24,
"metadata": {
"description": "Enter daily quota reset hour in UTC (0 to 23). Values outside the range
will get a random reset hour."
}
},
"warningThreshold": {
"type": "int",
"defaultValue": 90,
"minValue": 1,
"maxValue": 100,
"metadata": {
"description": "Enter the % value of daily quota after which warning mail to be sent. "
}
}
},
"variables": {
"priceArray": [
"Basic",
"Application Insights Enterprise"
],
"pricePlan": "[take(variables('priceArray'),parameters('priceCode'))]",
"billingplan": "[concat(parameters('appName'),'/', variables('pricePlan')[0])]"
},
"resources": [
{
"type": "microsoft.insights/components",
"kind": "[parameters('appType')]",
"name": "[parameters('appName')]",
"apiVersion": "2014-04-01",
"location": "[parameters('appLocation')]",
"tags": {},
"properties": {
"ApplicationId": "[parameters('appName')]",
"retentionInDays": "[parameters('retentionInDays')]"
},
"dependsOn": []
},
{
"name": "[variables('billingplan')]",
"type": "microsoft.insights/components/CurrentBillingFeatures",
"location": "[parameters('appLocation')]",
"apiVersion": "2015-05-01",
"dependsOn": [
"[resourceId('microsoft.insights/components', parameters('appName'))]"
],
"properties": {
"CurrentBillingFeatures": "[variables('pricePlan')]",
"DataVolumeCap": {
"DataVolumeCap": {
"Cap": "[parameters('dailyQuota')]",
"WarningThreshold": "[parameters('warningThreshold')]",
"ResetTime": "[parameters('dailyQuotaResetTime')]"
}
}
}
]
}
Use the Resource Manager template to create a new Application Insights resource
1. In PowerShell, sign in to Azure using $Connect-AzAccount
2. Set your context to a subscription with Set-AzContext "<subscription ID>"
-ResourceGroupNameis the group where you want to create the new resources.
-TemplateFile must occur before the custom parameters.
-appName The name of the resource to create.
You can add other parameters - you'll find their descriptions in the parameters section of the template.
To see a list of many other properties of your Application Insights resource, use:
Refer to the detailed documentation for the parameters for these cmdlets.
To set the data retention to 365 days using the template above, run:
The following script can also be used to change retention. Copy this script to save as
Set-ApplicationInsightsRetention.ps1 .
Param(
[Parameter(Mandatory = $True)]
[string]$SubscriptionId,
[Parameter(Mandatory = $True)]
[string]$ResourceGroupName,
[Parameter(Mandatory = $True)]
[string]$Name,
[Parameter(Mandatory = $True)]
[string]$RetentionInDays
)
$ErrorActionPreference = 'Stop'
if (-not (Get-Module Az.Accounts)) {
Import-Module Az.Accounts
}
$azProfile =
[Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
if (-not $azProfile.Accounts.Count) {
Write-Error "Ensure you have logged in before calling this function."
}
$currentAzureContext = Get-AzContext
$profileClient = New-Object Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient($azProfile)
$token = $profileClient.AcquireAccessToken($currentAzureContext.Tenant.TenantId)
$UserToken = $token.AccessToken
$RequestUri =
"https://management.azure.com/subscriptions/$($SubscriptionId)/resourceGroups/$($ResourceGroupName)/provider
s/Microsoft.Insights/components/$($Name)?api-version=2015-05-01"
$Headers = @{
"Authorization" = "Bearer $UserToken"
"x-ms-client-tenant-id" = $currentAzureContext.Tenant.TenantId
}
## Get Component object via ARM
$GetResponse = Invoke-RestMethod -Method "GET" -Uri $RequestUri -Headers $Headers
Set-ApplicationInsightsRetention `
[-SubscriptionId] <String> `
[-ResourceGroupName] <String> `
[-Name] <String> `
[-RetentionInDays <Int>]
To set the daily cap properties, use same cmdlet. For instance, to set the cap to 300 GB/day,
To set the pricing plan, use same cmdlet with the -PricingPlan specified:
You can also set the pricing plan on an existing Application Insights resource using the Resource Manager
template above, omitting the "microsoft.insights/components" resource and the dependsOn node from the
billing resource. For instance, to set it to the Per GB plan (formerly called the Basic plan), run:
PRICECODE PLAN
"hidden- "[concat('hidden-link:',
link:/subscriptions/.../../components/MyAppName" resourceId('microsoft.insights/components',
parameters('appName')))]"
"/subscriptions/.../../alertrules/myAlertName- "[resourceId('Microsoft.Insights/alertrules',
myAppName-subsId", variables('alertRuleName'))]",
"/subscriptions/.../../webtests/myTestName- "[resourceId('Microsoft.Insights/webtests',
myAppName", parameters('webTestName'))]",
"myWebTest-myAppName" "[variables(testName)]"'
"myTestName-myAppName-subsId" "[variables('alertRuleName')]"
"myAppName" "[parameters('appName')]"
Next steps
Other automation articles:
Create an Application Insights resource - quick method without using a template.
Set up Alerts
Create web tests
Send Azure Diagnostics to Application Insights
Deploy to Azure from GitHub
Create release annotations
Using PowerShell to set up Application Insights for
Azure Cloud Services
10/23/2019 • 2 minutes to read • Edit Online
Microsoft Azure can be configured to send Azure Diagnostics to Azure Application Insights. The diagnostics relate
to Azure Cloud Services and Azure VMs. They complement the telemetry that you send from within the app using
the Application Insights SDK. As part of automating the process of creating new resources in Azure, you can
configure diagnostics using PowerShell.
Azure template
If the web app is in Azure and you create your resources using an Azure Resource Manager template, you can
configure Application Insights by adding this to the resources node:
{
resources: [
/* Create Application Insights resource */
{
"apiVersion": "2015-05-01",
"type": "microsoft.insights/components",
"name": "nameOfAIAppResource",
"location": "centralus",
"kind": "web",
"properties": { "ApplicationId": "nameOfAIAppResource" },
"dependsOn": [
"[concat('Microsoft.Web/sites/', myWebAppName)]"
]
}
]
}
$primary_storagekey = (Get-AzStorageKey `
-StorageAccountName "$diagnostics_storagename").Primary
$storage_context = New-AzStorageContext `
-StorageAccountName $diagnostics_storagename `
-StorageAccountKey $primary_storagekey
$webrole_diagconfig = `
New-AzureServiceDiagnosticsExtensionConfig `
-Role "WebRole" -Storage_context $storageContext `
-DiagnosticsConfigurationPath $webrole_diagconfigpath
$workerrole_diagconfig = `
New-AzureServiceDiagnosticsExtensionConfig `
-Role "WorkerRole" `
-StorageContext $storage_context `
-DiagnosticsConfigurationPath $workerrole_diagconfigpath
New-AzureDeployment `
-ServiceName $service_name `
-Slot Production `
-Package $service_package `
-Configuration $service_config `
-ExtensionConfiguration @($webrole_diagconfig,$workerrole_diagconfig)
$service_name = "MyService"
$diagnostics_storagename = "myservicediagnostics"
$webrole_diagconfigpath = "MyService.WebRole.PubConfig.xml"
$workerrole_diagconfigpath = "MyService.WorkerRole.PubConfig.xml"
$primary_storagekey = (Get-AzStorageKey `
-StorageAccountName "$diagnostics_storagename").Primary
$storage_context = New-AzStorageContext `
-StorageAccountName $diagnostics_storagename `
-StorageAccountKey $primary_storagekey
Set-AzureServiceDiagnosticsExtension `
-StorageContext $storage_context `
-DiagnosticsConfigurationPath $webrole_diagconfigpath `
-ServiceName $service_name `
-Slot Production `
-Role "WebRole"
Set-AzureServiceDiagnosticsExtension `
-StorageContext $storage_context `
-DiagnosticsConfigurationPath $workerrole_diagconfigpath `
-ServiceName $service_name `
-Slot Production `
-Role "WorkerRole"
See also
Monitor Azure Cloud Services apps with Application Insights
Send Azure Diagnostics to Application Insights
Automate configuring alerts
Azure Monitor CLI quick start samples
12/6/2019 • 3 minutes to read • Edit Online
This article shows you sample command-line interface (CLI) commands to help you access Azure Monitor
features. Azure Monitor allows you to AutoScale Cloud Services, Virtual Machines, and Web Apps and to send
alert notifications or call web URLs based on values of configured telemetry data.
Prerequisites
If you haven't already installed the Azure CLI, follow the instructions for Install the Azure CLI. You can also use
Azure Cloud Shell to run the CLI as an interactive experience in your browser. See a full reference of all available
commands in the Azure Monitor CLI reference.
Log in to Azure
The first step is to login to your Azure account.
az login
After running this command, you have to sign in via the instructions on the screen. All commands work in the
context of your default subscription.
To list the details of your current subscription, use the following command.
az account show
To view a list of all supported Azure Monitor commands, perform the following.
az monitor -h
Log profiles
Use the information in this section to work with log profiles.
Get a log profile
Diagnostics
Use the information in this section to work with diagnostic settings.
Get a diagnostic setting
Autoscale
Use the information in this section to work with autoscale settings. You need to modify these examples.
Get autoscale settings for a resource group
az monitor autoscale list --resource-group <group name>
NOTE
This article has been updated to use the new Azure PowerShell Az module. You can still use the AzureRM module, which will
continue to receive bug fixes until at least December 2020. To learn more about the new Az module and AzureRM
compatibility, see Introducing the new Azure PowerShell Az module. For Az module installation instructions, see Install
Azure PowerShell.
You can use Azure Resource Manager templates to create and configure Log Analytics workspaces in Azure
Monitor. Examples of the tasks you can perform with templates include:
Create a workspace including setting pricing tier and capacity reservation
Add a solution
Create saved searches
Create a computer group
Enable collection of IIS logs from computers with the Windows agent installed
Collect performance counters from Linux and Windows computers
Collect events from syslog on Linux computers
Collect events from Windows event logs
Collect custom logs from Windows computer
Add the log analytics agent to an Azure virtual machine
Configure log analytics to index data collected using Azure diagnostics
This article provides template samples that illustrate some of the configuration that you can perform with
templates.
API versions
The following table lists the API version for the resources used in this example.
{
"$schema": "https://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "String",
"metadata": {
"description": "Specifies the name of the workspace."
}
},
"pricingTier": {
"type": "string",
"allowedValues": [
"pergb2018",
"Free",
"Standalone",
"PerNode",
"Standard",
"Premium"
],
"defaultValue": "pergb2018",
"metadata": {
"description": "Pricing tier: PerGB2018 or legacy tiers (Free, Standalone, PerNode, Standard or
Premium) which are not available to all customers."
}
},
"location": {
"type": "String",
"allowedValues": [
"australiacentral",
"australiaeast",
"australiasoutheast",
"brazilsouth",
"canadacentral",
"centralindia",
"centralus",
"eastasia",
"eastus",
"eastus2",
"francecentral",
"japaneast",
"koreacentral",
"northcentralus",
"northeurope",
"northeurope",
"southafricanorth",
"southcentralus",
"southeastasia",
"uksouth",
"ukwest",
"westcentralus",
"westeurope",
"westus",
"westus2"
],
"metadata": {
"description": "Specifies the location in which to create the workspace."
}
}
},
"resources": [
{
"type": "Microsoft.OperationalInsights/workspaces",
"name": "[parameters('workspaceName')]",
"apiVersion": "2017-03-15-preview",
"location": "[parameters('location')]",
"properties": {
"sku": {
"name": "CapacityReservation",
"capacityReservationLevel": 100
},
"retentionInDays": 120,
"features": {
"searchVersion": 1,
"legacy": 0,
"enableLogAccessUsingOnlyResourcePermissions": true
}
}
}
]
}
For command line, use the following commands from the folder containing the template:
The deployment can take a few minutes to complete. When it finishes, you see a message similar to the following
that includes the result:
Configure a Log Analytics workspace
The following template sample illustrates how to:
1. Add solutions to the workspace
2. Create saved searches
3. Create a computer group
4. Enable collection of IIS logs from computers with the Windows agent installed
5. Collect Logical Disk perf counters from Linux computers (% Used Inodes; Free Megabytes; % Used Space;
Disk Transfers/sec; Disk Reads/sec; Disk Writes/sec)
6. Collect syslog events from Linux computers
7. Collect Error and Warning events from the Application Event Log from Windows computers
8. Collect Memory Available Mbytes performance counter from Windows computers
9. Collect IIS logs and Windows Event logs written by Azure diagnostics to a storage account
10. Collect custom logs from Windows computer
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "string",
"metadata": {
"description": "Workspace name"
}
},
"pricingTier": {
"type": "string",
"allowedValues": [
"PerGB2018",
"Free",
"Standalone",
"PerNode",
"Standard",
"Premium"
],
"defaultValue": "pergb2018",
"metadata": {
"description": "Pricing tier: pergb2018 or legacy tiers (Free, Standalone, PerNode, Standard or
Premium) which are not available to all customers."
}
},
"dataRetention": {
"type": "int",
"defaultValue": 30,
"minValue": 7,
"maxValue": 730,
"metadata": {
"description": "Number of days of retention. Workspaces in the legacy Free pricing tier can only have
7 days."
}
},
"immediatePurgeDataOn30Days": {
"type": "bool",
"defaultValue": "[bool('false')]",
"metadata": {
"description": "If set to true when changing retention to 30 days, older data will be immediately
deleted. Use this with extreme caution. This only applies when retention is being set to 30 days."
}
},
"location": {
"type": "string",
"allowedValues": [
"australiacentral",
"australiaeast",
"australiasoutheast",
"brazilsouth",
"canadacentral",
"centralindia",
"centralus",
"eastasia",
"eastus",
"eastus2",
"francecentral",
"japaneast",
"koreacentral",
"northcentralus",
"northeurope",
"southafricanorth",
"southcentralus",
"southeastasia",
"uksouth",
"ukwest",
"westcentralus",
"westeurope",
"westus",
"westus2"
],
"metadata": {
"description": "Specifies the location in which to create the workspace."
}
},
"applicationDiagnosticsStorageAccountName": {
"type": "string",
"metadata": {
"description": "Name of the storage account with Azure diagnostics output"
}
},
"applicationDiagnosticsStorageAccountResourceGroup": {
"type": "string",
"metadata": {
"description": "The resource group name containing the storage account with Azure diagnostics
output"
}
},
"customLogName": {
"type": "string",
"metadata": {
"description": "The custom log name"
}
}
},
"variables": {
"Updates": {
"Name": "[Concat('Updates', '(', parameters('workspaceName'), ')')]",
"GalleryName": "Updates"
},
"AntiMalware": {
"Name": "[concat('AntiMalware', '(', parameters('workspaceName'), ')')]",
"GalleryName": "AntiMalware"
},
"SQLAssessment": {
"Name": "[Concat('SQLAssessment', '(', parameters('workspaceName'), ')')]",
"GalleryName": "SQLAssessment"
},
"diagnosticsStorageAccount": "
[resourceId(parameters('applicationDiagnosticsStorageAccountResourceGroup'),
'Microsoft.Storage/storageAccounts', parameters('applicationDiagnosticsStorageAccountName'))]"
},
"resources": [
{
"apiVersion": "2017-03-15-preview",
"type": "Microsoft.OperationalInsights/workspaces",
"name": "[parameters('workspaceName')]",
"location": "[parameters('location')]",
"properties": {
"retentionInDays": "[parameters('dataRetention')]",
"features": {
"immediatePurgeDataOn30Days": "[parameters('immediatePurgeDataOn30Days')]"
},
"sku": {
"name": "[parameters('pricingTier')]",
"name": "CapacityReservation",
"capacityReservationLevel": 100
}
},
"resources": [
{
"apiVersion": "2015-03-20",
"name": "VMSS Queries2",
"type": "savedSearches",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"properties": {
"Category": "VMSS",
"ETag": "*",
"DisplayName": "VMSS Instance Count",
"Query": "Event | where Source == \"ServiceFabricNodeBootstrapAgent\" | summarize AggregatedValue
= count() by Computer",
"Version": 1
}
},
{
"apiVersion": "2015-11-01-preview",
"type": "datasources",
"name": "sampleWindowsEvent1",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"kind": "WindowsEvent",
"properties": {
"eventLogName": "Application",
"eventTypes": [
{
"eventType": "Error"
},
{
"eventType": "Warning"
}
]
}
},
{
"apiVersion": "2015-11-01-preview",
"apiVersion": "2015-11-01-preview",
"type": "datasources",
"name": "sampleWindowsPerfCounter1",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"kind": "WindowsPerformanceCounter",
"properties": {
"objectName": "Memory",
"instanceName": "*",
"intervalSeconds": 10,
"counterName": "Available MBytes"
}
},
{
"apiVersion": "2015-11-01-preview",
"type": "datasources",
"name": "sampleIISLog1",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"kind": "IISLogs",
"properties": {
"state": "OnPremiseEnabled"
}
},
{
"apiVersion": "2015-11-01-preview",
"type": "datasources",
"name": "sampleSyslog1",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"kind": "LinuxSyslog",
"properties": {
"syslogName": "kern",
"syslogSeverities": [
{
"severity": "emerg"
},
{
"severity": "alert"
},
{
"severity": "crit"
},
{
"severity": "err"
},
{
"severity": "warning"
}
]
}
},
{
"apiVersion": "2015-11-01-preview",
"type": "datasources",
"name": "sampleSyslogCollection1",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"kind": "LinuxSyslogCollection",
"properties": {
"state": "Enabled"
}
},
{
"apiVersion": "2015-11-01-preview",
"type": "datasources",
"type": "datasources",
"name": "sampleLinuxPerf1",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"kind": "LinuxPerformanceObject",
"properties": {
"performanceCounters": [
{
"counterName": "% Used Inodes"
},
{
"counterName": "Free Megabytes"
},
{
"counterName": "% Used Space"
},
{
"counterName": "Disk Transfers/sec"
},
{
"counterName": "Disk Reads/sec"
},
{
"counterName": "Disk Writes/sec"
}
],
"objectName": "Logical Disk",
"instanceName": "*",
"intervalSeconds": 10
}
},
{
"apiVersion": "2015-11-01-preview",
"type": "dataSources",
"name": "[concat(parameters('workspaceName'), parameters('customLogName'))]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', '/', parameters('workspaceName'))]"
],
"kind": "CustomLog",
"properties": {
"customLogName": "[parameters('customLogName')]",
"description": "this is a description",
"extractions": [
{
"extractionName": "TimeGenerated",
"extractionProperties": {
"dateTimeExtraction": {
"regex": [
{
"matchIndex": 0,
"numberdGroup": null,
"pattern": "((\\d{2})|(\\d{4}))-([0-1]\\d)-(([0-3]\\d)|(\\d))\\s((\\d)|([0-1]\\d)|
(2[0-4])):[0-5][0-9]:[0-5][0-9]"
}
]
}
},
"extractionType": "DateTime"
}
],
"inputs": [
{
"location": {
"fileSystemLocations": {
"linuxFileTypeLogPaths": null,
"windowsFileTypeLogPaths": [
"[concat('c:\\Windows\\Logs\\',parameters('customLogName'))]"
]
}
}
},
"recordDelimiter": {
"regexDelimiter": {
"matchIndex": 0,
"numberdGroup": null,
"pattern": "(^.*((\\d{2})|(\\d{4}))-([0-1]\\d)-(([0-3]\\d)|(\\d))\\s((\\d)|([0-1]\\d)|
(2[0-4])):[0-5][0-9]:[0-5][0-9].*$)"
}
}
}
]
}
},
{
"apiVersion": "2015-11-01-preview",
"type": "datasources",
"name": "sampleLinuxPerfCollection1",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"kind": "LinuxPerformanceCollection",
"properties": {
"state": "Enabled"
}
},
{
"apiVersion": "2015-03-20",
"name": "
[concat(parameters('applicationDiagnosticsStorageAccountName'),parameters('workspaceName'))]",
"type": "storageinsightconfigs",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"properties": {
"containers": [
"wad-iis-logfiles"
],
"tables": [
"WADWindowsEventLogsTable"
],
"storageAccount": {
"id": "[variables('diagnosticsStorageAccount')]",
"key": "[listKeys(variables('diagnosticsStorageAccount'),'2015-06-15').key1]"
}
}
},
{
"apiVersion": "2015-11-01-preview",
"location": "[parameters('location')]",
"name": "[variables('Updates').Name]",
"type": "Microsoft.OperationsManagement/solutions",
"id": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/',
resourceGroup().name, '/providers/Microsoft.OperationsManagement/solutions/', variables('Updates').Name)]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"properties": {
"workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces/',
parameters('workspaceName'))]"
},
"plan": {
"name": "[variables('Updates').Name]",
"publisher": "Microsoft",
"product": "[Concat('OMSGallery/', variables('Updates').GalleryName)]",
"promotionCode": ""
}
},
{
"apiVersion": "2015-11-01-preview",
"apiVersion": "2015-11-01-preview",
"location": "[parameters('location')]",
"name": "[variables('AntiMalware').Name]",
"type": "Microsoft.OperationsManagement/solutions",
"id": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/',
resourceGroup().name, '/providers/Microsoft.OperationsManagement/solutions/',
variables('AntiMalware').Name)]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"properties": {
"workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces/',
parameters('workspaceName'))]"
},
"plan": {
"name": "[variables('AntiMalware').Name]",
"publisher": "Microsoft",
"product": "[Concat('OMSGallery/', variables('AntiMalware').GalleryName)]",
"promotionCode": ""
}
},
{
"apiVersion": "2015-11-01-preview",
"location": "[parameters('location')]",
"name": "[variables('SQLAssessment').Name]",
"type": "Microsoft.OperationsManagement/solutions",
"id": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/',
resourceGroup().name, '/providers/Microsoft.OperationsManagement/solutions/',
variables('SQLAssessment').Name)]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'))]"
],
"properties": {
"workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces/',
parameters('workspaceName'))]"
},
"plan": {
"name": "[variables('SQLAssessment').Name]",
"publisher": "Microsoft",
"product": "[Concat('OMSGallery/', variables('SQLAssessment').GalleryName)]",
"promotionCode": ""
}
}
]
}
],
"outputs": {
"workspaceName": {
"type": "string",
"value": "[parameters('workspaceName')]"
},
"provisioningState": {
"type": "string",
"value": "[reference(resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName')), '2015-11-01-preview').provisioningState]"
},
"source": {
"type": "string",
"value": "[reference(resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName')), '2015-11-01-preview').source]"
},
"customerId": {
"type": "string",
"value": "[reference(resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName')), '2015-11-01-preview').customerId]"
},
"pricingTier": {
"type": "string",
"value": "[reference(resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName')), '2015-11-01-preview').sku.name]"
},
"retentionInDays": {
"type": "int",
"value": "[reference(resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName')), '2015-11-01-preview').retentionInDays]"
},
"immediatePurgeDataOn30Days": {
"type": "bool",
"value": "[reference(resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName')), '2015-11-01-preview').features.immediatePurgeDataOn30Days]"
},
"portalUrl": {
"type": "string",
"value": "[reference(resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName')), '2015-11-01-preview').portalUrl]"
}
}
}
Command line
Next steps
Deploy Windows agent to Azure VMs using Resource Manager template.
Deploy Linux agent to Azure VMs using Resource Manager template.
Design and build a management solution in Azure
(Preview)
1/14/2020 • 4 minutes to read • Edit Online
NOTE
This is preliminary documentation for creating management solutions in Azure which are currently in preview. Any schema
described below is subject to change.
Management solutions provide packaged management scenarios that customers can add to their Log Analytics
workspace. This article presents a basic process to design and build a management solution that is suitable for
most common requirements. If you are new to building management solutions then you can use this process as a
starting point and then leverage the concepts for more complex solutions as your requirements evolve.
Data sources
The first step in designing a solution is determining the data that you require from the Log Analytics repository.
This data may be collected by a data source or another solution, or your solution may need to provide the
process to collect it.
There are a number of ways data sources that can be collected in the Log Analytics repository as described in
Data sources in Log Analytics. This includes events in the Windows Event Log or generated by Syslog in addition
to performance counters for both Windows and Linux clients. You can also gather data from Azure resources
collected by Azure Monitor.
If you require data that's not accessible through any of the available data sources, then you can use the HTTP
Data Collector API which allows you to write data to the Log Analytics repository from any client that can call a
REST API. The most common means of custom data collection in a management solution is to create a runbook
in Azure Automation that collects the required data from Azure or external resources and uses the Data Collector
API to write to the repository.
Log searches
Log searches are used to extract and analyze data in the Log Analytics repository. They are used by views and
alerts in addition to allowing the user to perform ad hoc analysis of data in the repository.
You should define any queries that you think will be helpful to the user even if they aren't used by any views or
alerts. These will be available to them as Saved Searches in the portal, and you can also include them in a List of
Queries visualization part in your custom view.
Alerts
Alerts in Log Analytics identify issues through log searches against the data in the repository. They either notify
the user or automatically run an action in response. You should identify different alert conditions for your
application and include corresponding alert rules in your solution file.
If the issue can potentially be corrected with an automated process, then you'll typically create a runbook in Azure
Automation to perform this remediation. Most Azure services can be managed with cmdlets which the runbook
would leverage to perform such functionality.
If your solution requires external functionality in response to an alert, then you can use a webhook response. This
allows you to call an external web service sending information from the alert.
Views
Views in Log Analytics are used to visualize data from the Log Analytics repository. Each solution will typically
contain a single view with a tile that is displayed on the user's main dashboard. The view can contain any number
of visualization parts to provide different visualizations of the collected data to the user.
You create custom views using the View Designer which you can later export for inclusion in your solution file.
Next steps
Learn how to create a solution file for your management solution.
Learn the details of Authoring Azure Resource Manager templates.
Search Azure Quickstart Templates for samples of different Resource Manager templates.
Creating a management solution file in Azure
(Preview)
1/14/2020 • 7 minutes to read • Edit Online
NOTE
This is preliminary documentation for creating management solutions in Azure which are currently in preview. Any schema
described below is subject to change.
Management solutions in Azure are implemented as Resource Manager templates. The main task in learning how
to author management solutions is learning how to author a template. This article provides unique details of
templates used for solutions and how to configure typical solution resources.
Tools
You can use any text editor to work with solution files, but we recommend leveraging the features provided in
Visual Studio or Visual Studio Code as described in the following articles.
Creating and deploying Azure resource groups through Visual Studio
Working with Azure Resource Manager Templates in Visual Studio Code
Structure
The basic structure of a management solution file is the same as a Resource Manager Template, which is as
follows. Each of the sections below describes the top-level elements and their contents in a solution.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0",
"parameters": { },
"variables": { },
"resources": [ ],
"outputs": { }
}
Parameters
Parameters are values that you require from the user when they install the management solution. There are
standard parameters that all solutions will have, and you can add additional parameters as required for your
particular solution. How users will provide parameter values when they install your solution will depend on the
particular parameter and how the solution is being installed.
When a user installs your management solution through the Azure Marketplace or Azure QuickStart templates
they are prompted to select a Log Analytics workspace and Automation account. These are used to populate the
values of each of the standard parameters. The user is not prompted to directly provide values for the standard
parameters, but they are prompted to provide values for any additional parameters.
A sample parameter is shown below.
"startTime": {
"type": "string",
"metadata": {
"description": "Enter time for starting VMs by resource group.",
"control": "datetime",
"category": "Schedule"
}
ATTRIBUTE DESCRIPTION
type Data type for the parameter. The input control displayed for
the user depends on the data type.
Standard parameters
The following table lists the standard parameters for all management solutions. These values are populated for
the user instead of prompting for them when your solution is installed from the Azure Marketplace or Quickstart
templates. The user must provide values for them if the solution is installed with another method.
NOTE
The user interface in the Azure Marketplace and Quickstart templates is expecting the parameter names in the table. If you
use different parameter names then the user will be prompted for them, and they will not be automatically populated.
Following is the structure of the standard parameters that you can copy and paste into your solution file.
"parameters": {
"workspaceName": {
"type": "string",
"metadata": {
"description": "A valid Log Analytics workspace name"
}
},
"accountName": {
"type": "string",
"metadata": {
"description": "A valid Azure Automation account name"
}
},
"workspaceRegionId": {
"type": "string",
"metadata": {
"description": "Region of the Log Analytics workspace"
}
},
"regionId": {
"type": "string",
"metadata": {
"description": "Region of the Azure Automation account"
}
},
"pricingTier": {
"type": "string",
"metadata": {
"description": "Pricing tier of both Log Analytics workspace and Azure Automation account"
}
}
}
You refer to parameter values in other elements of the solution with the syntax parameters('parameter name').
For example, to access the workspace name, you would use parameters('workspaceName')
Variables
Variables are values that you will use in the rest of the management solution. These values are not exposed to the
user installing the solution. They are intended to provide the author with a single location where they can manage
values that may be used multiple times throughout the solution. You should put any values specific to your
solution in variables as opposed to hard coding them in the resources element. This makes the code more
readable and allows you to easily change these values in later versions.
Following is an example of a variables element with typical parameters used in solutions.
"variables": {
"SolutionVersion": "1.1",
"SolutionPublisher": "Contoso",
"SolutionName": "My Solution",
"LogAnalyticsApiVersion": "2015-11-01-preview",
"AutomationApiVersion": "2015-10-31"
},
You refer to variable values through the solution with the syntax variables('variable name'). For example, to
access the SolutionName variable, you would use variables('SolutionName').
You can also define complex variables that multiple sets of values. These are particularly useful in management
solutions where you are defining multiple properties for different types of resources. For example, you could
restructure the solution variables shown above to the following.
"variables": {
"Solution": {
"Version": "1.1",
"Publisher": "Contoso",
"Name": "My Solution"
},
"LogAnalyticsApiVersion": "2015-11-01-preview",
"AutomationApiVersion": "2015-10-31"
},
In this case, you refer to variable values through the solution with the syntax variables('variable
name').property. For example, to access the Solution Name variable, you would use
variables('Solution').Name.
Resources
Resources define the different resources that your management solution will install and configure. This will be the
largest and most complex portion of the template. You can get the structure and complete description of resource
elements in Authoring Azure Resource Manager templates. Different resources that you will typically define are
detailed in other articles in this documentation.
Dependencies
The dependsOn element specifies a dependency on another resource. When the solution is installed, a resource
is not created until all of its dependencies have been created. For example, your solution might start a runbook
when it's installed using a job resource. The job resource would be dependent on the runbook resource to make
sure that the runbook is created before the job is created.
Log Analytics workspace and Automation account
Management solutions require a Log Analytics workspace to contain views and an Automation account to contain
runbooks and related resources. These must be available before the resources in the solution are created and
should not be defined in the solution itself. The user will specify a workspace and account when they deploy your
solution, but as the author you should consider the following points.
Solution resource
Each solution requires a resource entry in the resources element that defines the solution itself. This will have a
type of Microsoft.OperationsManagement/solutions and have the following structure. This includes standard
parameters and variables that are typically used to define properties of the solution.
{
"name": "[concat(variables('Solution').Name, '[' ,parameters('workspaceName'), ']')]",
"location": "[parameters('workspaceRegionId')]",
"tags": { },
"type": "Microsoft.OperationsManagement/solutions",
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"dependsOn": [
<list-of-resources>
],
"properties": {
"workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspaceName'))]",
"referencedResources": [
<list-of-referenced-resources>
],
"containedResources": [
<list-of-contained-resources>
]
},
"plan": {
"name": "[concat(variables('Solution').Name, '[' ,parameters('workspaceName'), ']')]",
"Version": "[variables('Solution').Version]",
"product": "[variables('ProductName')]",
"publisher": "[variables('Solution').Publisher]",
"promotionCode": ""
}
}
Dependencies
The solution resource must have a dependency on every other resource in the solution since they need to exist
before the solution can be created. You do this by adding an entry for each resource in the dependsOn element.
Properties
The solution resource has the properties in the following table. This includes the resources referenced and
contained by the solution which defines how the resource is managed after the solution is installed. Each resource
in the solution should be listed in either the referencedResources or the containedResources property.
PROPERTY DESCRIPTION
The example above is for a solution with a runbook, a schedule, and view. The schedule and runbook are
referenced in the properties element so they are not removed when the solution is removed. The view is
contained so it is removed when the solution is removed.
Plan
The plan entity of the solution resource has the properties in the following table.
PROPERTY DESCRIPTION
Next steps
Add saved searches and alerts to your management solution.
Add views to your management solution.
Add runbooks and other Automation resources to your management solution.
Learn the details of Authoring Azure Resource Manager templates.
Search Azure Quickstart Templates for samples of different Resource Manager templates.
Adding Azure Automation resources to a
management solution (Preview)
12/23/2019 • 12 minutes to read • Edit Online
NOTE
This is preliminary documentation for creating management solutions which are currently in preview. Any schema described
below is subject to change.
Management solutions will typically include runbooks in Azure Automation to automate processes such as
collecting and processing monitoring data. In addition to runbooks, Automation accounts includes assets such as
variables and schedules that support the runbooks used in the solution. This article describes how to include
runbooks and their related resources in a solution.
NOTE
The samples in this article use parameters and variables that are either required or common to management solutions and
described in Design and build a management solution in Azure
Prerequisites
This article assumes that you're already familiar with the following information.
How to create a management solution.
The structure of a solution file.
How to author Resource Manager templates
Automation account
All resources in Azure Automation are contained in an Automation account. As described in Log Analytics
workspace and Automation account the Automation account isn't included in the management solution but must
exist before the solution is installed. If it isn't available, then the solution install will fail.
The name of each Automation resource includes the name of its Automation account. This is done in the solution
with the accountName parameter as in the following example of a runbook resource.
Runbooks
You should include any runbooks used by the solution in the solution file so that they're created when the solution
is installed. You cannot contain the body of the runbook in the template though, so you should publish the
runbook to a public location where it can be accessed by any user installing your solution.
Azure Automation runbook resources have a type of Microsoft.Automation/automationAccounts/runbooks
and have the following structure. This includes common variables and parameters so that you can copy and paste
this code snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', variables('Runbook').Name)]",
"type": "Microsoft.Automation/automationAccounts/runbooks",
"apiVersion": "[variables('AutomationApiVersion')]",
"dependsOn": [
],
"location": "[parameters('regionId')]",
"tags": { },
"properties": {
"runbookType": "[variables('Runbook').Type]",
"logProgress": "true",
"logVerbose": "true",
"description": "[variables('Runbook').Description]",
"publishContentLink": {
"uri": "[variables('Runbook').Uri]",
"version": [variables('Runbook').Version]"
}
}
}
PROPERTY DESCRIPTION
uri - Uri to the content of the runbook. This will be a .ps1 file
for PowerShell and Script runbooks, and an exported graphical
runbook file for a Graph runbook.
version - Version of the runbook for your own tracking.
Automation jobs
When you start a runbook in Azure Automation, it creates an automation job. You can add an automation job
resource to your solution to automatically start a runbook when the management solution is installed. This
method is typically used to start runbooks that are used for initial configuration of the solution. To start a runbook
at regular intervals, create a schedule and a job schedule
Job resources have a type of Microsoft.Automation/automationAccounts/jobs and have the following
structure. This includes common variables and parameters so that you can copy and paste this code snippet into
your solution file and change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', parameters('Runbook').JobGuid)]",
"type": "Microsoft.Automation/automationAccounts/jobs",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"dependsOn": [
"[concat('Microsoft.Automation/automationAccounts/', parameters('accountName'), '/runbooks/',
variables('Runbook').Name)]"
],
"tags": { },
"properties": {
"runbook": {
"name": "[variables('Runbook').Name]"
},
"parameters": {
"Parameter1": "[[variables('Runbook').Parameter1]",
"Parameter2": "[[variables('Runbook').Parameter2]"
}
}
}
The properties for automation jobs are described in the following table.
PROPERTY DESCRIPTION
runbook Single name entity with the name of the runbook to start.
The job includes the runbook name and any parameter values to be sent to the runbook. The job should depend
on the runbook that it's starting since the runbook must be created before the job. If you have multiple runbooks
that should be started you can define their order by having a job depend on any other jobs that should be run first.
The name of a job resource must contain a GUID which is typically assigned by a parameter. You can read more
about GUID parameters in Creating a management solution file in Azure.
Certificates
Azure Automation certificates have a type of Microsoft.Automation/automationAccounts/certificates and
have the following structure. This includes common variables and parameters so that you can copy and paste this
code snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', variables('Certificate').Name)]",
"type": "Microsoft.Automation/automationAccounts/certificates",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"tags": { },
"dependsOn": [
],
"properties": {
"base64Value": "[variables('Certificate').Base64Value]",
"thumbprint": "[variables('Certificate').Thumbprint]"
}
}
The properties for Certificates resources are described in the following table.
PROPERTY DESCRIPTION
Credentials
Azure Automation credentials have a type of Microsoft.Automation/automationAccounts/credentials and
have the following structure. This includes common variables and parameters so that you can copy and paste this
code snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', variables('Credential').Name)]",
"type": "Microsoft.Automation/automationAccounts/credentials",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"tags": { },
"dependsOn": [
],
"properties": {
"userName": "[parameters('credentialUsername')]",
"password": "[parameters('credentialPassword')]"
}
}
The properties for Credential resources are described in the following table.
PROPERTY DESCRIPTION
Schedules
Azure Automation schedules have a type of Microsoft.Automation/automationAccounts/schedules and
have the following structure. This includes common variables and parameters so that you can copy and paste this
code snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', variables('Schedule').Name)]",
"type": "microsoft.automation/automationAccounts/schedules",
"apiVersion": "[variables('AutomationApiVersion')]",
"tags": { },
"dependsOn": [
],
"properties": {
"description": "[variables('Schedule').Description]",
"startTime": "[parameters('scheduleStartTime')]",
"timeZone": "[parameters('scheduleTimeZone')]",
"isEnabled": "[variables('Schedule').IsEnabled]",
"interval": "[variables('Schedule').Interval]",
"frequency": "[variables('Schedule').Frequency]"
}
}
The properties for schedule resources are described in the following table.
PROPERTY DESCRIPTION
day
hour
Schedules must have a start time with a value greater than the current time. You cannot provide this value with a
variable since you would have no way of knowing when it's going to be installed.
Use one of the following two strategies when using schedule resources in a solution.
Use a parameter for the start time of the schedule. This will prompt the user to provide a value when they
install the solution. If you have multiple schedules, you could use a single parameter value for more than one of
them.
Create the schedules using a runbook that starts when the solution is installed. This removes the requirement
of the user to specify a time, but you can't contain the schedule in your solution so it will be removed when the
solution is removed.
Job schedules
Job schedule resources link a runbook with a schedule. They have a type of
Microsoft.Automation/automationAccounts/jobSchedules and have the following structure. This includes
common variables and parameters so that you can copy and paste this code snippet into your solution file and
change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', variables('Schedule').LinkGuid)]",
"type": "microsoft.automation/automationAccounts/jobSchedules",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"dependsOn": [
"[resourceId('Microsoft.Automation/automationAccounts/runbooks/', parameters('accountName'),
variables('Runbook').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/schedules/', parameters('accountName'),
variables('Schedule').Name)]"
],
"tags": {
},
"properties": {
"schedule": {
"name": "[variables('Schedule').Name]"
},
"runbook": {
"name": "[variables('Runbook').Name]"
}
}
}
The properties for job schedules are described in the following table.
PROPERTY DESCRIPTION
schedule name Single name entity with the name of the schedule.
runbook name Single name entity with the name of the runbook.
Variables
Azure Automation variables have a type of Microsoft.Automation/automationAccounts/variables and have
the following structure. This includes common variables and parameters so that you can copy and paste this code
snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', variables('Variable').Name)]",
"type": "microsoft.automation/automationAccounts/variables",
"apiVersion": "[variables('AutomationApiVersion')]",
"tags": { },
"dependsOn": [
],
"properties": {
"description": "[variables('Variable').Description]",
"isEncrypted": "[variables('Variable').Encrypted]",
"type": "[variables('Variable').Type]",
"value": "[variables('Variable').Value]"
}
}
The properties for variable resources are described in the following table.
PROPERTY DESCRIPTION
type This property currently has no effect. The data type of the
variable will be determined by the initial value.
NOTE
The type property currently has no effect on the variable being created. The data type for the variable will be determined by
the value.
If you set the initial value for the variable, it must be configured as the correct data type. The following table
provides the different data types allowable and their syntax. Note that values in JSON are expected to always be
enclosed in quotes with any special characters within the quotes. For example, a string value would be specified by
quotes around the string (using the escape character (\)) while a numeric value would be specified with one set of
quotes.
Modules
Your management solution does not need to define global modules used by your runbooks because they will
always be available in your Automation account. You do need to include a resource for any other module used by
your runbooks.
Integration modules have a type of Microsoft.Automation/automationAccounts/modules and have the
following structure. This includes common variables and parameters so that you can copy and paste this code
snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('accountName'), '/', variables('Module').Name)]",
"type": "Microsoft.Automation/automationAccounts/modules",
"apiVersion": "[variables('AutomationApiVersion')]",
"dependsOn": [
],
"properties": {
"contentLink": {
"uri": "[variables('Module').Uri]"
}
}
}
The properties for module resources are described in the following table.
PROPERTY DESCRIPTION
uri - Uri to the content of the module. This will be a .ps1 file
for PowerShell and Script runbooks, and an exported graphical
runbook file for a Graph runbook.
version - Version of the module for your own tracking.
The runbook should depend on the module resource to ensure that it's created before the runbook.
Updating modules
If you update a management solution that includes a runbook that uses a schedule, and the new version of your
solution has a new module used by that runbook, then the runbook may use the old version of the module. You
should include the following runbooks in your solution and create a job to run them before any other runbooks.
This will ensure that any modules are updated as required before the runbooks are loaded.
Update-ModulesinAutomationToLatestVersion will ensure that all of the modules used by runbooks in your
solution are the latest version.
ReRegisterAutomationSchedule-MS -Mgmt will reregister all of the schedule resources to ensure that the
runbooks linked to them with use the latest modules.
Sample
Following is a sample of a solution that include that includes the following resources:
Runbook. This is a sample runbook stored in a public GitHub repository.
Automation job that starts the runbook when the solution is installed.
Schedule and job schedule to start the runbook at regular intervals.
Certificate.
Credential.
Variable.
Module. This is the OMSIngestionAPI module for writing data to Log Analytics.
The sample uses standard solution parameters variables that would commonly be used in a solution as opposed
to hardcoding values in the resource definitions.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "string",
"metadata": {
"Description": "Name of Log Analytics workspace."
}
},
"accountName": {
"type": "string",
"metadata": {
"Description": "Name of Automation account."
}
},
"workspaceregionId": {
"type": "string",
"metadata": {
"Description": "Region of Log Analytics workspace."
}
},
"regionId": {
"type": "string",
"metadata": {
"Description": "Region of Automation account."
}
},
"pricingTier": {
"type": "string",
"metadata": {
"Description": "Pricing tier of both Log Analytics workspace and Azure Automation account."
}
},
"certificateBase64Value": {
"type": "string",
"metadata": {
"Description": "Base 64 value for certificate."
}
},
"certificateThumbprint": {
"type": "securestring",
"metadata": {
"Description": "Thumbprint for certificate."
}
},
"credentialUsername": {
"type": "string",
"metadata": {
"Description": "Username for credential."
}
},
"credentialPassword": {
"type": "securestring",
"metadata": {
"Description": "Password for credential."
}
},
"scheduleStartTime": {
"type": "string",
"metadata": {
"Description": "Start time for schedule."
}
},
"scheduleTimeZone": {
"type": "string",
"metadata": {
"Description": "Time zone for schedule."
}
},
"scheduleLinkGuid": {
"type": "string",
"metadata": {
"description": "GUID for the schedule link to runbook.",
"control": "guid"
}
},
"runbookJobGuid": {
"type": "string",
"metadata": {
"description": "GUID for the runbook job.",
"control": "guid"
}
}
},
"variables": {
"SolutionName": "MySolution",
"SolutionVersion": "1.0",
"SolutionPublisher": "Contoso",
"ProductName": "SampleSolution",
"LogAnalyticsApiVersion": "2015-11-01-preview",
"AutomationApiVersion": "2015-10-31",
"Runbook": {
"Name": "MyRunbook",
"Description": "Sample runbook",
"Type": "PowerShell",
"Uri": "https://raw.githubusercontent.com/user/myrepo/master/samples/MyRunbook.ps1",
"JobGuid": "[parameters('runbookJobGuid')]"
},
"Certificate": {
"Name": "MyCertificate",
"Base64Value": "[parameters('certificateBase64Value')]",
"Thumbprint": "[parameters('certificateThumbprint')]"
},
"Credential": {
"Name": "MyCredential",
"UserName": "[parameters('credentialUsername')]",
"Password": "[parameters('credentialPassword')]"
},
"Schedule": {
"Name": "MySchedule",
"Description": "Sample schedule",
"IsEnabled": "true",
"Interval": "1",
"Frequency": "hour",
"StartTime": "[parameters('scheduleStartTime')]",
"TimeZone": "[parameters('scheduleTimeZone')]",
"LinkGuid": "[parameters('scheduleLinkGuid')]"
},
"Variable": {
"Name": "MyVariable",
"Description": "Sample variable",
"Encrypted": 0,
"Type": "string",
"Value": "'This is my string value.'"
},
"Module": {
"Name": "OMSIngestionAPI",
"Uri": "https://devopsgallerystorage.blob.core.windows.net/packages/omsingestionapi.1.3.0.nupkg"
}
},
"resources": [
{
"name": "[concat(variables('SolutionName'), '[' ,parameters('workspacename'), ']')]",
"location": "[parameters('workspaceRegionId')]",
"tags": { },
"tags": { },
"type": "Microsoft.OperationsManagement/solutions",
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"dependsOn": [
"[resourceId('Microsoft.Automation/automationAccounts/runbooks/', parameters('accountName'),
variables('Runbook').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/jobs/', parameters('accountName'),
variables('Runbook').JobGuid)]",
"[resourceId('Microsoft.Automation/automationAccounts/certificates/', parameters('accountName'),
variables('Certificate').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/credentials/', parameters('accountName'),
variables('Credential').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/schedules/', parameters('accountName'),
variables('Schedule').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/jobSchedules/', parameters('accountName'),
variables('Schedule').LinkGuid)]",
"[resourceId('Microsoft.Automation/automationAccounts/variables/', parameters('accountName'),
variables('Variable').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/modules/', parameters('accountName'),
variables('Module').Name)]"
],
"properties": {
"workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspacename'))]",
"referencedResources": [
"[resourceId('Microsoft.Automation/automationAccounts/modules/', parameters('accountName'),
variables('Module').Name)]"
],
"containedResources": [
"[resourceId('Microsoft.Automation/automationAccounts/runbooks/', parameters('accountName'),
variables('Runbook').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/jobs/', parameters('accountName'),
variables('Runbook').JobGuid)]",
"[resourceId('Microsoft.Automation/automationAccounts/certificates/', parameters('accountName'),
variables('Certificate').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/credentials/', parameters('accountName'),
variables('Credential').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/schedules/', parameters('accountName'),
variables('Schedule').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/jobSchedules/', parameters('accountName'),
variables('Schedule').LinkGuid)]",
"[resourceId('Microsoft.Automation/automationAccounts/variables/', parameters('accountName'),
variables('Variable').Name)]"
]
},
"plan": {
"name": "[concat(variables('SolutionName'), '[' ,parameters('workspaceName'), ']')]",
"Version": "[variables('SolutionVersion')]",
"product": "[variables('ProductName')]",
"publisher": "[variables('SolutionPublisher')]",
"promotionCode": ""
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Runbook').Name)]",
"type": "Microsoft.Automation/automationAccounts/runbooks",
"apiVersion": "[variables('AutomationApiVersion')]",
"dependsOn": [
],
"location": "[parameters('regionId')]",
"tags": { },
"properties": {
"runbookType": "[variables('Runbook').Type]",
"logProgress": "true",
"logVerbose": "true",
"description": "[variables('Runbook').Description]",
"publishContentLink": {
"uri": "[variables('Runbook').Uri]",
"version": "1.0.0.0"
}
}
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Runbook').JobGuid)]",
"type": "Microsoft.Automation/automationAccounts/jobs",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"dependsOn": [
"[concat('Microsoft.Automation/automationAccounts/', parameters('accountName'), '/runbooks/',
variables('Runbook').Name)]"
],
"tags": { },
"properties": {
"runbook": {
"name": "[variables('Runbook').Name]"
},
"parameters": {
"targetSubscriptionId": "[subscription().subscriptionId]",
"resourcegroup": "[resourceGroup().name]",
"automationaccount": "[parameters('accountName')]"
}
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Certificate').Name)]",
"type": "Microsoft.Automation/automationAccounts/certificates",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"tags": { },
"dependsOn": [
],
"properties": {
"Base64Value": "[variables('Certificate').Base64Value]",
"Thumbprint": "[variables('Certificate').Thumbprint]"
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Credential').Name)]",
"type": "Microsoft.Automation/automationAccounts/credentials",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"tags": { },
"dependsOn": [
],
"properties": {
"userName": "[variables('Credential').UserName]",
"password": "[variables('Credential').Password]"
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Schedule').Name)]",
"type": "microsoft.automation/automationAccounts/schedules",
"apiVersion": "[variables('AutomationApiVersion')]",
"tags": { },
"dependsOn": [
],
"properties": {
"description": "[variables('Schedule').Description]",
"startTime": "[variables('Schedule').StartTime]",
"timeZone": "[variables('Schedule').TimeZone]",
"isEnabled": "[variables('Schedule').IsEnabled]",
"interval": "[variables('Schedule').Interval]",
"frequency": "[variables('Schedule').Frequency]"
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Schedule').LinkGuid)]",
"type": "microsoft.automation/automationAccounts/jobSchedules",
"apiVersion": "[variables('AutomationApiVersion')]",
"apiVersion": "[variables('AutomationApiVersion')]",
"location": "[parameters('regionId')]",
"dependsOn": [
"[resourceId('Microsoft.Automation/automationAccounts/runbooks/', parameters('accountName'),
variables('Runbook').Name)]",
"[resourceId('Microsoft.Automation/automationAccounts/schedules/', parameters('accountName'),
variables('Schedule').Name)]"
],
"tags": {
},
"properties": {
"schedule": {
"name": "[variables('Schedule').Name]"
},
"runbook": {
"name": "[variables('Runbook').Name]"
}
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Variable').Name)]",
"type": "microsoft.automation/automationAccounts/variables",
"apiVersion": "[variables('AutomationApiVersion')]",
"tags": { },
"dependsOn": [
],
"properties": {
"description": "[variables('Variable').Description]",
"isEncrypted": "[variables('Variable').Encrypted]",
"type": "[variables('Variable').Type]",
"value": "[variables('Variable').Value]"
}
},
{
"name": "[concat(parameters('accountName'), '/', variables('Module').Name)]",
"type": "Microsoft.Automation/automationAccounts/modules",
"apiVersion": "[variables('AutomationApiVersion')]",
"dependsOn": [
],
"properties": {
"contentLink": {
"uri": "[variables('Module').Uri]"
}
}
}
],
"outputs": { }
}
Next steps
Add a view to your solution to visualize collected data.
Adding Log Analytics saved searches and alerts to
management solution (Preview)
1/14/2020 • 10 minutes to read • Edit Online
IMPORTANT
As announced earlier, log analytics workspace(s) created after June 1, 2019 - will be able to manage alert rules using only
Azure scheduledQueryRules REST API, Azure Resource Manager Template and PowerShell cmdlet. Customers can easily
switch their preferred means of alert rule management for older workspaces to leverage Azure Monitor scheduledQueryRules
as default and gain many new benefits like the ability to use native PowerShell cmdlets, increased lookback time period in
rules, creation of rules in separate resource group or subscription and much more.
NOTE
This is preliminary documentation for creating management solutions which are currently in preview. Any schema described
below is subject to change.
Management solutions will typically include saved searches in Log Analytics to analyze data collected by the
solution. They may also define alerts to notify the user or automatically take action in response to a critical issue.
This article describes how to define Log Analytics saved searches and alerts in a Resource Management template
so they can be included in management solutions.
NOTE
The samples in this article use parameters and variables that are either required or common to management solutions and
described in Design and build a management solution in Azure
Prerequisites
This article assumes that you're already familiar with how to create a management solution and the structure of a
Resource Manager template and solution file.
Saved Searches
Include saved searches in a solution to allow users to query data collected by your solution. Saved searches appear
under Saved Searches in the Azure portal. A saved search is also required for each alert.
Log Analytics saved search resources have a type of Microsoft.OperationalInsights/workspaces/savedSearches and
have the following structure. This includes common variables and parameters so that you can copy and paste this
code snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('workspaceName'), '/', variables('SavedSearch').Name)]",
"type": "Microsoft.OperationalInsights/workspaces/savedSearches",
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"dependsOn": [
],
"tags": { },
"properties": {
"etag": "*",
"query": "[variables('SavedSearch').Query]",
"displayName": "[variables('SavedSearch').DisplayName]",
"category": "[variables('SavedSearch').Category]"
}
}
PROPERTY DESCRIPTION
category The category for the saved search. Any saved searches in the
same solution will often share a single category so they are
grouped together in the console.
NOTE
You may need to use escape characters in the query if it includes characters that could be interpreted as JSON. For example,
if your query was AzureActivity | OperationName:"Microsoft.Compute/virtualMachines/write", it should be written in
the solution file as AzureActivity | OperationName:/"Microsoft.Compute/virtualMachines/write".
Alerts
Azure Log alerts are created by Azure Alert rules that run specified log queries at regular intervals. If the results of
the query match specified criteria, an alert record is created and one or more actions are run using Action Groups.
For users that extend alerts to Azure, actions are now controlled in Azure action groups. When a workspace and its
alerts are extended to Azure, you can retrieve or add actions by using the Action Group - Azure Resource Manager
Template. Alert rules in legacy management solution are made up of the following three different resources.
Saved search. Defines the log search that is run. Multiple alert rules can share a single saved search.
Schedule. Defines how often the log search is run. Each alert rule has one and only one schedule.
Alert action. Each alert rule has one action group resource or action resource (legacy) with a type of Alert that
defines the details of the alert such as the criteria for when an alert record is created and the alert's severity.
Action group resource can have a list of configured actions to take when alert is fired - such as voice call, SMS,
email, webhook, ITSM tool, automation runbook, logic app, etc.
Saved search resources are described above. The other resources are described below.
Schedule resource
A saved search can have one or more schedules with each schedule representing a separate alert rule. The
schedule defines how often the search is run and the time interval over which the data is retrieved. Schedule
resources have a type of Microsoft.OperationalInsights/workspaces/savedSearches/schedules/ and have the
following structure. This includes common variables and parameters so that you can copy and paste this code
snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('workspaceName'), '/', variables('SavedSearch').Name, '/',
variables('Schedule').Name)]",
"type": "Microsoft.OperationalInsights/workspaces/savedSearches/schedules/",
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'), '/savedSearches/',
variables('SavedSearch').Name)]"
],
"properties": {
"etag": "*",
"interval": "[variables('Schedule').Interval]",
"queryTimeSpan": "[variables('Schedule').TimeSpan]",
"enabled": "[variables('Schedule').Enabled]"
}
}
The properties for schedule resources are described in the following table.
The schedule resource should depend on the saved search so that it's created before the schedule.
NOTE
Schedule Name must be unique in a given workspace; two schedules cannot have the same ID even if they are associated
with different saved searches. Also name for all saved searches, schedules, and actions created with the Log Analytics API
must be in lowercase.
Actions
A schedule can have multiple actions. An action may define one or more processes to perform such as sending a
mail or starting a runbook, or it may define a threshold that determines when the results of a search match some
criteria. Some actions will define both so that the processes are performed when the threshold is met. Actions can
be defined using [action group] resource or action resource.
There are two types of action resource specified by the Type property. A schedule requires one Alert action, which
defines the details of the alert rule and what actions are taken when an alert is created. Action resources have a
type of Microsoft.OperationalInsights/workspaces/savedSearches/schedules/actions .
Alert actions have the following structure. This includes common variables and parameters so that you can copy
and paste this code snippet into your solution file and change the parameter names.
{
"name": "[concat(parameters('workspaceName'), '/', variables('SavedSearch').Name, '/',
variables('Schedule').Name, '/', variables('Alert').Name)]",
"type": "Microsoft.OperationalInsights/workspaces/savedSearches/schedules/actions",
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'), '/savedSearches/',
variables('SavedSearch').Name, '/schedules/', variables('Schedule').Name)]"
],
"properties": {
"etag": "*",
"type": "Alert",
"name": "[variables('Alert').Name]",
"description": "[variables('Alert').Description]",
"severity": "[variables('Alert').Severity]",
"threshold": {
"operator": "[variables('Alert').Threshold.Operator]",
"value": "[variables('Alert').Threshold.Value]",
"metricsTrigger": {
"triggerCondition": "[variables('Alert').Threshold.Trigger.Condition]",
"operator": "[variables('Alert').Trigger.Operator]",
"value": "[variables('Alert').Trigger.Value]"
},
},
"AzNsNotification": {
"GroupIds": "[variables('MyAlert').AzNsNotification.GroupIds]",
"CustomEmailSubject": "[variables('MyAlert').AzNsNotification.CustomEmailSubject]",
"CustomWebhookPayload": "[variables('MyAlert').AzNsNotification.CustomWebhookPayload]"
}
}
}
The properties for Alert action resources are described in the following tables.
critical
warning
informational
Threshold
This section is required. It defines the properties for the alert threshold.
gt = greater than
lt = less than
Met r i c sT r i gger
Total
Consecutive
gt = greater than
lt = less than
Throttling
This section is optional. Include this section if you want to suppress alerts from the same rule for some amount of
time after an alert is created.
Sample
Following is a sample of a solution that includes the following resources:
Saved search
Schedule
Action group
The sample uses standard solution parameters variables that would commonly be used in a solution as opposed to
hardcoding values in the resource definitions.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0",
"parameters": {
"workspaceName": {
"type": "string",
"metadata": {
"Description": "Name of Log Analytics workspace"
}
},
"workspaceregionId": {
"type": "string",
"metadata": {
"Description": "Region of Log Analytics workspace"
}
},
"actiongroup": {
"type": "string",
"metadata": {
"Description": "List of action groups for alert actions separated by semicolon"
}
}
},
"variables": {
"SolutionName": "MySolution",
"SolutionVersion": "1.0",
"SolutionPublisher": "Contoso",
"ProductName": "SampleSolution",
"LogAnalyticsApiVersion-Search": "2017-03-15-preview",
"LogAnalyticsApiVersion-Solution": "2015-11-01-preview",
"MySearch": {
"displayName": "Error records by hour",
"query": "MyRecord_CL | summarize AggregatedValue = avg(Rating_d) by Instance_s,
bin(TimeGenerated, 60m)",
"category": "Samples",
"name": "Samples-Count of data"
},
"MyAlert": {
"Name": "[toLower(concat('myalert-',uniqueString(resourceGroup().id, deployment().name)))]",
"DisplayName": "My alert rule",
"Description": "Sample alert. Fires when 3 error records found over hour interval.",
"Severity": "critical",
"ThresholdOperator": "gt",
"ThresholdValue": 3,
"Schedule": {
"Name": "[toLower(concat('myschedule-',uniqueString(resourceGroup().id,
deployment().name)))]",
"Interval": 15,
"TimeSpan": 60
},
"MetricsTrigger": {
"TriggerCondition": "Consecutive",
"Operator": "gt",
"Value": 3
},
"ThrottleMinutes": 60,
"AzNsNotification": {
"GroupIds": [
"[parameters('actiongroup')]"
],
"CustomEmailSubject": "Sample alert"
}
}
},
"resources": [
{
"name": "[concat(variables('SolutionName'), '[' ,parameters('workspacename'), ']')]",
"location": "[parameters('workspaceRegionId')]",
"tags": { },
"type": "Microsoft.OperationsManagement/solutions",
"apiVersion": "[variables('LogAnalyticsApiVersion-Solution')]",
"dependsOn": [
"[resourceId('Microsoft.OperationalInsights/workspaces/savedSearches',
parameters('workspacename'), variables('MySearch').Name)]",
"[resourceId('Microsoft.OperationalInsights/workspaces/savedSearches/schedules',
parameters('workspacename'), variables('MySearch').Name, variables('MyAlert').Schedule.Name)]",
"[resourceId('Microsoft.OperationalInsights/workspaces/savedSearches/schedules/actions',
parameters('workspacename'), variables('MySearch').Name, variables('MyAlert').Schedule.Name,
variables('MyAlert').Name)]"
],
"properties": {
"workspaceResourceId": "[resourceId('Microsoft.OperationalInsights/workspaces',
parameters('workspacename'))]",
"referencedResources": [
],
"containedResources": [
"[resourceId('Microsoft.OperationalInsights/workspaces/savedSearches',
parameters('workspacename'), variables('MySearch').Name)]",
"[resourceId('Microsoft.OperationalInsights/workspaces/savedSearches/schedules',
parameters('workspacename'), variables('MySearch').Name, variables('MyAlert').Schedule.Name)]",
"[resourceId('Microsoft.OperationalInsights/workspaces/savedSearches/schedules/actions',
parameters('workspacename'), variables('MySearch').Name, variables('MyAlert').Schedule.Name,
variables('MyAlert').Name)]"
]
]
},
"plan": {
"name": "[concat(variables('SolutionName'), '[' ,parameters('workspaceName'), ']')]",
"Version": "[variables('SolutionVersion')]",
"product": "[variables('ProductName')]",
"publisher": "[variables('SolutionPublisher')]",
"promotionCode": ""
}
},
{
"name": "[concat(parameters('workspaceName'), '/', variables('MySearch').Name)]",
"type": "Microsoft.OperationalInsights/workspaces/savedSearches",
"apiVersion": "[variables('LogAnalyticsApiVersion-Search')]",
"dependsOn": [ ],
"tags": { },
"properties": {
"etag": "*",
"query": "[variables('MySearch').query]",
"displayName": "[variables('MySearch').displayName]",
"category": "[variables('MySearch').category]"
}
},
{
"name": "[concat(parameters('workspaceName'), '/', variables('MySearch').Name, '/',
variables('MyAlert').Schedule.Name)]",
"type": "Microsoft.OperationalInsights/workspaces/savedSearches/schedules/",
"apiVersion": "[variables('LogAnalyticsApiVersion-Search')]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'),
'/savedSearches/', variables('MySearch').Name)]"
],
"properties": {
"etag": "*",
"interval": "[variables('MyAlert').Schedule.Interval]",
"queryTimeSpan": "[variables('MyAlert').Schedule.TimeSpan]",
"enabled": true
}
},
{
"name": "[concat(parameters('workspaceName'), '/', variables('MySearch').Name, '/',
variables('MyAlert').Schedule.Name, '/', variables('MyAlert').Name)]",
"type": "Microsoft.OperationalInsights/workspaces/savedSearches/schedules/actions",
"apiVersion": "[variables('LogAnalyticsApiVersion-Search')]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'),
'/savedSearches/', variables('MySearch').Name, '/schedules/', variables('MyAlert').Schedule.Name)]"
],
"properties": {
"etag": "*",
"Type": "Alert",
"Name": "[variables('MyAlert').DisplayName]",
"Description": "[variables('MyAlert').Description]",
"Severity": "[variables('MyAlert').Severity]",
"Threshold": {
"Operator": "[variables('MyAlert').ThresholdOperator]",
"Value": "[variables('MyAlert').ThresholdValue]",
"MetricsTrigger": {
"TriggerCondition": "[variables('MyAlert').MetricsTrigger.TriggerCondition]",
"Operator": "[variables('MyAlert').MetricsTrigger.Operator]",
"Value": "[variables('MyAlert').MetricsTrigger.Value]"
}
},
"Throttling": {
"DurationInMinutes": "[variables('MyAlert').ThrottleMinutes]"
},
"AzNsNotification": {
"GroupIds": "[variables('MyAlert').AzNsNotification.GroupIds]",
"CustomEmailSubject": "[variables('MyAlert').AzNsNotification.CustomEmailSubject]"
}
}
}
}
]
}
The following parameter file provides samples values for this solution.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspacename": {
"value": "myWorkspace"
},
"accountName": {
"value": "myAccount"
},
"workspaceregionId": {
"value": "East US"
},
"regionId": {
"value": "East US 2"
},
"pricingTier": {
"value": "Free"
},
"actiongroup": {
"value": "/subscriptions/3b540246-808d-4331-99aa-
917b808a9166/resourcegroups/myTestGroup/providers/microsoft.insights/actiongroups/sample"
}
}
}
Next steps
Add views to your management solution.
Add Automation runbooks and other resources to your management solution.
Views in management solutions (Preview)
10/17/2019 • 4 minutes to read • Edit Online
NOTE
This is preliminary documentation for creating management solutions which are currently in preview. Any schema described
below is subject to change.
Management solutions will typically include one or more views to visualize data. This article describes how to
export a view created by the View Designer and include it in a management solution.
NOTE
The samples in this article use parameters and variables that are either required or common to management solutions and
described in Design and build a management solution in Azure
Prerequisites
This article assumes that you're already familiar with how to create a management solution and the structure of a
solution file.
Overview
To include a view in a management solution, you create a resource for it in the solution file. The JSON that
describes the view's detailed configuration is typically complex though and not something that a typical solution
author would be able to create manually. The most common method is to create the view using the View
Designer, export it, and then add its detailed configuration to the solution.
The basic steps to add a view to a solution are as follows. Each step is described in further detail in the sections
below.
1. Export the view to a file.
2. Create the view resource in the solution.
3. Add the view details.
{
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"name": "[concat(parameters('workspaceName'), '/', variables('ViewName'))]",
"type": "Microsoft.OperationalInsights/workspaces/views",
"location": "[parameters('workspaceregionId')]",
"id": "[Concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name,
'/providers/Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'),'/views/',
variables('ViewName'))]",
"dependson": [
],
"properties": {
"Id": "[variables('ViewName')]",
"Name": "[variables('ViewName')]",
"DisplayName": "[variables('ViewName')]",
"Description": "",
"Author": "[variables('ViewAuthor')]",
"Source": "Local",
"Dashboard": ,
"OverviewTile":
}
}
Add the following variables to the variables element of the solution file and replace the values to those for your
solution.
"LogAnalyticsApiVersion": "<api-version>",
"ViewAuthor": "Your name."
"ViewDescription": "Optional description of the view."
"ViewName": "Provide a name for the view here."
Note that you could copy the entire view resource from your exported view file, but you would need to make the
following changes for it to work in your solution.
The type for the view resource needs to be changed from views to
Microsoft.OperationalInsights/workspaces.
The name property for the view resource needs to be changed to include the workspace name.
The dependency on the workspace needs to be removed since the workspace resource isn't defined in the
solution.
DisplayName property needs to be added to the view. The Id, Name, and DisplayName must all match.
Parameter names must be changed to match the required set of parameters.
Variables should be defined in the solution and used in the appropriate properties.
Log Analytics API version
All Log Analytics resources defined in a Resource Manager template have a property apiVersion that defines the
version of the API the resource should use. This version is different for views with queries that use the legacy and
the upgraded query language.
The following table specifies the Log Analytics API versions for views in legacy and upgraded workspaces:
Example
For example, the following sample shows a simple solution file with a view. Ellipses (...) are shown for the
Dashboard and OverviewTile contents for space reasons.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"workspaceName": {
"type": "string"
},
"accountName": {
"type": "string"
},
"workspaceRegionId": {
"type": "string"
},
"regionId": {
"type": "string"
},
"pricingTier": {
"type": "string"
}
},
"variables": {
"SolutionVersion": "1.1",
"SolutionPublisher": "Contoso",
"SolutionName": "Contoso Solution",
"LogAnalyticsApiVersion": "2015-11-01-preview",
"ViewAuthor": "user@contoso.com",
"ViewDescription": "This is a sample view.",
"ViewName": "Contoso View"
},
"resources": [
{
"name": "[concat(variables('SolutionName'), '(' ,parameters('workspacename'), ')')]",
"location": "[parameters('workspaceRegionId')]",
"tags": { },
"type": "Microsoft.OperationsManagement/solutions",
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"dependsOn": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspacename'), '/views/',
variables('ViewName'))]"
variables('ViewName'))]"
],
"properties": {
"workspaceResourceId": "[concat(resourceGroup().id,
'/providers/Microsoft.OperationalInsights/workspaces/', parameters('workspacename'))]",
"referencedResources": [
],
"containedResources": [
"[concat('Microsoft.OperationalInsights/workspaces/', parameters('workspaceName'),
'/views/', variables('ViewName'))]"
]
},
"plan": {
"name": "[concat(variables('SolutionName'), '(' ,parameters('workspaceName'), ')')]",
"Version": "[variables('SolutionVersion')]",
"product": "ContosoSolution",
"publisher": "[variables('SolutionPublisher')]",
"promotionCode": ""
}
},
{
"apiVersion": "[variables('LogAnalyticsApiVersion')]",
"name": "[concat(parameters('workspaceName'), '/', variables('ViewName'))]",
"type": "Microsoft.OperationalInsights/workspaces/views",
"location": "[parameters('workspaceregionId')]",
"id": "[Concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/',
resourceGroup().name, '/providers/Microsoft.OperationalInsights/workspaces/',
parameters('workspaceName'),'/views/', variables('ViewName'))]",
"dependson": [
],
"properties": {
"Id": "[variables('ViewName')]",
"Name": "[variables('ViewName')]",
"DisplayName": "[variables('ViewName')]",
"Description": "[variables('ViewDescription')]",
"Author": "[variables('ViewAuthor')]",
"Source": "Local",
"Dashboard": ...,
"OverviewTile": ...
}
}
]
}
Next steps
Learn complete details of creating management solutions.
Include Automation runbooks in your management solution.
Best practices for creating management solutions in
Azure (Preview)
10/17/2019 • 2 minutes to read • Edit Online
NOTE
This is preliminary documentation for creating management solutions in Azure which are currently in preview. Any schema
described below is subject to change.
This article provides best practices for creating a management solution file in Azure. This information will be
updated as additional best practices are identified.
Data sources
Data sources can be configured with a Resource Manager template, but they should not be included in a
solution file. The reason is that configuring data sources is not currently idempotent meaning that your
solution could overwrite existing configuration in the user's workspace.
For example, your solution may require Warning and Error events from the Application event log. If you
specify this as a data source in your solution, you risk removing Information events if the user had this
configured in their workspace. If you included all events, then you may be collecting excessive Information
events in the user's workspace.
If your solution requires data from one of the standard data sources, then you should define this as a
prerequisite. State in documentation that the customer must configure the data source on their own.
Add a Data Flow Verification message to any views in your solution to instruct the user on data sources that
need to be configured for required data to be collected. This message is displayed on the tile of the view
when required data is not found.
Runbooks
Add an Automation schedule for each runbook in your solution that needs to run on a schedule.
Include the IngestionAPI module in your solution to be used by runbooks writing data to the Log Analytics
repository. Configure the solution to reference this resource so that it remains if the solution is removed. This
allows multiple solutions to share the module.
Use Automation variables to provide values to the solution that users may want to change later. Even if the
solution is configured to contain the variable, it's value can still be changed.
Views
All solutions should include a single view that is displayed in the user's portal. The view can contain multiple
visualization parts to illustrate different sets of data.
Add a Data Flow Verification message to any views in your solution to instruct the user on data sources that
need to be configured for required data to be collected.
Configure the solution to contain the view so that it's removed if the solution is removed.
Alerts
Define the recipients list as a parameter in the solution file so the user can define them when they install the
solution.
Configure the solution to reference alert rules so that user's can change their configuration. They may want to
make changes such as modifying the recipient list, changing the threshold of the alert, or disabling the alert rule.
Next steps
Walk through the basic process of designing and building a management solution.
Learn how to create a solution file.
Add saved searches and alerts to your management solution.
Add views to your management solution.
Add Automation runbooks and other resources to your management solution.
Connect Operations Manager to Azure Monitor
12/13/2019 • 17 minutes to read • Edit Online
NOTE
This article was recently updated to use the term Azure Monitor logs instead of Log Analytics. Log data is still stored
in a Log Analytics workspace and is still collected and analyzed by the same Log Analytics service. We are updating
the terminology to better reflect the role of logs in Azure Monitor. See Azure Monitor terminology changes for
details.
To maintain your existing investment in System Center Operations Manager and use extended capabilities
with Azure Monitor, you can integrate Operations Manager with your Log Analytics workspace. This allows
you to leverage the opportunities of logs in Azure Monitor while continuing to use Operations Manager to:
Monitor the health of your IT services with Operations Manager
Maintain integration with your ITSM solutions supporting incident and problem management
Manage the lifecycle of agents deployed to on-premises and public cloud IaaS virtual machines that you
monitor with Operations Manager
Integrating with System Center Operations Manager adds value to your service operations strategy by
using the speed and efficiency of Azure Monitor in collecting, storing, and analyzing log data from
Operations Manager. Azure Monitor log queries help correlate and work towards identifying the faults of
problems and surfacing recurrences in support of your existing problem management process. The
flexibility of the query engine to examine performance, event and alert data, with rich dashboards and
reporting capabilities to expose this data in meaningful ways, demonstrates the strength Azure Monitor
brings in complimenting Operations Manager.
The agents reporting to the Operations Manager management group collect data from your servers based
on the Log Analytics data sources and solutions you have enabled in your workspace. Depending on the
solutions enabled, their data are either sent directly from an Operations Manager management server to
the service, or because of the volume of data collected on the agent-managed system, are sent directly from
the agent to a Log Analytics workspace. The management server forwards the data directly to the service; it
is never written to the operational or data warehouse database. When a management server loses
connectivity with Azure Monitor, it caches the data locally until communication is re-established. If the
management server is offline due to planned maintenance or unplanned outage, another management
server in the management group resumes connectivity with Azure Monitor.
The following diagram shows the connection between the management servers and agents in a System
Center Operations Manager management group and Azure Monitor, including the direction and ports.
If your IT security policies do not allow computers on your network to connect to the Internet, management
servers can be configured to connect to the Log Analytics gateway to receive configuration information and
send collected data depending on the solutions enabled. For more information and steps on how to
configure your Operations Manager management group to communicate through a Log Analytics gateway
to Azure Monitor, see Connect computers to Azure Monitor using the Log Analytics gateway.
Prerequisites
Before starting, review the following requirements.
Azure Monitor only supports System Center Operations Manager 2016 or later, Operations
Manager 2012 SP1 UR6 or later, and Operations Manager 2012 R2 UR2 or later. Proxy support was
added in Operations Manager 2012 SP1 UR7 and Operations Manager 2012 R2 UR3.
Integrating System Center Operations Manager 2016 with US Government cloud requires an
updated Advisor management pack included with Update Rollup 2 or later. System Center
Operations Manager 2012 R2 requires an updated Advisor management pack included with Update
Rollup 3 or later.
All Operations Manager agents must meet minimum support requirements. Ensure that agents are
at the minimum update, otherwise Windows agent communication may fail and generate errors in
the Operations Manager event log.
A Log Analytics workspace. For further information, review Log Analytics workspace overview.
You authenticate to Azure with an account that is a member of the Log Analytics Contributor role.
Supported Regions - Only the following Azure regions are supported by System Center Operations
Manager to connect to a Log Analytics workspace:
West Central US
Australia South East
West Europe
East US
South East Asia
Japan East
UK South
Central India
Canada Central
West US 2
NOTE
Recent changes to Azure APIs will prevent customers from being able to successfully configure integration between
their management group and Azure Monitor for the first time. For customers who have already integrated their
management group with the service, you are not impacted unless you need to reconfigure your existing connection.
A new management pack has been released for the following versions of Operations Manager:
For System Center Operations Manager 2019, this management pack is included with the source media and
installed during setup of a new management group or during an upgrade.
Operations Manager 1801 management pack is also applicable for Operations Manager 1807.
For System Center Operations Manager 1801, download the management pack from here.
For System Center 2016 - Operations Manager, download the management pack from here.
For System Center Operations Manager 2012 R2, download the management pack from here.
Network
The information below list the proxy and firewall configuration information required for the Operations
Manager agent, management servers, and Operations console to communicate with Azure Monitor. Traffic
from each component is outbound from your network to Azure Monitor.
Agent
Management server
*.service.opinsights.azure.com 443
service.systemcenteradvisor.com 443
*.service.opinsights.azure.com 443
RESOURCE PORT NUMBER BYPASS HTTP INSPECTION
After completing the following steps to integrate with Azure Monitor, you can remove the configuration by
running netsh winhttp reset proxy and then use the Configure proxy server option in the Operations
console to specify the proxy or Log Analytics gateway server.
1. In the Operations Manager console, select the Administration workspace.
2. Expand the Operations Management Suite node and click Connection.
3. Click the Register to Operations Management Suite link.
4. On the Operations Management Suite Onboarding Wizard: Authentication page, enter the
email address or phone number and password of the administrator account that is associated with
your OMS subscription, and click Sign in.
NOTE
The Operations Management Suite name has been retired.
5. After you are successfully authenticated, on the Operations Management Suite Onboarding
Wizard: Select Workspace page, you are prompted to select your Azure tenant, subscription, and
Log Analytics workspace. If you have more than one workspace, select the workspace you want to
register with the Operations Manager management group from the drop-down list, and then click
Next.
NOTE
Operations Manager only supports one Log Analytics workspace at a time. The connection and the
computers that were registered to Azure Monitor with the previous workspace are removed from Azure
Monitor.
6. On the Operations Management Suite Onboarding Wizard: Summary page, confirm your
settings and if they are correct, click Create.
7. On the Operations Management Suite Onboarding Wizard: Finish page, click Close.
Add agent-managed computers
After configuring integration with your Log Analytics workspace, it only establishes a connection with the
service, no data is collected from the agents reporting to your management group. This won’t happen until
after you configure which specific agent-managed computers collect log data for Azure Monitor. You can
either select the computer objects individually or you can select a group that contains Windows computer
objects. You cannot select a group that contains instances of another class, such as logical disks or SQL
databases.
1. Open the Operations Manager console and select the Administration workspace.
2. Expand the Operations Management Suite node and click Connection.
3. Click the Add a Computer/Group link under the Actions heading on the right-side of the pane.
4. In the Computer Search dialog box, you can search for computers or groups monitored by Operations
Manager. Select computers or groups including the Operations Manager Management Server to
onboard to Azure Monitor, click Add, and then click OK.
You can view computers and groups configured to collect data from the Managed Computers node under
Operations Management Suite in the Administration workspace of the Operations console. From here,
you can add or remove computers and groups as necessary.
Configure proxy settings in the Operations console
Perform the following steps if an internal proxy server is between the management group and Azure
Monitor. These settings are centrally managed from the management group and distributed to agent-
managed systems that are included in the scope to collect log data for Azure Monitor. This is beneficial for
when certain solutions bypass the management server and send data directly to the service.
1. Open the Operations Manager console and select the Administration workspace.
2. Expand Operations Management Suite, and then click Connections.
3. In the OMS Connection view, click Configure Proxy Server.
4. On Operations Management Suite Wizard: Proxy Server page, select Use a proxy server to
access the Operations Management Suite, and then type the URL with the port number, for example,
http://corpproxy:80 and then click Finish.
If your proxy server requires authentication, perform the following steps to configure credentials and
settings that need to propagate to managed computers that reports to Azure Monitor in the management
group.
1. Open the Operations Manager console and select the Administration workspace.
2. Under RunAs Configuration, select Profiles.
3. Open the System Center Advisor Run As Profile Proxy profile.
4. In the Run As Profile Wizard, click Add to use a Run As account. You can create a Run As account or use
an existing account. This account needs to have sufficient permissions to pass through the proxy server.
5. To set the account to manage, choose A selected class, group, or object, click Select… and then click
Group… to open the Group Search box.
6. Search for and then select Microsoft System Center Advisor Monitoring Server Group. Click OK
after selecting the group to close the Group Search box.
7. Click OK to close the Add a Run As account box.
8. Click Save to complete the wizard and save your changes.
After the connection is created and you configure which agents will collect and report log data to Azure
Monitor, the following configuration is applied in the management group, not necessarily in order:
The Run As Account Microsoft.SystemCenter.Advisor.RunAsAccount.Certificate is created. It is
associated with the Run As profile Microsoft System Center Advisor Run As Profile Blob and is
targeting two classes - Collection Server and Operations Manager Management Group.
Two connectors are created. The first is named Microsoft.SystemCenter.Advisor.DataConnector and
is automatically configured with a subscription that forwards all alerts generated from instances of all
classes in the management group to Azure Monitor. The second connector is Advisor Connector,
which is responsible for communicating with Azure Monitor and sharing data.
Agents and groups that you have selected to collect data in the management group is added to the
Microsoft System Center Advisor Monitoring Server Group.
NOTE
The Operations Management Suite Onboarding Wizard: Select Workspace page presents the existing
workspace that is in use.
WARNING
Verify you do not have any custom management packs with the word Advisor or IntelligencePack in the
name before proceeding, otherwise the following steps delete them from the management group.
3. Next type,
Get-SCOMManagementPack -name "*IntelligencePack*" | Remove-SCOMManagementPack -ErrorAction
SilentlyContinue
4. To remove any management packs remaining which have a dependency on other System Center
Advisor management packs, use the script RecursiveRemove.ps1 you downloaded from the TechNet
Script Center earlier.
NOTE
The step to remove the Advisor management packs with PowerShell will not automatically delete the
Microsoft System Center Advisor or Microsoft System Center Advisor Internal management packs. Do not
attempt to delete them.
5. Open the Operations Manager Operations console with an account that is a member of the
Operations Manager Administrators role.
6. Under Administration, select the Management Packs node and in the Look for: box, type
Advisor and verify the following management packs are still imported in your management group:
Microsoft System Center Advisor
Microsoft System Center Advisor Internal
7. In the Azure portal, click the Settings tile.
8. Select Connected Sources.
9. In the table under the System Center Operations Manager section, you should see the name of the
management group you want to remove from the workspace. Under the column Last Data, click
Remove.
NOTE
The Remove link will not be available until after 14 days if there is no activity detected from the connected
management group.
10. A window will appear asking you to confirm that you want to proceed with the removal. Click Yes to
proceed.
To delete the two connectors - Microsoft.SystemCenter.Advisor.DataConnector and Advisor Connector,
save the PowerShell script below to your computer and execute using the following examples:
NOTE
The computer you run this script from, if not a management server, should have the Operations Manager command
shell installed depending on the version of your management group.
param(
[String] $connectorName,
[String] $msName="localhost"
)
$mg = new-object Microsoft.EnterpriseManagement.ManagementGroup $msName
$admin = $mg.GetConnectorFrameworkAdministration()
##########################################################################################
# Configures a connector with the specified name.
##########################################################################################
function New-Connector([String] $name)
{
$connectorForTest = $null;
foreach($connector in $admin.GetMonitoringConnectors())
{
if($connectorName.Name -eq ${name})
{
$connectorForTest = Get-SCOMConnector -id $connector.id
}
}
if ($connectorForTest -eq $null)
{
$testConnector = New-Object Microsoft.EnterpriseManagement.ConnectorFramework.ConnectorInfo
$testConnector.Name = $name
$testConnector.Description = "${name} Description"
$testConnector.DiscoveryDataIsManaged = $false
$connectorForTest = $admin.Setup($testConnector)
$connectorForTest.Initialize();
}
return $connectorForTest
}
##########################################################################################
# Removes a connector with the specified name.
##########################################################################################
function Remove-Connector([String] $name)
{
$testConnector = $null
foreach($connector in $admin.GetMonitoringConnectors())
{
if($connector.Name -eq ${name})
{
$testConnector = Get-SCOMConnector -id $connector.id
}
}
if ($testConnector -ne $null)
{
if($testConnector.Initialized)
{
foreach($alert in $testConnector.GetMonitoringAlerts())
{
$alert.ConnectorId = $null;
$alert.Update("Delete Connector");
}
$testConnector.Uninitialize()
$testConnector.Uninitialize()
}
$connectorIdForTest = $admin.Cleanup($testConnector)
}
}
##########################################################################################
# Delete a connector's Subscription
##########################################################################################
function Delete-Subscription([String] $name)
{
foreach($testconnector in $admin.GetMonitoringConnectors())
{
if($testconnector.Name -eq $name)
{
$connector = Get-SCOMConnector -id $testconnector.id
}
}
$subs = $admin.GetConnectorSubscriptions()
foreach($sub in $subs)
{
if($sub.MonitoringConnectorId -eq $connector.id)
{
$admin.DeleteConnectorSubscription($admin.GetConnectorSubscription($sub.Id))
}
}
}
#New-Connector $connectorName
write-host "Delete-Subscription"
Delete-Subscription $connectorName
write-host "Remove-Connector"
Remove-Connector $connectorName
In the future if you plan on reconnecting your management group to a Log Analytics workspace, you need
to re-import the Microsoft.SystemCenter.Advisor.Resources.\<Language>\.mpb management pack file.
Depending on the version of System Center Operations Manager deployed in your environment, you can
find this file in the following location:
On the source media under the \ManagementPacks folder for System Center 2016 - Operations Manager
and higher.
From the most recent update rollup applied to your management group. For Operations Manager 2012,
the source folder is
%ProgramFiles%\Microsoft System Center 2012\Operations Manager\Server\Management Packs for Update
Rollups
and for 2012 R2, it is located in
System Center 2012 R2\Operations Manager\Server\Management Packs for Update Rollups .
Next steps
To add functionality and gather data, see Add Azure Monitor solutions from the Solutions Gallery.
Optimize your environment with the System Center
Operations Manager Health Check (Preview) solution
12/16/2019 • 15 minutes to read • Edit Online
You can use the System Center Operations Manager Health Check solution to assess the risk and health of your
System Center Operations Manager management group on a regular interval. This article helps you install,
configure, and use the solution so that you can take corrective actions for potential problems.
This solution provides a prioritized list of recommendations specific to your deployed server infrastructure. The
recommendations are categorized across four focus areas, which help you quickly understand the risk and take
corrective action.
The recommendations made are based on the knowledge and experience gained by Microsoft engineers from
thousands of customer visits. Each recommendation provides guidance about why an issue might matter to you
and how to implement the suggested changes.
You can choose focus areas that are most important to your organization and track your progress toward running
a risk free and healthy environment.
After you've added the solution and an assessment is performed, summary information for focus areas is shown
on the System Center Operations Manager Health Check dashboard for your infrastructure. The following
sections describe how to use the information on the System Center Operations Manager Health Check
dashboard, where you can view and then take recommended actions for your Operations Manager environment.
Installing and configuring the solution
The solution works with Microsoft System Center 2012 Operations Manager Service Pack 1, Microsoft System
Center 2012 R2 Operations Manager, Microsoft System Center 2016 Operations Manager, Microsoft System
Center 2016 Operations Manager and Microsoft System Center Operations Manager 1807. A supported version
of .NET Framework 4.6.2 must be installed on each management server.
Use the following information to install and configure the solution.
Before you can use the Health Check solution in Log Analytics, you must have the solution installed. Install
the solution from Azure marketplace.
After adding the solution to the workspace, the System Center Operations Manager Health Check tile
on the dashboard displays an additional configuration required message. Click on the tile and follow the
configuration steps mentioned in the page
NOTE
Configuration of System Center Operations Manager can be done using a script by following the steps mentioned in the
configuration page of the solution in Log Analytics.
To configure the assessment through Operations Manager Operations console, perform the steps below in the
following order:
1. Set the Run As account for System Center Operations Manager Health Check
2. Configure the System Center Operations Manager Health Check rule
-- Create login for the user, comment this line if login is already created.
CREATE LOGIN [UserName] FROM WINDOWS
-- Add database user for all the databases on SQL Server Instance, this is required for connecting to
individual databases.
-- NOTE: This command must be run anytime new databases are added to SQL Server instances.
EXEC sp_msforeachdb N'USE [?]; CREATE USER [UserName] FOR LOGIN [UserName];'
Use msdb
GRANT SELECT To [UserName]
Go
--Replace the Operations Manager database name with the one in your environment
Use [OperationsManager];
GRANT SELECT To [UserName]
GO
--Replace the Operations Manager DatawareHouse database name with the one in your environment
Use [OperationsManagerDW];
GRANT SELECT To [UserName]
GO
--Replace the Operations Manager Audit Collection database name with the one in your environment
Use [OperationsManagerAC];
GRANT SELECT To [UserName]
GO
While still in this window, configure the run frequency using the next procedure.
Configure the run frequency
The assessment is configured to run every 10,080 minutes (or seven days) by default. You can override the value
to a minimum value of 1440 minutes (or one day). The value represents the minimum time gap required between
successive assessment runs. To override the interval, use the steps below.
1. In the Authoring workspace of the Operations Manager console, search for the rule Microsoft System
Center Operations Manager Run Health Check Rule in the Rules section.
2. In the search results, select the one that includes the text Type: Management Server.
3. Right-click the rule and then click Override the Rule > For all objects of class: Management Server.
4. Change the Interval parameter value to your desired interval value. In the example below, the value is set
to 1440 minutes (one day).
If the value is set to less than 1440 minutes, then the rule runs on a one day interval. In this example, the
rule ignores the interval value and runs at a frequency of one day.
7. You can take corrective actions suggested in Suggested Actions. When the item has been addressed, later
assessments will record that recommended actions were taken and your compliance score will increase.
Corrected items appear as Passed Objects.
Ignore recommendations
If you have recommendations that you want to ignore, you can create a text file that Log Analytics uses to prevent
recommendations from appearing in your assessment results.
To identify recommendations that you want to ignore
1. In the Azure portal on the Log Analytics workspace page for your selected workspace, click the Log Search
menu item.
2. Use the following query to list recommendations that have failed for computers in your environment.
NOTE
If your workspace has been upgraded to the new Log Analytics query language, then the above query would change
to the following.
SCOMAssessmentRecommendationRecommendation | where RecommendationResult == "Failed" | sort by
Computer asc | project Computer, RecommendationId, Recommendation
3. Choose recommendations that you want to ignore. You'll use the values for RecommendationId in the next
procedure.
To create and use an IgnoreRecommendations.txt text file
1. Create a file named IgnoreRecommendations.txt.
2. Paste or type each RecommendationId for each recommendation that you want Log Analytics to ignore on a
separate line and then save and close the file.
3. Put the file in the following folder on each computer where you want Log Analytics to ignore
recommendations.
4. On the Operations Manager management server - SystemDrive:\Program Files\Microsoft System Center 2012
R2\Operations Manager\Server.
To verify that recommendations are ignored
1. After the next scheduled assessment runs, by default every seven days, the specified recommendations are
marked Ignored and will not appear on the health check dashboard.
2. You can use the following Log Search queries to list all the ignored recommendations.
Type=SCOMAssessmentRecommendationRecommendationResult=Ignored | select Computer, RecommendationId,
Recommendation | sort Computer
NOTE
If your workspace has been upgraded to the new Log Analytics query language, then the above query would change
to the following.
SCOMAssessmentRecommendationRecommendation | where RecommendationResult == "Ignore" | sort by
Computer asc | project Computer, RecommendationId, Recommendation
3. If you decide later that you want to see ignored recommendations, remove any IgnoreRecommendations.txt
files, or you can remove RecommendationIDs from them.
There is a Failed to connect to the SQL Instance (….). message in prerequisite failures. What is the issue?
AdvisorAssessment.exe, the process that collects data, runs under the HealthService process on the management
server. As part of the health check, the process attempts to connect to the SQL Server where the Operations
Manager database is present. This error can occur when firewall rules block the connection to the SQL Server
instance.
What type of data is collected? The following types of data are collected: - WMI data - Registry data - EventLog
data - Operations Manager data through Windows PowerShell, SQL Queries and File information collector.
Why do I have to configure a Run As Account? With Operations Manager, various SQL queries are run. In order
for them to run, you must use a Run As Account with necessary permissions. In addition, local administrator
credentials are required to query WMI.
Why display only the top 10 recommendations? Instead of giving you an exhaustive, overwhelming list of tasks,
we recommend that you focus on addressing the prioritized recommendations first. After you address them,
additional recommendations will become available. If you prefer to see the detailed list, you can view all
recommendations using Log Search.
Is there a way to ignore a recommendation? Yes, see the Ignore recommendations.
Next steps
Search logs to learn how to analyze detailed System Center Operations Manager Health Check data and
recommendations.
Connect Configuration Manager to Azure Monitor
1/22/2020 • 6 minutes to read • Edit Online
You can connect your Microsoft Endpoint Configuration Manager environment to Azure Monitor to sync device
collection data and reference these collections in Azure Monitor and Azure Automation.
Prerequisites
Azure Monitor supports Configuration Manager current branch, version 1606 and higher.
NOTE
The feature to connect Configuration Manager with a Log Analytics workspace is optional and not enabled by default. You
must enable this feature before using it. For more information, see Enable optional features from updates.
Configuration overview
The following steps summarize the steps to configure Configuration Manager integration with Azure Monitor.
1. In Azure Active Directory, register Configuration Manager as a Web Application and/or Web API app, and
ensure that you have the client ID and client secret key from the registration from Azure Active Directory.
See Use portal to create Active Directory application and service principal that can access resources for
detailed information about how to accomplish this step.
2. In Azure Active Directory, grant Configuration Manager (the registered web app) with permission to access
Azure Monitor.
3. In Configuration Manager, add a connection using the Azure Services wizard.
4. Download and install the Log Analytics agent for Windows on the computer running the Configuration
Manager service connection point site system role. The agent sends Configuration Manager data to the
Log Analytics workspace in Azure Monitor.
5. In Azure Monitor, import collections from Configuration Manager as computer groups.
6. In Azure Monitor, view data from Configuration Manager as computer groups.
NOTE
You must specify permissions in the Log Analytics workspace for Configuration Manager. Otherwise, you receive an error
message when you use the configuration wizard in Configuration Manager.
1. In the Azure portal, click All services found in the upper left-hand corner. In the list of resources, type Log
Analytics. As you begin typing, the list filters based on your input. Select Log Analytics.
2. In your list of Log Analytics workspaces, select the workspace to modify.
3. From the left pane, select Access control (IAM ).
4. In the Access control (IAM ) page, click Add role assignment and the Add role assignment pane
appears.
5. In the Add role assignment pane, under the Role drop-down list select the Contributor role.
6. Under the Assign access to drop-down list, select the Configuration Manager application created in AD
earlier, and then click OK.
NOTE
You must connect the top-tier site in your hierarchy to Azure Monitor. If you connect a standalone primary site to Azure
Monitor and then add a central administration site to your environment, you have to delete and recreate the connection
within the new hierarchy.
1. In the Administration workspace of Configuration Manager, select Clouds Services and then select
Azure Services.
2. Right-click Azure Services and then select Configure Azure Services. The Configure Azure Services
page appears.
3. On the General screen, confirm that you have done the following actions and that you have details for
each item, then select Next.
4. On the Azure Services page of the Azure Services Wizard:
a. Specify a Name for the object in Configuration Manager.
b. Specify an optional Description to help you identify the service.
c. Select the Azure service OMS Connector.
NOTE
OMS is now referred to as Log Analytics which is a feature of Azure Monitor.
5. Select Next to continue to the Azure app properties page of the Azure Services Wizard.
6. On the App page of the Azure Services Wizard, first select the Azure environment from the list and then
click Import.
7. On the Import Apps page, specify the following information:
a. Specify the Azure AD Tenant Name for the app.
b. Specify for Azure AD Tenant ID the Azure AD tenant. You can find this information on the Azure
Active Directory Properties page.
c. Specify for Application Name the application name.
d. Specify for Client ID, the Application ID of the created Azure AD app created earlier.
e. Specify for Secret key, the Client secret key of the created Azure AD app.
f. Specify for Secret Key Expiry, the expiration date of your key.
g. Specify for App ID URI, the App ID URI of the created Azure AD app created earlier.
h. Select Verify and to the right the results should show Successfully verified!.
8. On the Configuration page, review the information to verify the Azure subscriptions, Azure resource
group, and Operations Management Suite workspace fields are pre-populated indicating the Azure
AD application has sufficient permissions in the resource group. If the fields are empty, it indicates your
application does not have the rights required. Select the device collections to collect and forward to the
workspace and then select Add.
9. Review the options on the Confirm the settings page, and select Next to begin creating and configuring
the connection.
10. When configuration is finished, the Completion page appears. Select Close.
After you have linked Configuration Manager to Azure Monitor, you can add or remove collections, and view the
properties of the connection.
Import collections
After you've added a Log Analytics connection to Configuration Manager and installed the agent on the computer
running the Configuration Manager service connection point site system role, the next step is to import
collections from Configuration Manager in Azure Monitor as computer groups.
After you have completed initial configuration to import device collections from your hierarchy, the collection
information is retrieved every 3 hours to keep the membership current. You can choose to disable this at any time.
1. In the Azure portal, click All services found in the upper left-hand corner. In the list of resources, type Log
Analytics. As you begin typing, the list filters based on your input. Select Log Analytics workspaces.
2. In your list of Log Analytics workspaces, select the workspace Configuration Manager is registered with.
3. Select Advanced settings.
4. Select Computer Groups and then select SCCM.
5. Select Import Configuration Manager collection memberships and then click Save.
View data from Configuration Manager
After you've added a Log Analytics connection to Configuration Manager and installed the agent on the computer
running the Configuration Manager service connection point site system role, data from the agent is sent to the
Log Analytics workspace in Azure Monitor. In Azure Monitor, your Configuration Manager collections appear as
computer groups. You can view the groups from the Configuration Manager page under Settings\Computer
Groups.
After the collections are imported, you can see how many computers with collection memberships have been
detected. You can also see the number of collections that have been imported.
When you click either one, log query editor opens displaying either all of the imported groups or all computers
that belong to each group. Using Log Search, you can perform further in-depth analysis the collection
membership data.
Next steps
Use Log Search to view detailed information about your Configuration Manager data.
Azure Monitor service limits
12/13/2019 • 5 minutes to read • Edit Online
Alerts
RESOURCE DEFAULT LIMIT MAXIMUM LIMIT
Metric alerts (classic) 100 active alert rules per subscription. Call support.
Metric alerts 1000 active alert rules per subscription Call support.
in Azure public, Azure China 21Vianet
and Azure Government clouds.
Activity log alerts 100 active alert rules per subscription. Same as default.
Action groups
RESOURCE DEFAULT LIMIT MAXIMUM LIMIT
Azure app push 10 Azure app actions per action group. Call support.
Query language Azure Monitor uses the same Kusto query language as Azure
Data Explorer. See Azure Monitor log query language
differences for KQL language elements not supported in Azure
Monitor.
Azure regions Log queries can experience excessive overhead when data
spans Log Analytics workspaces in multiple Azure regions. See
Query limits for details.
Cross resource queries Maximum number of Application Insights resources and Log
Analytics workspaces in a single query limited to 100.
Cross-resource query is not supported in View Designer.
Cross-resource query in log alerts is supported in the new
scheduledQueryRules API.
See Cross-resource query limits for details.
Current Per GB pricing tier No limit 30 - 730 days Data retention beyond 31
(introduced April 2018) days is available for
additional charges. Learn
more about Azure Monitor
pricing.
Legacy Per Node (OMS) No limit 30 to 730 days Data retention beyond 31
(introduced April 2016) days is available for
additional charges. Learn
more about Azure Monitor
pricing.
Azure portal
Maximum records returned by a log 10,000 Reduce results using query scope, time
query range, and filters in the query.
Maximum size for a single post 30 MB Split larger volumes into multiple posts.
Maximum size for field values 32 KB Fields longer than 32 KB are truncated.
Search API
Maximum request rate 200 requests per 30 seconds per AAD See Rate limits for details.
user or client IP address
Data export Not currently available Use Azure Function or Logic App to
aggregate and export data.
Operation
|where OperationCategory == "Ingestion"
|where Detail startswith "The rate of data crossed the threshold"
NOTE
Depending on how long you've been using Log Analytics, you might have access to legacy pricing tiers. Learn more about
Log Analytics legacy pricing tiers.
Application Insights
There are some limits on the number of metrics and events per application, that is, per instrumentation key. Limits
depend on the pricing plan that you choose.
Total data per day 100 GB You can reduce data by setting a cap. If
you need more data, you can increase
the limit in the portal, up to 1,000 GB.
For capacities greater than 1,000 GB,
send email to
AIDataCap@microsoft.com.
RESOURCE DEFAULT LIMIT NOTE
Availability multi-step test detailed 90 days This resource provides detailed results
results retention of each step.
For more information, see About pricing and quotas in Application Insights.
Next Steps
Azure Monitor pricing
Monitoring usage and estimated costs in Azure Monitor
Manage usage and costs for Application Insights
Dependency auto-collection
10/20/2019 • 2 minutes to read • Edit Online
Below is the currently supported list of dependency calls that are automatically detected as dependencies without
requiring any additional modification to your application's code. These dependencies are visualized in the
Application Insights Application map and Transaction diagnostics views. If your dependency isn't on the list below,
you can still track it manually with a track dependency call.
.NET
APP FRAMEWORKS VERSIONS
ASP.NET MVC 4+
Communication libraries
Storage clients
ADO.NET 4.5+
Java
APP SERVERS VERSIONS
Tomcat 7, 8
JBoss EAP 6, 7
Jetty 9
App frameworks
Spring 3.0
APP SERVERS VERSIONS
Communication libraries
Storage clients
Oracle 1+†
MySql 1+†
Logging libraries
Logback 1+
Log4j 1.2+
Metrics libraries
JMX 1.0+
NOTE
*Except reactive programing support.
†Requires installation of JVM Agent.
Node.js
COMMUNICATION LIBRARIES VERSIONS
Storage clients
Redis 2.x
Logging libraries
console 0.10+
Bunyan 1.x
JavaScript
COMMUNICATION LIBRARIES VERSIONS
XMLHttpRequest All
Next steps
Set up custom dependency tracking for .NET.
Set up custom dependency tracking for Java.
Write custom dependency telemetry
See data model for Application Insights types and data model.
Check out platforms supported by Application Insights.
Telemetry correlation in Application Insights
12/9/2019 • 12 minutes to read • Edit Online
In the world of microservices, every logical operation requires work to be done in various components of the
service. You can monitor each of these components separately by using Application Insights. Application
Insights supports distributed telemetry correlation, which you use to detect which component is responsible for
failures or performance degradation.
This article explains the data model used by Application Insights to correlate telemetry sent by multiple
components. It covers context-propagation techniques and protocols. It also covers the implementation of
correlation tactics on different languages and platforms.
In a microservices environment, traces from components can go to different storage items. Every component
can have its own instrumentation key in Application Insights. To get telemetry for the logical operation,
Application Insights queries data from every storage item. When the number of storage items is large, you'll
need a hint about where to look next. The Application Insights data model defines two fields to solve this
problem: request.source and dependency.target . The first field identifies the component that initiated the
dependency request. The second field identifies which component returned the response of the dependency call.
Example
Let's look at an example. An application called Stock Prices shows the current market price of a stock by using an
external API called Stock. The Stock Prices application has a page called Stock page that the client web browser
opens by using GET /Home/Stock . The application queries the Stock API by using the HTTP call
GET /api/stock/value .
When the call GET /api/stock/value is made to an external service, you need to know the identity of that server
so you can set the dependency.target field appropriately. When the external service doesn't support monitoring,
target is set to the host name of the service (for example, stock-prices-api.com ). But if the service identifies
itself by returning a predefined HTTP header, target contains the service identity that allows Application
Insights to build a distributed trace by querying telemetry from that service.
Correlation headers
Application Insights is transitioning to W3C Trace-Context, which defines:
traceparent : Carries the globally unique operation ID and unique identifier of the call.
tracestate : Carries system -specific tracing context.
The latest version of the Application Insights SDK supports the Trace-Context protocol, but you might need to
opt in to it. (Backward compatibility with the previous correlation protocol supported by the Application Insights
SDK will be maintained.)
The correlation HTTP protocol, also called Request-Id, is being deprecated. This protocol defines two headers:
Request-Id : Carries the globally unique ID of the call.
Correlation-Context : Carries the name-value pairs collection of the distributed trace properties.
Application Insights also defines the extension for the correlation HTTP protocol. It uses Request-Context name-
value pairs to propagate the collection of properties used by the immediate caller or callee. The Application
Insights SDK uses this header to set the dependency.target and request.source fields.
Enable W3C distributed tracing support for classic ASP.NET apps
NOTE
Starting with Microsoft.ApplicationInsights.Web and Microsoft.ApplicationInsights.DependencyCollector , no
configuration is needed.
W3C Trace-Context support is implemented in a backward-compatible way. Correlation is expected to work with
applications that are instrumented with previous versions of the SDK (without W3C support).
If you want to keep using the legacy Request-Id protocol, you can disable Trace-Context by using this
configuration:
Activity.DefaultIdFormat = ActivityIdFormat.Hierarchical;
Activity.ForceDefaultIdFormat = true;
If you run an older version of the SDK, we recommend that you update it or apply the following configuration to
enable Trace-Context. This feature is available in the Microsoft.ApplicationInsights.Web and
Microsoft.ApplicationInsights.DependencyCollector packages, starting with version 2.8.0 -beta1. It's disabled by
default. To enable it, make these changes to ApplicationInsights.config :
Under RequestTrackingTelemetryModule , add the EnableW3CHeadersExtraction element and set its value to
true .
Under DependencyTrackingTelemetryModule , add the EnableW3CHeadersInjection element and set its value to
true .
Add W3COperationCorrelationTelemetryInitializer under TelemetryInitializers . It will look similar to this
example:
<TelemetryInitializers>
<Add Type="Microsoft.ApplicationInsights.Extensibility.W3C.W3COperationCorrelationTelemetryInitializer,
Microsoft.ApplicationInsights"/>
...
</TelemetryInitializers>
NOTE
Starting with Microsoft.ApplicationInsights.AspNetCore version 2.8.0, no configuration is needed.
W3C Trace-Context support is implemented in a backward-compatible way. Correlation is expected to work with
applications that are instrumented with previous versions of the SDK (without W3C support).
If you want to keep using the legacy Request-Id protocol, you can disable Trace-Context by using this
configuration:
Activity.DefaultIdFormat = ActivityIdFormat.Hierarchical;
Activity.ForceDefaultIdFormat = true;
If you run an older version of the SDK, we recommend that you update it or apply the following configuration to
enable Trace-Context.
This feature is in version 2.5.0-beta1 and in
Microsoft.ApplicationInsights.AspNetCore
Microsoft.ApplicationInsights.DependencyCollector version 2.8.0 -beta1. It's disabled by default. To enable it, set
ApplicationInsightsServiceOptions.RequestCollectionOptions.EnableW3CDistributedTracing to true :
<Add
type="com.microsoft.applicationinsights.web.extensibility.modules.WebRequestTrackingTelemetryMo
dule>
<Param name = "W3CEnabled" value ="true"/>
<Param name ="enableW3CBackCompat" value = "true" />
</Add>
<Instrumentation>
<BuiltIn enabled="true">
<HTTP enabled="true" W3C="true" enableW3CBackCompat="true"/>
</BuiltIn>
</Instrumentation>
NOTE
Backward compatibility mode is enabled by default, and the enableW3CBackCompat parameter is optional. Use it
only when you want to turn backward compatibility off.
Ideally, you would turn this off when all your services have been updated to newer versions of SDKs that support
the W3C protocol. We highly recommend that you move to these newer SDKs as soon as possible.
IMPORTANT
Make sure the incoming and outgoing configurations are exactly the same.
Operation_Id TraceId
app = Flask(__name__)
middleware = FlaskMiddleware(
app,
exporter=AzureExporter(),
sampler=ProbabilitySampler(rate=1.0),
)
@app.route('/')
def hello():
return 'Hello World!'
if __name__ == '__main__':
app.run(host='localhost', port=8080, threaded=True)
This code runs a sample Flask application on your local machine, listening to port 8080 . To correlate trace
context, you send a request to the endpoint. In this example, you can use a curl command:
By looking at the Trace-Context header format, you can derive the following information:
version : 00
trace-id : 4bf92f3577b34da6a3ce929d0e0e4736
parent-id/span-id : 00f067aa0ba902b7
trace-flags : 01
If you look at the request entry that was sent to Azure Monitor, you can see fields populated with the trace
header information. You can find this data under Logs (Analytics) in the Azure Monitor Application Insights
resource.
The id field is in the format <trace-id>.<span-id> , where the trace-id is taken from the trace header that was
passed in the request and the span-id is a generated 8-byte array for this span.
The operation_ParentId field is in the format <trace-id>.<parent-id> , where both the trace-id and the
parent-id are taken from the trace header that was passed in the request.
Log correlation
OpenCensus Python enables you to correlate logs by adding a trace ID, a span ID, and a sampling flag to log
records. You add these attributes by installing OpenCensus logging integration. The following attributes will be
added to Python LogRecord objects: traceId , spanId , and traceSampled . Note that this takes effect only for
loggers that are created after the integration.
Here's a sample application that demonstrates this:
import logging
config_integration.trace_integrations(['logging'])
logging.basicConfig(format='%(asctime)s traceId=%(traceId)s spanId=%(spanId)s %(message)s')
tracer = Tracer(sampler=AlwaysOnSampler())
logger = logging.getLogger(__name__)
logger.warning('Before the span')
with tracer.span(name='hello'):
logger.warning('In the span')
logger.warning('After the span')
When this code runs, the following prints in the console:
Notice that there's a spanId present for the log message that's within the span. This is the same spanId that
belongs to the span named hello .
You can export the log data by using AzureLogHandler . For more information, see this article.
NOTE
Only calls made via Apache HttpClient are supported for the correlation feature. Both Spring RestTemplate and Feign can
be used with Apache HttpClient under the hood.
Currently, automatic context propagation across messaging technologies (like Kafka, RabbitMQ, and Azure
Service Bus) isn't supported. It is possible to code such scenarios manually by using the trackDependency and
trackRequest methods. In these methods, a dependency telemetry represents a message being enqueued by a
producer. The request represents a message being processed by a consumer. In this case, both operation_id and
operation_parentId should be propagated in the message's properties.
Role name
You might want to customize the way component names are displayed in the Application Map. To do so, you can
manually set the cloud_RoleName by taking one of the following actions:
With Application Insights Java SDK 2.5.0 and later, you can specify the cloud_RoleName by adding
<RoleName> to your ApplicationInsights.xml file:
If you use Spring Boot with the Application Insights Spring Boot Starter, you just need to set your custom
name for the application in the application.properties file:
spring.application.name=<name-of-app>
The Spring Boot Starter automatically assigns cloudRoleName to the value you enter for the
spring.application.name property.
Next steps
Write custom telemetry.
For advanced correlation scenarios in ASP.NET Core and ASP.NET, see Track custom operations.
Learn more about setting cloud_RoleName for other SDKs.
Onboard all components of your microservice on Application Insights. Check out the supported platforms.
See the data model for Application Insights types.
Learn how to extend and filter telemetry.
Review the Application Insights config reference.
Application Insights NuGet packages
12/23/2019 • 4 minutes to read • Edit Online
Below is the current list of stable release NuGet Packages for Application Insights.
Listeners/collectors/appenders
PACKAGE NAME STABLE VERSION DESCRIPTION DOWNLOAD
Service Fabric
PACKAGE NAME STABLE VERSION DESCRIPTION DOWNLOAD
Status monitor
PACKAGE NAME STABLE VERSION DESCRIPTION DOWNLOAD
These packages make up part of the core functionality of the runtime monitoring in Status Monitor. You don't need
to download these packages directly, just use the Status Monitor installer. If you want to understand more about
how these packages work under the hood this blog post from one of our developers is a good start.
Additional packages
PACKAGE NAME STABLE VERSION DESCRIPTION DOWNLOAD
Next steps
Monitor ASP.NET Core.
Profile ASP.NET Core Azure Linux web apps.
Debug ASP.NET snapshots.
Supported languages
10/29/2019 • 2 minutes to read • Edit Online
C#|VB (.NET)
Java
JavaScript
Node.JS
Python (preview )
Logging frameworks
ILogger
Log4Net, NLog, or System.Diagnostics.Trace
Java, Log4J, or Logback
LogStash plugin
Azure Monitor
Unsupported SDKs
We're aware that several other community-supported SDKs exist. However, Azure Monitor only provides
support when using the supported SDKs listed on this page. We’re constantly assessing opportunities to
expand our support for other languages, so follow our GitHub Announcements page to receive the latest
SDK news.
Application Insights overriding default endpoints
12/16/2019 • 3 minutes to read • Edit Online
To send data from Application Insights to certain regions, you'll need to override the default endpoint addresses. Each SDK requires
slightly different modifications, all of which are described in this article. These changes require adjusting the sample code and replacing
the placeholder values for QuickPulse_Endpoint_Address , TelemetryChannel_Endpoint_Address , and Profile_Query_Endpoint_address with
the actual endpoint addresses for your specific region. The end of this article contains links to the endpoint addresses for regions where
this configuration is required.
NOTE
The applicationinsights.config file is automatically overwritten anytime a SDK upgrade is performed. After performing an SDK upgrade be sure to re-
enter the region specific endpoint values.
<ApplicationInsights>
...
<TelemetryModules>
<Add Type="Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse.QuickPulseTelemetryModule,
Microsoft.AI.PerfCounterCollector">
<QuickPulseServiceEndpoint>Custom_QuickPulse_Endpoint_Address</QuickPulseServiceEndpoint>
</Add>
</TelemetryModules>
...
<TelemetryChannel>
<EndpointAddress>TelemetryChannel_Endpoint_Address</EndpointAddress>
</TelemetryChannel>
...
<ApplicationIdProvider
Type="Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId.ApplicationInsightsApplicationIdProvider,
Microsoft.ApplicationInsights">
<ProfileQueryEndpoint>Profile_Query_Endpoint_address</ProfileQueryEndpoint>
</ApplicationIdProvider>
...
</ApplicationInsights>
ASP.NET Core
Modify the appsettings.json file in your project as follows to adjust the main endpoint:
"ApplicationInsights": {
"InstrumentationKey": "instrumentationkey",
"TelemetryChannel": {
"EndpointAddress": "TelemetryChannel_Endpoint_Address"
}
}
The values for Live Metrics and the Profile Query Endpoint can only be set via code. To override the default values for all endpoint
values via code, make the following changes in the ConfigureServices method of the Startup.cs file:
using Microsoft.ApplicationInsights.Extensibility.Implementation.ApplicationId;
using Microsoft.ApplicationInsights.Extensibility.PerfCounterCollector.QuickPulse; //Place at top of Startup.cs file
services.ConfigureTelemetryModule<QuickPulseTelemetryModule>((module, o) =>
module.QuickPulseServiceEndpoint="QuickPulse_Endpoint_Address");
[assembly: WebJobsStartup(typeof(Example.Startup))]
namespace Example
{
class Startup : FunctionsStartup
{
public override void Configure(IWebJobsBuilder builder)
{
var quickPulseFactory = builder.Services.FirstOrDefault(sd => sd.ServiceType == typeof(ITelemetryModule) &&
sd.ImplementationType == typeof(QuickPulseTelemetryModule));
if (quickPulseFactory != null)
{
builder.Services.Remove(quickPulseFactory);
}
Java
Modify the applicationinsights.xml file to change the default endpoint address.
<?xml version="1.0" encoding="utf-8"?>
<ApplicationInsights xmlns="http://schemas.microsoft.com/ApplicationInsights/2013/Settings">
<InstrumentationKey>ffffeeee-dddd-cccc-bbbb-aaaa99998888</InstrumentationKey>
<TelemetryModules>
<Add type="com.microsoft.applicationinsights.web.extensibility.modules.WebRequestTrackingTelemetryModule"/>
<Add type="com.microsoft.applicationinsights.web.extensibility.modules.WebSessionTrackingTelemetryModule"/>
<Add type="com.microsoft.applicationinsights.web.extensibility.modules.WebUserTrackingTelemetryModule"/>
</TelemetryModules>
<TelemetryInitializers>
<Add type="com.microsoft.applicationinsights.web.extensibility.initializers.WebOperationIdTelemetryInitializer"/>
<Add type="com.microsoft.applicationinsights.web.extensibility.initializers.WebOperationNameTelemetryInitializer"/>
<Add type="com.microsoft.applicationinsights.web.extensibility.initializers.WebSessionTelemetryInitializer"/>
<Add type="com.microsoft.applicationinsights.web.extensibility.initializers.WebUserTelemetryInitializer"/>
<Add type="com.microsoft.applicationinsights.web.extensibility.initializers.WebUserAgentTelemetryInitializer"/>
</TelemetryInitializers>
<!--Add the following Channel value to modify the Endpoint address-->
<Channel type="com.microsoft.applicationinsights.channel.concrete.inprocess.InProcessTelemetryChannel">
<EndpointAddress>TelemetryChannel_Endpoint_Address</EndpointAddress>
</Channel>
</ApplicationInsights>
Spring Boot
Modify the application.properties file and add:
azure.application-insights.channel.in-process.endpoint-address= TelemetryChannel_Endpoint_Address
Node.js
JavaScript
<script type="text/javascript">
var sdkInstance="appInsightsSDK";window[sdkInstance]="appInsights";var
aiName=window[sdkInstance],aisdk=window[aiName]||function(e){function n(e){t[e]=function(){var n=arguments;t.queue.push(function()
{t[e].apply(t,n)})}}var t={config:e};t.initialize=!0;var i=document,a=window;setTimeout(function(){var
n=i.createElement("script");n.src=e.url||"https://az416426.vo.msecnd.net/scripts/b/ai.2.min.js",i.getElementsByTagName("script")
[0].parentNode.appendChild(n)});try{t.cookie=i.cookie}catch(e){}t.queue=[],t.version=2;for(var r=
["Event","PageView","Exception","Trace","DependencyData","Metric","PageViewPerformance"];r.length;)n("track"+r.pop());n("startTrack
Page"),n("stopTrackPage");var
s="Track"+r[0];if(n("start"+s),n("stop"+s),n("setAuthenticatedUserContext"),n("clearAuthenticatedUserContext"),n("flush"),!
(!0===e.disableExceptionTracking||e.extensionConfig&&e.extensionConfig.ApplicationInsightsAnalytics&&!0===e.extensionConfig.Applica
tionInsightsAnalytics.disableExceptionTracking)){n("_"+(r="onerror"));var o=a[r];a[r]=function(e,n,i,a,s){var
c=o&&o(e,n,i,a,s);return!0!==c&&t["_"+r]
({message:e,url:n,lineNumber:i,columnNumber:a,error:s}),c},e.autoExceptionInstrumented=!0}return t}(
{
instrumentationKey:"INSTRUMENTATION_KEY",
endpointUrl: "TelemetryChannel_Endpoint_Address"
}
);window[aiName]=aisdk,aisdk.queue&&0===aisdk.queue.length&&aisdk.trackPageView({});
</script>
If you currently use the Application Insights REST API which is normally accessed via `api.applicationinsights.io' you will need to use an
endpoint that is local to your region:
NOTE
Codeless agent/extension based monitoring for Azure App Services is currently not supported in these regions. As soon as this functionality
becomes available this article will be updated.
Next steps
To learn more about the custom modifications for Azure Government, consult the detailed guidance for Azure monitoring and
management.
To learn more about Azure China, consult the Azure China Playbook.
Application Insights telemetry data model
10/15/2019 • 2 minutes to read • Edit Online
Azure Application Insights sends telemetry from your web application to the Azure portal, so that you can
analyze the performance and usage of your application. The telemetry model is standardized so that it is
possible to create platform and language-independent monitoring.
Data collected by Application Insights models this typical application execution pattern:
The following types of telemetry are used to monitor the execution of your app. The following three types are
typically automatically collected by the Application Insights SDK from the web application framework:
Request - Generated to log a request received by your app. For example, the Application Insights web
SDK automatically generates a Request telemetry item for each HTTP request that your web app
receives.
An Operation is the threads of execution that processes a request. You can also write code to monitor
other types of operation, such as a "wake up" in a web job or function that periodically processes data.
Each operation has an ID. This ID that can be used to group all telemetry generated while your app is
processing the request. Each operation either succeeds or fails, and has a duration of time.
Exception - Typically represents an exception that causes an operation to fail.
Dependency - Represents a call from your app to an external service or storage such as a REST API or
SQL. In ASP.NET, dependency calls to SQL are defined by System.Data . Calls to HTTP endpoints are
defined by System.Net .
Application Insights provides three additional data types for custom telemetry:
Trace - used either directly, or through an adapter to implement diagnostics logging using an
instrumentation framework that is familiar to you, such as Log4Net or System.Diagnostics .
Event - typically used to capture user interaction with your service, to analyze usage patterns.
Metric - used to report periodic scalar measurements.
Every telemetry item can define the context information like application version or user session id. Context is a
set of strongly typed fields that unblocks certain scenarios. When application version is properly initialized,
Application Insights can detect new patterns in application behavior correlated with redeployment. Session id
can be used to calculate the outage or an issue impact on users. Calculating distinct count of session id values
for certain failed dependency, error trace or critical exception gives a good understanding of an impact.
Application Insights telemetry model defines a way to correlate telemetry to the operation of which it’s a part.
For example, a request can make a SQL Database calls and recorded diagnostics info. You can set the
correlation context for those telemetry items that tie it back to the request telemetry.
Schema improvements
Application Insights data model is a simple and basic yet powerful way to model your application telemetry.
We strive to keep the model simple and slim to support essential scenarios and allow to extend the schema for
advanced use.
To report data model or schema problems and suggestions use GitHub ApplicationInsights-Home repository.
Next steps
Write custom telemetry
Learn how to extend and filter telemetry.
Use sampling to minimize amount of telemetry based on data model.
Check out platforms supported by Application Insights.
Request telemetry: Application Insights data model
12/8/2019 • 3 minutes to read • Edit Online
A request telemetry item (in Application Insights) represents the logical sequence of execution triggered by an
external request to your application. Every request execution is identified by unique ID and url containing all
the execution parameters. You can group requests by logical name and define the source of this request. Code
execution can result in success or fail and has a certain duration . Both success and failure executions may be
grouped further by resultCode . Start time for the request telemetry defined on the envelope level.
Request telemetry supports the standard extensibility model using custom properties and measurements .
Name
Name of the request represents code path taken to process the request. Low cardinality value to allow better
grouping of requests. For HTTP requests it represents the HTTP method and URL path template like
GET /values/{id} without the actual id value.
Application Insights web SDK sends request name "as is" with regards to letter case. Grouping on UI is case-
sensitive so GET /Home/Index is counted separately from GET /home/INDEX even though often they result in the
same controller and action execution. The reason for that is that urls in general are case-sensitive. You may want
to see if all 404 happened for the urls typed in uppercase. You can read more on request name collection by
ASP.NET Web SDK in the blog post.
Max length: 1024 characters
ID
Identifier of a request call instance. Used for correlation between request and other telemetry items. ID should be
globally unique. For more information, see correlation page.
Max length: 128 characters
Url
Request URL with all query string parameters.
Max length: 2048 characters
Source
Source of the request. Examples are the instrumentation key of the caller or the ip address of the caller. For more
information, see correlation page.
Max length: 1024 characters
Duration
Request duration in format: DD.HH:MM:SS.MMMMMM . Must be positive and less than 1000 days. This field is required
as request telemetry represents the operation with the beginning and the end.
Response code
Result of a request execution. HTTP status code for HTTP requests. It may be HRESULT value or exception type for
other request types.
Max length: 1024 characters
Success
Indication of successful or unsuccessful call. This field is required. When not set explicitly to false - a request is
considered to be successful. Set this value to false if operation was interrupted by exception or returned error
result code.
For the web applications, Application Insights define a request as successful when the response code is less than
400 or equal to 401 . However there are cases when this default mapping does not match the semantic of the
application. Response code 404 may indicate "no records", which can be part of regular flow. It also may indicate
a broken link. For the broken links, you can even implement more advanced logic. You can mark broken links as
failures only when those links are located on the same site by analyzing url referrer. Or mark them as failures
when accessed from the company's mobile application. Similarly 301 and 302 indicates failure when accessed
from the client that doesn't support redirect.
Partially accepted content 206 may indicate a failure of an overall request. For instance, Application Insights
endpoint receives a batch of telemetry items as a single request. It returns 206 when some items in the batch
were not processed successfully. Increasing rate of 206 indicates a problem that needs to be investigated. Similar
logic applies to 207 Multi-Status where the success may be the worst of separate response codes.
You can read more on request result code and status code in the blog post.
Custom properties
Name-value collection of custom properties. This collection is used to extend standard telemetry with the custom
dimensions. Examples are deployment slot that produced telemetry or telemetry-item specific property like order
number.
Max key length: 150 Max value length: 8192
Custom measurements
Collection of custom measurements. Use this collection to report named measurement associated with the
telemetry item. Typical use cases are:
the size of Dependency Telemetry payload
the number of queue items processed by Request Telemetry
time that customer took to complete the step in wizard step completion Event Telemetry.
You can query custom measurements in Application Analytics:
customEvents
| where customMeasurements != ""
| summarize avg(todouble(customMeasurements["Completion Time"]) * itemCount)
NOTE
Custom measurements are associated with the telemetry item they belong to. They are subject to sampling with the
telemetry item containing those measurements. To track a measurement that has a value independent from other telemetry
types, use Metric telemetry.
Max key length: 150
Next steps
Write custom request telemetry
See data model for Application Insights types and data model.
Learn how to configure ASP.NET Core application with Application Insights.
Check out platforms supported by Application Insights.
Dependency telemetry: Application Insights data
model
12/5/2019 • 2 minutes to read • Edit Online
Dependency Telemetry (in Application Insights) represents an interaction of the monitored component with a
remote component such as SQL or an HTTP endpoint.
Name
Name of the command initiated with this dependency call. Low cardinality value. Examples are stored procedure
name and URL path template.
ID
Identifier of a dependency call instance. Used for correlation with the request telemetry item corresponding to
this dependency call. For more information, see correlation page.
Data
Command initiated by this dependency call. Examples are SQL statement and HTTP URL with all query
parameters.
Type
Dependency type name. Low cardinality value for logical grouping of dependencies and interpretation of other
fields like commandName and resultCode. Examples are SQL, Azure table, and HTTP.
Target
Target site of a dependency call. Examples are server name, host address. For more information, see correlation
page.
Duration
Request duration in format: DD.HH:MM:SS.MMMMMM . Must be less than 1000 days.
Result code
Result code of a dependency call. Examples are SQL error code and HTTP status code.
Success
Indication of successful or unsuccessful call.
Custom properties
Name-value collection of custom properties. This collection is used to extend standard telemetry with the custom
dimensions. Examples are deployment slot that produced telemetry or telemetry-item specific property like order
number.
Max key length: 150 Max value length: 8192
Custom measurements
Collection of custom measurements. Use this collection to report named measurement associated with the
telemetry item. Typical use cases are:
the size of Dependency Telemetry payload
the number of queue items processed by Request Telemetry
time that customer took to complete the step in wizard step completion Event Telemetry.
You can query custom measurements in Application Analytics:
customEvents
| where customMeasurements != ""
| summarize avg(todouble(customMeasurements["Completion Time"]) * itemCount)
NOTE
Custom measurements are associated with the telemetry item they belong to. They are subject to sampling with the
telemetry item containing those measurements. To track a measurement that has a value independent from other
telemetry types, use Metric telemetry.
Next steps
Set up dependency tracking for .NET.
Set up dependency tracking for Java.
Write custom dependency telemetry
See data model for Application Insights types and data model.
Check out platforms supported by Application Insights.
Exception telemetry: Application Insights data model
12/5/2019 • 2 minutes to read • Edit Online
In Application Insights, an instance of Exception represents a handled or unhandled exception that occurred during
execution of the monitored application.
Problem Id
Identifier of where the exception was thrown in code. Used for exceptions grouping. Typically a combination of
exception type and a function from the call stack.
Max length: 1024 characters
Severity level
Trace severity level. Value can be Verbose , Information , Warning , Error , Critical .
Exception details
(To be extended)
Custom properties
Name-value collection of custom properties. This collection is used to extend standard telemetry with the custom
dimensions. Examples are deployment slot that produced telemetry or telemetry-item specific property like order
number.
Max key length: 150 Max value length: 8192
Custom measurements
Collection of custom measurements. Use this collection to report named measurement associated with the
telemetry item. Typical use cases are:
the size of Dependency Telemetry payload
the number of queue items processed by Request Telemetry
time that customer took to complete the step in wizard step completion Event Telemetry.
You can query custom measurements in Application Analytics:
customEvents
| where customMeasurements != ""
| summarize avg(todouble(customMeasurements["Completion Time"]) * itemCount)
NOTE
Custom measurements are associated with the telemetry item they belong to. They are subject to sampling with the
telemetry item containing those measurements. To track a measurement that has a value independent from other telemetry
types, use Metric telemetry.
Max key length: 150
Next steps
See data model for Application Insights types and data model.
Learn how to diagnose exceptions in your web apps with Application Insights.
Check out platforms supported by Application Insights.
Trace telemetry: Application Insights data model
12/12/2019 • 2 minutes to read • Edit Online
Trace telemetry (in Application Insights) represents printf style trace statements that are text-searched. Log4Net ,
NLog , and other text-based log file entries are translated into instances of this type. The trace does not have
measurements as an extensibility.
Message
Trace message.
Max length: 32768 characters
Severity level
Trace severity level. Value can be Verbose , Information , Warning , Error , Critical .
Custom properties
Name-value collection of custom properties. This collection is used to extend standard telemetry with the custom
dimensions. Examples are deployment slot that produced telemetry or telemetry-item specific property like order
number.
Max key length: 150 Max value length: 8192
Next steps
Explore .NET trace logs in Application Insights.
Explore Java trace logs in Application Insights.
See data model for Application Insights types and data model.
Write custom trace telemetry
Check out platforms supported by Application Insights.
Event telemetry: Application Insights data model
10/20/2019 • 2 minutes to read • Edit Online
You can create event telemetry items (in Application Insights) to represent an event that occurred in your
application. Typically it is a user interaction such as button click or order checkout. It can also be an application life
cycle event like initialization or configuration update.
Semantically, events may or may not be correlated to requests. However, if used properly, event telemetry is more
important than requests or traces. Events represent business telemetry and should be a subject to separate, less
aggressive sampling.
Name
Event name. To allow proper grouping and useful metrics, restrict your application so that it generates a small
number of separate event names. For example, don't use a separate name for each generated instance of an event.
Max length: 512 characters
Custom properties
Name-value collection of custom properties. This collection is used to extend standard telemetry with the custom
dimensions. Examples are deployment slot that produced telemetry or telemetry-item specific property like order
number.
Max key length: 150 Max value length: 8192
Custom measurements
Collection of custom measurements. Use this collection to report named measurement associated with the
telemetry item. Typical use cases are:
the size of Dependency Telemetry payload
the number of queue items processed by Request Telemetry
time that customer took to complete the step in wizard step completion Event Telemetry.
You can query custom measurements in Application Analytics:
customEvents
| where customMeasurements != ""
| summarize avg(todouble(customMeasurements["Completion Time"]) * itemCount)
NOTE
Custom measurements are associated with the telemetry item they belong to. They are subject to sampling with the
telemetry item containing those measurements. To track a measurement that has a value independent from other telemetry
types, use Metric telemetry.
Next steps
See data model for Application Insights types and data model.
Write custom event telemetry
Check out platforms supported by Application Insights.
Metric telemetry: Application Insights data model
12/8/2019 • 2 minutes to read • Edit Online
There are two types of metric telemetry supported by Application Insights: single measurement and pre-
aggregated metric. Single measurement is just a name and value. Pre-aggregated metric specifies minimum and
maximum value of the metric in the aggregation interval and standard deviation of it.
Pre-aggregated metric telemetry assumes that aggregation period was one minute.
There are several well-known metric names supported by Application Insights. These metrics placed into
performanceCounters table.
Metric representing system and process counters:
Name
Name of the metric you'd like to see in Application Insights portal and UI.
Value
Single value for measurement. Sum of individual measurements for the aggregation.
Count
Metric weight of the aggregated metric. Should not be set for a measurement.
Min
Minimum value of the aggregated metric. Should not be set for a measurement.
Max
Maximum value of the aggregated metric. Should not be set for a measurement.
Standard deviation
Standard deviation of the aggregated metric. Should not be set for a measurement.
Custom properties
Metric with the custom property CustomPerfCounter set to true indicate that the metric represents the windows
performance counter. These metrics placed in performanceCounters table. Not in customMetrics. Also the name of
this metric is parsed to extract category, counter, and instance names.
Name-value collection of custom properties. This collection is used to extend standard telemetry with the custom
dimensions. Examples are deployment slot that produced telemetry or telemetry-item specific property like order
number.
Max key length: 150 Max value length: 8192
Next steps
Learn how to use Application Insights API for custom events and metrics.
See data model for Application Insights types and data model.
Check out platforms supported by Application Insights.
Telemetry context: Application Insights data model
10/20/2019 • 3 minutes to read • Edit Online
Every telemetry item may have a strongly typed context fields. Every field enables a specific monitoring scenario.
Use the custom properties collection to store custom or application-specific contextual information.
Application version
Information in the application context fields is always about the application that is sending the telemetry.
Application version is used to analyze trend changes in the application behavior and its correlation to the
deployments.
Max length: 1024
Client IP address
The IP address of the client device. IPv4 and IPv6 are supported. When telemetry is sent from a service, the
location context is about the user that initiated the operation in the service. Application Insights extract the geo-
location information from the client IP and then truncate it. So client IP by itself cannot be used as end-user
identifiable information.
Max length: 46
Device type
Originally this field was used to indicate the type of the device the end user of the application is using. Today used
primarily to distinguish JavaScript telemetry with the device type 'Browser' from server-side telemetry with the
device type 'PC'.
Max length: 64
Operation id
A unique identifier of the root operation. This identifier allows to group telemetry across multiple components.
See telemetry correlation for details. The operation id is created by either a request or a page view. All other
telemetry sets this field to the value for the containing request or page view.
Max length: 128
Parent operation ID
The unique identifier of the telemetry item's immediate parent. See telemetry correlation for details.
Max length: 128
Operation name
The name (group) of the operation. The operation name is created by either a request or a page view. All other
telemetry items set this field to the value for the containing request or page view. Operation name is used for
finding all the telemetry items for a group of operations (for example 'GET Home/Index'). This context property is
used to answer questions like "what are the typical exceptions thrown on this page."
Max length: 1024
Session id
Session ID - the instance of the user's interaction with the app. Information in the session context fields is always
about the end user. When telemetry is sent from a service, the session context is about the user that initiated the
operation in the service.
Max length: 64
Anonymous user id
Anonymous user id. Represents the end user of the application. When telemetry is sent from a service, the user
context is about the user that initiated the operation in the service.
Sampling is one of the techniques to minimize the amount of collected telemetry. Sampling algorithm attempts to
either sample in or out all the correlated telemetry. Anonymous user id is used for sampling score generation. So
anonymous user id should be a random enough value.
Using anonymous user id to store user name is a misuse of the field. Use Authenticated user id.
Max length: 128
Authenticated user id
Authenticated user id. The opposite of anonymous user id, this field represents the user with a friendly name.
Since its PII information it is not collected by default by most SDK.
Max length: 1024
Account id
In multi-tenant applications this is the account ID or name, which the user is acting with. Examples may be
subscription ID for Azure portal or blog name blogging platform.
Max length: 1024
Cloud role
Name of the role the application is a part of. Maps directly to the role name in azure. Can also be used to
distinguish micro services, which are part of a single application.
Max length: 256
Next steps
Learn how to extend and filter telemetry.
See data model for Application Insights types and data model.
Check out standard context properties collection configuration.
Application Insights for Azure Functions supported
features
12/8/2019 • 2 minutes to read • Edit Online
Azure Functions offers built-in integration with Application Insights, which is available through the ILogger
Interface. Below is the list of currently supported features. Review Azure Functions' guide for Getting started.
Supported features
AZURE FUNCTIONS V1 V2 (IGNITE 2018)
Automatic collection of
• Dependencies
— HTTP Yes
— ServiceBus Yes
— EventHub Yes
— SQL Yes
Supported features
• Heartbeats Yes
Correlation
• ServiceBus Yes
AZURE FUNCTIONS V1 V2 (IGNITE 2018)
• EventHub Yes
Configurable
Performance Counters
Automatic collection of Performance Counters only work Windows machines.
Sampling
Azure Functions enables Sampling by default in their configuration. For more information, see Configure
Sampling.
If your project takes a dependency on the Application Insights SDK to do manual telemetry tracking, you may
experience strange behavior if your sampling configuration is different than the Functions' sampling configuration.
We recommend using the same configuration as Functions. With Functions v2, you can get the same
configuration using dependency injection in your constructor:
using Microsoft.ApplicationInsights;
using Microsoft.ApplicationInsights.Extensibility;
[FunctionName("Function1")]
public async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Function, "get", "post", Route = null)] HttpRequest req, ILogger
logger)
{
this.telemetryClient.TrackTrace("C# HTTP trigger function processed a request.");
}
}
Supported metrics with Azure Monitor
1/21/2020 • 133 minutes to read • Edit Online
Azure Monitor provides several ways to interact with metrics, including charting them in the portal, accessing
them through the REST API, or querying them using PowerShell or CLI. Below is a complete list of all metrics
currently available with Azure Monitor's metric pipeline. Other metrics may be available in the portal or using
legacy APIs. This list below only includes metrics available using the consolidated Azure Monitor metric
pipeline. To query for and access these metrics please use the 2018-01-01 api-version
NOTE
Sending multi-dimensional metrics via diagnostic settings is not currently supported. Metrics with dimensions are
exported as flattened single dimensional metrics, aggregated across dimension values.
For example: The 'Incoming Messages' metric on an Event Hub can be explored and charted on a per queue level.
However, when exported via diagnostic settings the metric will be represented as all incoming messages across all queues
in the Event Hub.
For a list of platform metrics exportable via diagnostic settings, see this article.
Microsoft.AnalysisServices/servers
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.ApiManagement/service
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.AppConfiguration/configurationStores
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.AppPlatform/Spring
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
AppCpuUsagePe App CPU Usage Percent Average App JVM CPU AppName,Pod
rcentage Percentage Usage
Percentage
Microsoft.Automation/automationAccounts
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.Batch/batchAccounts
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.BatchAI/workspaces
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.Blockchain/blockchainMembers
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.Cache/redis
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.Cdn/cdnwebapplicationfirewallpolicies
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.ClassicCompute/virtualMachines
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Disk Read Disk Read CountPerSecond Average Disk Read IOPS. None
Operations/Sec Operations/Sec
Disk Write Disk Write CountPerSecond Average Disk Write IOPS. None
Operations/Sec Operations/Sec
Microsoft.ClassicCompute/domainNames/slots/roles
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Disk Read Disk Read CountPerSecond Average Disk Read IOPS. RoleInstanceId
Operations/Sec Operations/Sec
Disk Write Disk Write CountPerSecond Average Disk Write IOPS. RoleInstanceId
Operations/Sec Operations/Sec
Microsoft.ClassicStorage/storageAccounts
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.ClassicStorage/storageAccounts/blobServices
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.ClassicStorage/storageAccounts/tableServices
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.ClassicStorage/storageAccounts/fileServices
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
FileShareQuota File share quota Bytes Average The upper limit FileShare
size on the amount
of storage that
can be used by
Azure Files
Service in bytes.
Microsoft.ClassicStorage/storageAccounts/queueServices
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.CognitiveServices/accounts
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Disk Read Bytes Disk Read Bytes Bytes Total Bytes read from None
disk during
monitoring
period
Disk Write Bytes Disk Write Bytes Bytes Total Bytes written to None
disk during
monitoring
period
Disk Read Disk Read CountPerSecond Average Disk Read IOPS None
Operations/Sec Operations/Sec
Disk Write Disk Write CountPerSecond Average Disk Write IOPS None
Operations/Sec Operations/Sec
Per Disk Read Data Disk Read CountPerSecond Average Bytes/Sec read SlotId
Bytes/sec Bytes/Sec from a single
(Deprecated) disk during
monitoring
period
Per Disk Write Data Disk Write CountPerSecond Average Bytes/Sec written SlotId
Bytes/sec Bytes/Sec to a single disk
(Deprecated) during
monitoring
period
Per Disk Read Data Disk Read CountPerSecond Average Read IOPS from SlotId
Operations/Sec Operations/Sec a single disk
(Deprecated) during
monitoring
period
Per Disk Write Data Disk Write CountPerSecond Average Write IOPS from SlotId
Operations/Sec Operations/Sec a single disk
(Deprecated) during
monitoring
period
Per Disk QD Data Disk QD Count Average Data Disk Queue SlotId
(Deprecated) Depth(or Queue
Length)
OS Per Disk OS Disk Read CountPerSecond Average Read IOPS from None
Read Operations/Sec a single disk
Operations/Sec (Deprecated) during
monitoring
period for OS
disk
OS Per Disk OS Disk Write CountPerSecond Average Write IOPS from None
Write Operations/Sec a single disk
Operations/Sec (Deprecated) during
monitoring
period for OS
disk
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Data Disk Read Data Disk Read CountPerSecond Average Bytes/Sec read LUN
Bytes/sec Bytes/Sec from a single
(Preview) disk during
monitoring
period
Data Disk Write Data Disk Write CountPerSecond Average Bytes/Sec written LUN
Bytes/sec Bytes/Sec to a single disk
(Preview) during
monitoring
period
Data Disk Read Data Disk Read CountPerSecond Average Read IOPS from LUN
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period
Data Disk Write Data Disk Write CountPerSecond Average Write IOPS from LUN
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period
Data Disk Queue Data Disk Queue Count Average Data Disk Queue LUN
Depth Depth (Preview) Depth(or Queue
Length)
OS Disk Read OS Disk Read CountPerSecond Average Read IOPS from None
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period for OS
disk
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
OS Disk Write OS Disk Write CountPerSecond Average Write IOPS from None
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period for OS
disk
Microsoft.Compute/virtualMachineScaleSets
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Disk Read Bytes Disk Read Bytes Bytes Total Bytes read from VMName
disk during
monitoring
period
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Disk Write Bytes Disk Write Bytes Bytes Total Bytes written to VMName
disk during
monitoring
period
Disk Read Disk Read CountPerSecond Average Disk Read IOPS VMName
Operations/Sec Operations/Sec
Disk Write Disk Write CountPerSecond Average Disk Write IOPS VMName
Operations/Sec Operations/Sec
Per Disk Read Data Disk Read CountPerSecond Average Bytes/Sec read SlotId
Bytes/sec Bytes/Sec from a single
(Deprecated) disk during
monitoring
period
Per Disk Write Data Disk Write CountPerSecond Average Bytes/Sec written SlotId
Bytes/sec Bytes/Sec to a single disk
(Deprecated) during
monitoring
period
Per Disk Read Data Disk Read CountPerSecond Average Read IOPS from SlotId
Operations/Sec Operations/Sec a single disk
(Deprecated) during
monitoring
period
Per Disk Write Data Disk Write CountPerSecond Average Write IOPS from SlotId
Operations/Sec Operations/Sec a single disk
(Deprecated) during
monitoring
period
Per Disk QD Data Disk QD Count Average Data Disk Queue SlotId
(Deprecated) Depth(or Queue
Length)
OS Per Disk OS Disk Read CountPerSecond Average Read IOPS from None
Read Operations/Sec a single disk
Operations/Sec (Deprecated) during
monitoring
period for OS
disk
OS Per Disk OS Disk Write CountPerSecond Average Write IOPS from None
Write Operations/Sec a single disk
Operations/Sec (Deprecated) during
monitoring
period for OS
disk
Data Disk Read Data Disk Read CountPerSecond Average Bytes/Sec read LUN,VMName
Bytes/sec Bytes/Sec from a single
(Preview) disk during
monitoring
period
Data Disk Write Data Disk Write CountPerSecond Average Bytes/Sec written LUN,VMName
Bytes/sec Bytes/Sec to a single disk
(Preview) during
monitoring
period
Data Disk Read Data Disk Read CountPerSecond Average Read IOPS from LUN,VMName
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period
Data Disk Write Data Disk Write CountPerSecond Average Write IOPS from LUN,VMName
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period
Data Disk Queue Data Disk Queue Count Average Data Disk Queue LUN,VMName
Depth Depth (Preview) Depth(or Queue
Length)
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
OS Disk Read OS Disk Read CountPerSecond Average Read IOPS from VMName
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period for OS
disk
OS Disk Write OS Disk Write CountPerSecond Average Write IOPS from VMName
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period for OS
disk
Microsoft.Compute/virtualMachineScaleSets/virtualMachines
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Disk Read Bytes Disk Read Bytes Bytes Total Bytes read from None
disk during
monitoring
period
Disk Write Bytes Disk Write Bytes Bytes Total Bytes written to None
disk during
monitoring
period
Disk Read Disk Read CountPerSecond Average Disk Read IOPS None
Operations/Sec Operations/Sec
Disk Write Disk Write CountPerSecond Average Disk Write IOPS None
Operations/Sec Operations/Sec
Per Disk Read Data Disk Read CountPerSecond Average Bytes/Sec read SlotId
Bytes/sec Bytes/Sec from a single
(Deprecated) disk during
monitoring
period
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Per Disk Write Data Disk Write CountPerSecond Average Bytes/Sec written SlotId
Bytes/sec Bytes/Sec to a single disk
(Deprecated) during
monitoring
period
Per Disk Read Data Disk Read CountPerSecond Average Read IOPS from SlotId
Operations/Sec Operations/Sec a single disk
(Deprecated) during
monitoring
period
Per Disk Write Data Disk Write CountPerSecond Average Write IOPS from SlotId
Operations/Sec Operations/Sec a single disk
(Deprecated) during
monitoring
period
Per Disk QD Data Disk QD Count Average Data Disk Queue SlotId
(Deprecated) Depth(or Queue
Length)
OS Per Disk OS Disk Read CountPerSecond Average Read IOPS from None
Read Operations/Sec a single disk
Operations/Sec (Deprecated) during
monitoring
period for OS
disk
OS Per Disk OS Disk Write CountPerSecond Average Write IOPS from None
Write Operations/Sec a single disk
Operations/Sec (Deprecated) during
monitoring
period for OS
disk
Data Disk Read Data Disk Read CountPerSecond Average Bytes/Sec read LUN
Bytes/sec Bytes/Sec from a single
(Preview) disk during
monitoring
period
Data Disk Write Data Disk Write CountPerSecond Average Bytes/Sec written LUN
Bytes/sec Bytes/Sec to a single disk
(Preview) during
monitoring
period
Data Disk Read Data Disk Read CountPerSecond Average Read IOPS from LUN
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period
Data Disk Write Data Disk Write CountPerSecond Average Write IOPS from LUN
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period
Data Disk Queue Data Disk Queue Count Average Data Disk Queue LUN
Depth Depth (Preview) Depth(or Queue
Length)
OS Disk Read OS Disk Read CountPerSecond Average Read IOPS from None
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period for OS
disk
OS Disk Write OS Disk Write CountPerSecond Average Write IOPS from None
Operations/Sec Operations/Sec a single disk
(Preview) during
monitoring
period for OS
disk
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.ContainerInstance/containerGroups
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.ContainerRegistry/registries
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.CustomProviders/resourceproviders
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DataBoxEdge/dataBoxEdgeDevices
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DataFactory/datafactories
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DataFactory/factories
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DataLakeAnalytics/accounts
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DataLakeStore/accounts
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DBforMariaDB/servers
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DBforMySQL/servers
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DBforPostgreSQL/servers
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.DBforPostgreSQL/serversv2
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
Microsoft.Devices/IotHubs
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS
METRIC DISPLAY AGGREGATION
METRIC NAME UNIT TYPE DESCRIPTION DIMENSIONS