Hero Image

Azure Logging by Pawel Pindel

 

02 May 2022 | Pawel Pindel

Mastering Azure Logging: Best Practices for Monitoring and Troubleshooting

 

Nowadays app instrumentation is very important. It helps to monitor how our infrastructure and applications work, as well as detecting weird behaviour and security issues. Thanks to logs we can easily find out why a bug has occurred in the application enabling us to fix it quickly .Instrumentation also provides us with valuable information on how users use our app. On the market, there are many providers of logging and monitoring solutions. Here I discuss how we can monitor using Azure.

Logging to file system or blob storage

Azure allows the administrator to set up basic logging to the file system. This kind of logging is not the best way to do it but can be used for some quick developer checks. This is disabled after 12 hours by default because it can exceed storage capacity quickly. There is also another drawback of this kind of logging. If you use it together with scaling it is really difficult to collect logs from different instances and find what you need.

How to turn it on?

1.   Go to App Service logs tab in App Service menu.

2.  Turn on Application Logging (Filesystem or Blob).

Here you select severity level and for blob logging: storage account and retention period.

Downloading logs

If using Kudu you should go and find folder D:HomeLogFiles. There are the logs for Windows-based apps stored by default. In the case of hosting apps on Linux-based app services, they can be found in STDERR and STDOUT.

There is also an easy way to download logs with Azure CLI by executing the following command:

az web app log download –log-filesname.zip –resource-group $resourceGroup –name $applicationName

Blob

As mentioned above, there is the possibility to save logs in blob storage. However, that works only for Windows-based web apps. If it is turned on logs are saved in folders with the following structure: yearmonthdayhour.

App Center for Mobile and Desktop

All above is about how to save logs for web apps. But what if you develop a desktop or mobile app? Here App Center comes to the rescue. It is a great platform for monitoring and managing your application. It can easily integrate with Google Play Store or Apple Store. App Center not only lets you monitor but also build and distribute your software to end-users.

To start using it you need to install App Center SDK in your app and that’s it. App Center allows you to configure itself to let users choose whether or not to send you diagnostic data.

Logs retention

All logs are stored by default for 90 days but this can be shortened to 28. If you want to extend that period you might export data to Azure Blob Storage.

Copy of data collected by App Center can be sent to Application Insights for further analysis.

Application Insights

Application Insights – as part of Azure Monitor, it is the most complex solution for application performance management in Azure. Service automatically detects anomalies and contains many powerful analytics tools. It can visualise telemetry in a variety of ways:

  • live metrics streams – almost real-time performance values,
  • metrics explorer – a tool that shows how metrics vary over time,
  • alerts – sends the application administrator a message when a specified metric exceeds  the assumed threshold,
  • profiler – shows how a set of requests are delivered,
  • application map – displays components of the app and how they are linked to each other,
  • usage analysis – info about the user using your app

Application Insights use a language called KUSTO to query logs.

Backend

There are two ways to enable instrumentation in your app:

Run time

You can do it without any changes in your code. This can be done by enabling only the checkbox in the service you use. Turn on it by going to App Service -> Settings tab -> Application Insights and select Enable option.

It is easy and fast but has some limitations. Firstly, it works only on Windows-based app services. Secondly, it collets more basic data and it lacks the great logging features available with SDK.

Build-time

This means you have to install SDK to your application. There are many supported languages and frameworks like .NET, Java, Node.js, Python and probably many others created by the community, This way you are provided with a feature-rich solution to monitor your application.

Frontend

There is also a way to enable client-side instrumentation by including a javascript library in the frontend web page. This can be done just by adding an application setting called APPINSIGHTS_JAVASCRIPT_ENABLED with value 1.

Monitoring cloud resources

Azure Monitor is a great Azure service; it is the heart of logging and monitoring infrastructure. It provides a single management point for all infrastructure logs.

Service collects data from all tiers:

  • app monitoring – data about the performance of the code
  • Azure resources
  • subscription – management and operations information
  • tenant monitoring – for example monitoring of Active Directory

All the data from Monitor goes to Log Analytics where it is automatically indexed and datatypes and tables are created.

Log analytics

Log analytics store all data that can be later consumed by any service we can imagine. We can visualise data in the Azure portal by creating dashboards or exporting data for visualization in services like Power BI.

But as you can see, in the image above, there is a range of other important options. I would suggest, even more important.

Alerts

You can define alerts. Alerts are used to notify a specified person/people about exceeding the threshold. In order to do that you should go to Monitor -> Alerts and create an alert rule. Here you define the resource, conditions and action that should be taken when conditions are met. One condition costs monthly around $0.10.

Logic Apps

Logic apps can be triggered whenever a schedule is set. Then with Azure Monitor Logs connector it can run a query and list results or visualize them. This is how it sends data to other external systems. The same functionality can be used by a more business orientated solution called – Power Automate.

Other

There can be also Power shell scripts that will run in response to alerts or Rest API endpoint that will be triggered.