Satellite Repositories / Repo Configs

You may have satellite (local) repositories on your local On-Prem where your Linux VMs used to download the package updates and those On-Prem satellite repositories maybe are no more accessible from this newly spun VM in Azure!

Its imperative that first thing you do is to test that the system’s default package managers repositories are still accessible. Depending on your Linux distro, at least try doing a repo update:

sudo yum update

or

sudo apt update

Background on fixing Repos first

f the repos fail, the Azure platform’s attempt to install the default ‘Guest Agent’ may fail too! Why? Because this machine may not have the required Python or required SSL libraries required by the Guest Agent and when agent-installation process attempts to resolve the dependency, it may fail to reach out to repos, thus failing overall process. This may inadvertently produce lots of package manager logs because Azure will reattempt to install the agent and repeat the whole process. This may even fill up the /var/log at some point, and you now you know the rest!

Publisher, SKU, Offer..

Are you using Images/Image Gallery to upload your VHDs/VMDKs and then using to spin VMs? Even if you’re migrating using replication, you should ensure that your Virtual Machines are getting the right Publisher, Offer and SKUs. Its a nice thing not only for inventory purposes, but also for example when you turn on Auto Provisioning of OMS agent from the Azure Security Center (ASC), ASC will only onboard OMS agent when it sees that that Publisher:Offer:SKU tuple is in the approved list of OMS agent’s support OS configurations.

Making sure the SWAP partition is on the free ‘Temporary Disk’

The most shocking thing you’d notice after spinning a VM would be that the SWAP partition of your Linux box will be residing on the Primary Partition and not the Temporary Disk unless you configure the Guest Agent. This is indeed very frustrating first thing to find that your newly spun Linux VM is not using the included-in-the-pricing Temporary Disk and just ignores it!

Good news is that in Azure Linux VM, ‘Guest Agent’ can make the swap management hassle free! It can keep and create the right sized swap partition on the right temporary disk by itself! More on Guest Agent below!

This page can help setup and resize a SWAP partition.

Licensing from On-Prem to Azure

You need to consider your licensing approach to using Linux, specially paid versions like Red Hat Linux. For example, when you ‘purchase’ a Red Hat Linux from Azure Marketplace, you are essentially buying a Pay as You Go license. This license puts you on update channel of RedHat in such a fashion that your machine can receive updates from the Red Hat’s satellite repositories hosted by Microsoft within Azure public cloud.

if for example you you bring your own Red Hat license form your on-prem setup, you’ll need to onboard this new VM into the Red Hat’s update channel.

Qualys Scanner Agent in Azure

You get Qualys vulnerability scanner free with Azure Defender for Servers. This free provided Qualys scanner will submit its findings to Azure Security Center for you to review. If you have purchased a Qualys license yourself (BYOK) then you might want to enable it and set the reporting daemon correctly.

Ensure that your Virtual Machine can reach Azure VM Host

On you have the machine ready, shell is there, you want to verify that your newly spun Azure Virtual Machine running Linux OS can talk to the Azure’s Virtual Machine Host that is hosting your virtual machine. Check this by doing

curl http://168.63.129.16

If you fail, then you would want to check firewall on your machine or SELinux or maybe routing table, or maybe a UDR

DNS Servers

By design, the DNS servers in an Azure VM should be pushed as part of the DHCP response from the Virtual Network (VNET) this machine has the Network Interface Card (NIC) from. If your VNET has custom DNS servers specified, this machine will be getting those addresses as part of the DHCP response, otherwise it will get the default Azure Magic address as DNS of 168.63.129.16.

systemd-resolved

The problem could happen if you were using a Linux distro (for example Ubuntu) that utilizes systemd-resolved. The daemon is itself is not a problem and perfectly compatible with Azure and can transparently integrate with a line of code. The problem could come if you have custom forward DNS resolver settings specified within the configuration of the daemon that may not respect the DHCP responded DNS servers.

Anything manipulating /etc/resolve.config

So it stands true with any program capable of modifying networking settings, including /etc/resolve.conf, for example NetworkManager

Quick Fix

A quick fix could be to disable systemd-resolved, supposing its Ubuntu along with NetworkManager

sudo systemctl disable systemd-resolved
sudo systemctl stop systemd-resolved

Then remove the symbolic link of /etc/resolve.conf going to the daemon’s dynamic file

rm /etc/resolv.conf

Remove the custom DNS setting in NetworkManager by modifying /etc/NetworkManager/NetworkManager.conf and changing the [main] section

dns=default

Finally restart the NetworkManager

sudo systemctl restart NetworkManager

Installing all necessary Agents

Which agents?

Well all necessary!

By default, all Linux VMs in Azure get (or should have auto installed) something called ‘Guest Agent’ (also aka Linux Agent or WAAgent or WALinuxAgent) shown in the Azure Portal VM blade under ‘Agent status’ and ‘Agent version’. Mind you, this will only be visible if the virtual machine is in running state.

This is the underlying platform hook that allows Azure’s Fabric to communicate with a Virtual Machine for a number of operations, including being able to install other ‘important’ extensions.

OMS Extension

This is known as OmsAgentForLinux. Also known as OMS Agent and the new formal name is Log Analytics Agent. Think of this as an endpoint log collection agent. When a VM is spun on Azure, Azure as platform has no visibility inside a guest VM and only have visibility to something referred to as Platform metrics. Not ready to install any extension on your VM and still want to do some minimalistic monitoring? Check out my guide on agent-less monitoring Azure VMs.

The beauty of OMS agent is that you can connect it with a Log Analytics agent and then collect all the data there and query!

Network Watcher Extension

This is known as AzureNetworkWatcherExtension . This extension can be used to utilize Azure Network Watcher (NW) on your VM. NW is a network performance monitoring, diagnostic, and analytics service that allows monitoring of Azure networks. More info here.

Linux Diagnostics Extension

Need a little more? One off debugging of our VM? Don’t want to the queryable logs but still want to see detailed logs, maybe in storage account? Collect detailed messages, kernel logs, syslog/rsyslog facilities? Try out Linux Diagnostics Extension.