Section 7 – Implement platform protection – Configure advanced security for compute – Security and access for container services

Run containers in a hybrid environment - Azure Architecture Center |  Microsoft Docs

And the AZ-500 study guide continues with the following topics:

  • Configure security for container services
  • Manage access to Azure Container Registry

What are containers?

A container is a pre-built software environment in which application code and its dependencies are preloaded into an image. There are some fundamental differences in technology.

Machine virtualization works at the hardware level and provides a way to run multiple instances of an operating system, while containers share a host operating system and run in separate processes.

Container versus Virtual machine

What if you want on-premises?

Operating system requirements

  • The Windows container feature is available on Windows Server 2022, Windows Server (Semi-Annual Channel), Windows Server 2019, Windows Server 2016, and Windows 10 Professional and Enterprise Editions (version 1607 and later).
  • The Hyper-V role must be installed before running Hyper-V isolation.
  • Windows Server Container hosts must have Windows installed to c:. This restriction does not apply if only Hyper-V isolated containers will be deployed.

Virtualized container hosts

If you’re running a Windows container host from a Hyper-V virtual machine, and also hosting Hyper-V isolation, you need to enable nested virtualization. Nested virtualization has the following requirements:

  • At least 4 GB RAM available for the virtualized Hyper-V host.
  • Windows Server 2022, Windows Server 2019, Windows Server 2016, or Windows 10 on the host system; and Windows Server (Nano Server or Server Core) on the virtual machine.
  • A processor with Intel VT-x (this feature is currently available for Intel and AMD processors).
  • The container host VM also needs at least two virtual processors.

Configure security for container services

But are going all cloud, so here goes.

Authentication

MethodHow to authenticateScenarios Azure role-based access control (Azure RBAC) Limitations 
Individual AD identity az acr login in Azure CLI

Connect-AzContainerRegistry in Azure PowerShell 
Interactive push/pull by developers, testers Yes AD token must be renewed every 3 hours 
AD service principal docker login

az acr login in Azure CLI

Connect-AzContainerRegistry in Azure PowerShell

Registry login settings in APIs or tooling

Kubernetes pull secret   
Unattended push from CI/CD pipeline

Unattended pull to Azure or external services 
Yes SP password default expiry is 1 year 
Managed identity for Azure resources docker login

az acr login in Azure CLI

Connect-AzContainerRegistry in Azure PowerShell
Unattended push from Azure CI/CD pipeline

Unattended pull to Azure services
Yes Use only from select Azure services that support managed identities for Azure resources
AKS cluster managed identity Attach registry when AKS cluster created or updated Unattended pull to AKS cluster in the same or a different subscriptionNo, pull access only Only available with AKS cluster

Can’t be used for cross-tenant authentication 
AKS cluster service principal Enable when AKS cluster created or updated Unattended pull to AKS cluster from registry in another AD tenant No, pull access only Only available with AKS cluster 
Admin user docker login Interactive push/pull by individual developer or tester

Portal deployment of image from registry to Azure App Service or Azure Container Instances
No, always pull and push access Single account per registry, not recommended for multiple users 
Repository-scoped access token docker login

az acr login in Azure CLI

Connect-AzContainerRegistry in Azure PowerShell

Kubernetes pull secret 
Interactive push/pull to repository by individual developer or tester

Unattended pull from repository by individual system or external device 
Yes Not currently integrated with AD identity 

RBAC-roles

Role/PermissionAccess Resource ManagerCreate/delete registryPush imagePull imageDelete image dataChange policiesSign images
OwnerXXXXXX
ContributorXXXXXX
ReaderXX
AcrPushXX
AcrPullX
AcrDeleteX
AcrImageSignerX

Container access

  • Azure Container Instances enables exposing your container groups directly to the internet with an IP address and a fully qualified domain name (FQDN). When you create a container instance, you can specify a custom DNS name label so your application is reachable at customlabel.azureregion.azurecontainer.io.
  • Azure Container Instances also supports executing a command in a running container by providing an interactive shell to help with application development and troubleshooting. Access takes places over HTTPS, using TLS to secure client connections.

Kubernetes

A Kubernetes cluster is divided into two components:

  • Control plane nodes provide the core Kubernetes services and orchestration of application workloads.
  • Nodes run your application workloads.
kubernetes cluster architecture

Terminology

Cluster master

When you create an AKS cluster, the cluster master is automatically created and configured. This cluster master is deployed as a user-abstracted managed Azure resource. There is no charge for the Cluster Master. You only pay for the nodes that are part of your AKS cluster.

The cluster master includes the following core Kubernetes components:

  • kube-apiserver – The API server is how the underlying Kubernetes APIs are exposed. This component provides the interaction for management tools, such as kubectl or the Kubernetes dashboard.
  • etcd – To maintain the state of your Kubernetes cluster and configuration, the highly available etcd is a key value store within Kubernetes.
  • kube-scheduler – When you create or scale applications, the Scheduler determines what nodes can run the workload and starts them.
  • kube-controller-manager – The Controller Manager oversees a number of smaller Controllers that perform actions such as replicating pods and handling node operations.

Master security

In AKS, the Kubernetes master component is part of a managed service provided by Microsoft. Each AKS cluster has a dedicated single-tenant Kubernetes master to provide API servers, schedulers, and more. This master is managed and maintained by Microsoft. By default, the Kubernetes API server uses a public IP address and a fully qualified domain name (FQDN). You can control access to API servers using Kubernetes and Azure Active Directory role-based access control.

Nodes and node pools

To run your applications and supporting services, you need a Kubernetes node. An AKS cluster has one or more nodes, which is an Azure virtual machine (VM) that runs the Kubernetes node components and container runtime:

  • The kubelet is the Kubernetes agent that processes the orchestration requests from the control plane and scheduling of running the requested containers.
  • Virtual networking is handled by the kube-proxy on each node. The proxy routes network traffic and manages IP addressing for services and pods.
  • The container runtime is the component that allows containerized applications to run and interact with additional resources such as the virtual network and storage. In AKS, Moby is used as the container runtime.
A VM kubelet connects to a container through the container runtime. The container access disks and files. The kube-proxy access virtual networking.

Node security

AKS nodes are Azure digital machines which you control and maintain. Linux nodes run an optimized Ubuntu distribution the use of the Moby field runtime. Windows Server nodes run an optimized Windows Server 2019 launch and additionally use the Moby field runtime. When an AKS cluster is created or scaled up, the nodes are robotically deployed with the trendy OS protection updates and configurations.

The Azure platform robotically applies OS protection patches to Linux nodes on a nightly basis. If a Linux OS protection replace calls for a number reboot, that reboot isn’t robotically performed. You can manually reboot the Linux nodes, or a not unusualplace technique is to apply Kured, an open-supply reboot daemon for Kubernetes. Kured runs as a DaemonSet and video display units every node for the presence of a document indicating that a reboot is required. Reboots are controlled throughout the cluster the use of the identical cordon and drain procedure as a cluster upgrade.

For Windows Server nodes, Windows Update does not automatically run and apply the latest updates. On a regular schedule around the Windows Update release cycle and your own validation process, you should perform an upgrade on the Windows Server node pool(s) in your AKS cluster. This upgrade process creates nodes that run the latest Windows Server image and patches, then removes the older nodes. Nodes are deployed into a private virtual network subnet, with no public IP addresses assigned. For troubleshooting and management purposes, SSH is enabled by default. This SSH access is only available using the internal IP address.

Services

To simplify the network configuration for application workloads, Kubernetes uses Services to logically group a set of pods together and provide network connectivity. The following Service types are available:

  • Cluster IP – Creates an internal IP address for use within the AKS cluster. Good for internal-only applications that support other workloads within the cluster.
  • NodePort – Creates a port mapping on the underlying node that allows the application to be accessed directly with the node IP address and port.
  • LoadBalancer – Creates an Azure load balancer resource, configures an external IP address, and connects the requested pods to the load balancer backend pool. To allow customers’ traffic to reach the application, load balancing rules are created on the desired ports.
  • ExternalName – Creates a specific DNS entry for easier application access.
Internal traffic uses Cluster IP to access the pod. Incoming direct traffic uses NodePort. Incoming non-direct traffic uses the load balancer.

How to create a container?

Open https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.ContainerInstance%2FcontainerGroups

Choose your own settings, You can define also an image source at this time

And also the image that you want to use

Networking screen let’s you choose between:

  • Public‘ will create a public IP address for your container instance.
  • Private‘ will allow you to choose a new or existing virtual network for your container instance. This is not yet available for Windows containers.
  • None‘ will not create either a public IP or virtual network. You will still be able to access your container logs using the command line.

Public

Private

None

Advanced

You can choose for restarting and it will determinate when your container should restart.

But your own environment variables and command overrides.

Command override

The first command to execute within the container. Separate each command with a comma and a space as seen in the example. If you’re using values that contain sensitive information like passwords in the command override field, it’s recommended that you use a secure environment variable to get it into the container instead, and reference the container.

Main page

Once the container instance is done provisioning, you will find the container under it.

And you can add system and user-managed identities (still in preview)

and you can directly connect to the container from the main page

Container registries

Open https://portal.azure.com/#blade/HubsExtension/BrowseResource/resourceType/Microsoft.ContainerRegistry%2Fregistries

And create a new container registry

In the first page you can choose a SKU

SKU pricing

BasicStandardPremium
Price per day€0.151€0.601€1.502
Included storage (GB)10100500
Premium offers enhanced throughput for docker pulls across multiple, concurrent nodes
Total web hooks210500
Geo ReplicationNot SupportedNot SupportedSupported
€1.502 per replicated region

But the private access is only available for Premium SKU, I don’t know why this isn’t mentioned in the pricing.

Networking

And done

Encryption

In the next screen you can choose your own key to encryption. If you have a Key Vault premium, you can also use Managed HSM pools to store the keys.

Creating a key

You an create a key or use existing one.

And one thing worth the mention is Key Vault rotation (Still in preview) but please use it, excellent feature.

And final settings

Container instance image from registry

First you have to enable the Admin user which is disabled by default

Once you have the Admin user enabled, you can create new Container instance and get images from the registry.

BYOK with Azure disks in Azure Kubernetes Service (AKS)

Azure Storage encrypts all data in a storage account at rest. By default, data is encrypted with Microsoft-managed keys. For additional control over encryption keys, you can supply customer-managed keys to use for encryption at rest for both the OS and data disks for your AKS clusters

Limitations

  • Data disk encryption support is limited to AKS clusters running Kubernetes version 1.17 and above.
  • Encryption of OS disk with customer-managed keys can only be enabled when creating an AKS cluster.

Prerequisites

  • You must enable soft delete and purge protection for Azure Key Vault when using Key Vault to encrypt managed disks.
  • You need the Azure CLI version 2.11.1 or later.

How to setup?

Add tags to the cluster

When you create or update an AKS cluster with the --tags parameter, the following are assigned the Azure tags that you’ve specified:

  • The AKS cluster
  • The route table that’s associated with the cluster
  • The public IP that’s associated with the cluster
  • The network security group that’s associated with the cluster
  • The virtual network that’s associated with the cluster

To create a cluster and assign Azure tags, run az aks create with the --tags parameter, as shown in the following command. Running the command creates a myAKSCluster in the myResourceGroup with the tags dept=IT and costcenter=9999.

Manage access to Azure Container Registry

RBAC-roles

Role/PermissionAccess Resource ManagerCreate/delete registryPush imagePull imageDelete image dataChange policiesSign images
OwnerXXXXXX
ContributorXXXXXX
ReaderXX
AcrPushXX
AcrPullX
AcrDeleteX
AcrImageSignerX

Operations

Security recommendations for Azure Container Instances

Use a private registry

A publicly available container image does not guarantee security. Container images consist of multiple software layers, and each software layer might have vulnerabilities. To help reduce the threat of attacks, you should store and retrieve images from a private registry, such as Azure Container Registry or Docker Trusted Registry. In addition to providing a managed private registry, Azure Container Registry supports service principal-based authentication through Azure Active Directory for basic authentication flows. This authentication includes role-based access for read-only (pull), write (push), and other permissions.

Monitor and scan container images

For example, Azure Container Registry optionally integrates with Microsoft Defender for Cloud to automatically scan all Linux images pushed to a registry. Microsoft Defender for Cloud’s integrated Qualys scanner detects image vulnerabilities, classifies them, and provides remediation guidance.

Security monitoring and image scanning solutions such as Twistlock and Aqua Security are also available through the Azure Marketplace.

Protect credentials

Containers can spread across several clusters and Azure regions. So, you must secure credentials required for logins or API access, such as passwords or tokens. Ensure that only privileged users can access those containers in transit and at rest. Inventory all credential secrets, and then require developers to use emerging secrets-management tools that are designed for container platforms. Make sure that your solution includes encrypted databases, TLS encryption for secrets data in transit, and least-privilege Azure role-based access control (Azure RBAC)Azure Key Vault is a cloud service that safeguards encryption keys and secrets (such as certificates, connection strings, and passwords) for containerized applications.

Scan for vulnerabilities

New vulnerabilities are discovered all the time, so scanning for and identifying vulnerabilities is a continuous process. Incorporate vulnerability scanning throughout the container lifecycle:

  • As a final check in your development pipeline, you should perform a vulnerability scan on containers before pushing the images to a public or private registry.
  • Continue to scan container images in the registry both to identify any flaws that were somehow missed during development and to address any newly discovered vulnerabilities that might exist in the code used in the container images.

Enforce least privileges in runtime

The concept of least privileges is a basic security best practice that also applies to containers. When a vulnerability is exploited, it generally gives the attacker access and privileges equal to those of the compromised application or process.

Monitor container activity and user access

As with any IT environment, you should consistently monitor activity and user access to your container ecosystem to quickly identify any suspicious or malicious activity. Azure provides container monitoring solutions including:

  • Azure Monitor for containers monitors the performance of your workloads deployed to Kubernetes environments hosted on Azure Kubernetes Service (AKS). Azure Monitor for containers gives you performance visibility by collecting memory and processor metrics from controllers, nodes, and containers that are available in Kubernetes through the Metrics API.
  • The Azure Container Monitoring solution helps you view and manage other Docker and Windows container hosts in a single location. For example:
    • View detailed audit information that shows commands used with containers.
    • Troubleshoot containers by viewing and searching centralized logs without having to remotely view Docker or Windows hosts.
    • Find containers that may be noisy and consume excess resources on a host.
    • View centralized CPU, memory, storage, and network usage and performance information for containers.

Monitor container resource activity

Monitor your resource activity, like files, network, and other resources that your containers access. Monitoring resource activity and consumption is useful both for performance monitoring and as a security measure.

Azure Monitor enables core monitoring for Azure services by allowing the collection of metrics, activity logs, and diagnostic logs. For example, the activity log tells you when new resources are created or modified.

Log all container administrative user access for auditing

Maintain an accurate audit trail of administrative access to your container ecosystem, including your Kubernetes cluster, container registry, and container images. These logs might be necessary for auditing purposes and will be useful as forensic evidence after any security incident.

Microsoft Defender for Containers

What are the benefits of Microsoft Defender for Containers?

Defender for Containers helps with the core aspects of container security:

  • Environment hardening – Defender for Containers protects your Kubernetes clusters whether they’re running on Azure Kubernetes Service, Kubernetes on-prem / IaaS, or Amazon EKS. By continuously assessing clusters, Defender for Containers provides visibility into misconfigurations and guidelines to help mitigate identified threats. Learn more in Hardening.
  • Vulnerability assessment – Vulnerability assessment and management tools for images stored in ACR registries and running in Azure Kubernetes Service. Learn more in Vulnerability assessment.
  • Run-time threat protection for nodes and clusters – Threat protection for clusters and Linux nodes generates security alerts for suspicious activities. Learn more in Run-time protection for Kubernetes nodes, clusters, and hosts.

Scanning images in ACR registries

Defender for Containers includes an integrated vulnerability scanner for scanning images in Azure Container Registry registries.

There are four triggers for an image scan:

  • On push – Whenever an image is pushed to your registry, Defender for Containers automatically scans that image. To trigger the scan of an image, push it to your repository.
  • Recently pulled – Since new vulnerabilities are discovered every day, Microsoft Defender for Containers also scans, on a weekly basis, any image that has been pulled within the last 30 days. There’s no extra charge for these rescans; as mentioned above, you’re billed once per image.
  • On import – Azure Container Registry has import tools to bring images to your registry from Docker Hub, Microsoft Container Registry, or another Azure container registry. Microsoft Defender for container Containers scans any supported images you import. Learn more in Import container images to a container registry.
  • Continuous scan– This trigger has two modes:
    • A Continuous scan based on an image pull. This scan is performed every seven days after an image was pulled, and only for 30 days after the image was pulled. This mode doesn’t require the security profile, or extension.
    • (Preview) Continuous scan for running images. This scan is performed every seven days for as long as the image runs. This mode runs instead of the above mode when the Defender profile, or extension is running on the cluster.

How to enable?

Open Defender for cloud from https://portal.azure.com/#blade/Microsoft_Azure_Security

And choose your existing subscription from the list

When you edit configuration you will see the following

Things to remember

What is a container?

How can you authenticate to a container.

RBAC-roles

Role/PermissionAccess Resource ManagerCreate/delete registryPush imagePull imageDelete image dataChange policiesSign images
OwnerXXXXXX
ContributorXXXXXX
ReaderXX
AcrPushXX
AcrPullX
AcrDeleteX
AcrImageSignerX

Bring You Own Key and how to use service tags

Security recommendations for Azure Container Instances

How to use Microsoft Defender for Containers and the pricing €6.1872/vCore/month ( This price includes 20 free scans per vCore)

Remember that Microsoft Defender for Kubernetes and Microsoft Defender for ACR has been deprecated

Link to main post

Author: Harri Jaakkonen

Leave a Reply

Your email address will not be published. Required fields are marked *