Section 10 – Mitigate threats using Microsoft Sentinel – Plan and implement the use of data connectors for ingestion of data sources

And in this 10th section on my SC-200 study guide we will go through the following topics.

Identify data sources to be ingested for Microsoft Sentinel

On August 31, 2024, the Log Analytics agent is deprecated. You should begin preparing for your migration to the AMA if your Microsoft Sentinel deployment uses the Log Analytics agent.

  • Solutions: As part of the Microsoft Sentinel solution, several data connections are installed together with pertinent information such analytics rules, workbooks, and playbooks.
  • Community connections: The Microsoft Sentinel community has created additional data connectors, which are available in the Azure Marketplace. The company that developed the community data connection is in charge of providing documentation.
  • Custom connectors: You can also build your own, custom connection if you have a data source that isn’t listed or isn’t presently supported.

You need to think about the following:

  • What data you need to monitor?
  • What source you logs come from?
  • What kind of data sources you have?
  • How do they log activities?
  • Methods for ingesting those logs

And the following connectors are free to use:

  • Azure AD Identity Protection
  • Azure Activity Logs
  • Office 365
  • Microsoft Defender for Cloud
  • Microsoft Defender for IoT
  • Microsoft 365 Defender
  • Microsoft Defender for Endpoint
  • Microsoft Defender for Identity
  • Microsoft Defender for Cloud Apps

You can find the connectors from Sentinel -> Data connectors

and also what you have connected under Sentinel management page.

Identify the prerequisites for a Microsoft Sentinel data connector

You need to Identity the following:

  • What Azure subscription to use
  • Whare are the Information sources
  • What connectors for data is needed
  • How the connectivity to the network is built
  • Authentication methods used
  • Permissions that are required
  • What detailed configurations you will need

OOTB could you help you to answer these questions

There are several different verticals available.

Category nameDescription
AeronauticsProducts, services, and content specific for the aeronautics industry
EducationProducts, services, and content specific for the education industry
FinanceProducts, services, and content specific for the finance industry
HealthcareProducts, services, and content specific for the healthcare industry
ManufacturingProducts, services, and content specific for the manufacturing industry
RetailProducts, services, and content specific for the retail industry

Of we choose example Identity, we can see the following.

And see a closer look an Amazon, you see the different content types it will provision.

Or as a comparison for Azure AD

In example here are the logs that will be inside the solution

And you also see the different analytic rules under the solution and the links to understand more on what they do.

See more from Learn

Or simply with Data connectors, it will show you the permissions, licensing and logs when you start to provisioning it.

Configure and use Microsoft Sentinel data connectors

Today Content Hub serves as the central location for all Data Connectors and additional out-of-the-box (OOTB) content. Only active Data Connectors will be accessible in this gallery starting in Q2 2023.

In example you can add Amazon to your Data connectors

When you click the “Open connector page” you will see what is needed for the connector to work.

See more here how to setup an AWS service connection

UEBA

If you have on-premises Identities and MDI enabled, you should enable also UEBA.

You will find it under M365 Defender data connector.

See more from Learn

You need UEBA to detect the following anomalies:

See more from Learn on User on UEBA

Configure Microsoft Sentinel data connectors by using Azure Policy

  1. Launch the Azure Policy Assignment wizard and follow the instructions displayed on the screen. To choose your resource assignment scope, click the three-dot button under Scope on the Basics tab.
  2. Choose your Microsoft Sentinel workspace from the Log Analytics workspace drop-down list in the Parameters tab, then mark as “True” all the log and metric categories you wish to ingest.
  3. Select the Remediation tab and check the Create a remediation task checkbox to apply the policy to your existing resources.

Choose you workspace and if this workspace is beyond the scope of the assignment, you must manually provide the policy assignment’s primary ID ‘Log Analytics Contributor’ access (or something similar).

And create a remediation task. The policy will be evaluated against existing resources. Up to 500 resources found to be non-compliant will be remediated.

And once done, you can see the Policy under https://portal.azure.com/#view/Microsoft_Azure_Policy/PolicyMenuBlade/~/Overview

You should enable Data connector with policies for the following reasons:

  1. Increased visibility: Collect data from multiple sources and analyze it in a single place to enforce policies and detect policy violations.
  2. Improved compliance: Enforce compliance with regulatory requirements, industry standards, and organizational policies using automated data collection and analysis.
  3. Enhanced security: Enforce security policies, detect security threats in real-time, and take immediate action to prevent damage to your environment.

Configure Microsoft Sentinel connectors for Microsoft 365 Defender and Microsoft Defender for Cloud

M365 Defender

  • Microsoft 365 Defender requires a valid license, as indicated in Microsoft 365 Defender requirements.
  • On the tenant from which you want to stream logs, your user must be assigned the Global Administrator or Security Administrator roles.
  • On your Microsoft Sentinel workspace, your user must have read and write permissions.
  • Any changes to the connector settings must be made by a user who is a member of the same Azure Active Directory tenant as your Microsoft Sentinel workspace.

You will start the import from Data connector and find Microsoft 365 Defender

First you will see the prerequisites

And then the configuration options and note “Turn off all incident creation rules”

Why this matters:

Microsoft 365 Defender automatically creates incidents out of alerts from Azure Active Directory Identity Protection, Microsoft Defender for Cloud Apps, Microsoft Defender for Office 365, Microsoft Defender for Endpoint, and Microsoft Defender for Identity. It is advised to disable incident generation rules for certain items in order to prevent duplicate occurrences in the incidents queue.
Be aware that any filters you include in your incident generation rules won’t be considered when using the direct incident integration.

  • Connect incidents and alerts allows for basic integration of Microsoft 365 Defender and Microsoft Sentinel, syncing events and notifications across the two systems.
  • Connect entities allow on-premises Active Directory user identities to be integrated into Microsoft Sentinel using Microsoft Defender for Identity (Your tenant must be onboarded to Microsoft Defender for Identity. You must have the MDI sensor installed.)
  • The ability to connect events allows for the collecting of raw advanced hunting events from Defender components.

Read more from Learn

Defender for Cloud

What is needed before:

  • Your Microsoft Sentinel workspace must grant you read and write access.
  • The subscriptions of the logs you stream must have the Security Reader role assigned to you.
  • For any subscription where you wish to enable the connection, Microsoft Defender for Cloud requires that at least one plan be enabled. You need to have the Security Admin role for that subscription in order to enable Microsoft Defender plans on it.

Similarly to M365 Defender, you will Defender for Cloud under Data connectors.

Once you open the page, you will see the prerequisites and configuration. You can enable data connector for all your subscriptions at once.

By turning on bi-directional sync, Microsoft Sentinel incidents with Microsoft Defender alerts will instantly update their status with the original alerts in Microsoft Defender. For instance, the related alert in Microsoft Defender will be closed immediately when a Sentinel event including a Microsoft Defender alert is closed.

You need to have the Contributor or Security Admin role on the necessary subscription in order to enable bi-directional sync.

Read more from Learn

Why you should enable these data connectors?

Microsoft 365 Defender and Microsoft Defender for Cloud can send a variety of alerts to Microsoft Sentinel. Here are some examples of the types of alerts that can be sent:

  • Endpoint Protection: Alerts related to endpoint protection, including malware detection, ransomware detection, and suspicious file execution.
  • Identity Protection: Alerts related to identity protection, including suspicious sign-ins, risky sign-ins, and anomalous authentication.
  • Threat Intelligence: Alerts related to threat intelligence, including indicators of compromise (IOCs), suspicious IP addresses, and suspicious domains.
  • Cloud App Security: Alerts related to cloud app security, including risky sign-ins, risky app usage, and data exfiltration.
  • Information Protection: Alerts related to information protection, including data leaks, data classification, and policy violations.
  • Device Management: Alerts related to device management, including non-compliant devices, device inventory changes, and security configuration changes.

Design and configure Syslog and CEF event collections

What’s the distinction between Syslog and CEF Syslog?


Most network and security systems offer either Syslog or CEF (Common Event Format) as a data transmission protocol. CEF has a benefit over Syslog in that it provides data normalization, making it more instantly usable.

What decisions you should make?

  • What data sources are you looking to use to gather Syslog and CEF events?
  • What security events are you looking to gather and examine?
  • What regulations must be followed while gathering and storing the data?

What you can use?

Design and configure Windows Security event collections

Windows-based operating systems produce Windows Security events, which are logs of security-related activities that have taken place on the system. For incident response and security monitoring, these occurrences are helpful.

Before you enable the Azure Monitor Agent-based connector, the system must have Azure Arc installed and enabled in order to collect events from any system that is not an Azure virtual machine.

This means the following:

  • Windows servers that are physically installed
  • Installed Windows servers on on-site virtual machines
  • Virtual machines with Windows servers running in non-Azure clouds

Once you select your resources, there will an System Assigned Managed Identity enabled on these machines, in addition to existing User Assigned Identities (if any).

You can define the following levels.

  • All events – All Windows Security and App Locker events.
  • Common – A standard set of events for auditing purposes.
  • Minimal – A small set of events that might indicate potential threats. By enabling this option, you won’t be able to have a full audit trail.
  • Custom – Allows you to filter and select the security events to stream by using Xpath queries.

If you want detailed information on the different events, read Jeffrey Appel’s blog on it.

Configure custom threat intelligence connectors

Integrating threat intelligence (TI) into Microsoft Sentinel helps you stay informed of the latest threats, identify and prioritize alerts, enrich security events, and improve threat hunting. It ultimately improves the accuracy and effectiveness of your security monitoring and response capabilities.

Comparison table from Microsoft

Method descriptionCapabilityServerlessComplexity
Codeless Connector Platform (CCP)
Best for less technical audiences to create SaaS connectors using a configuration file instead of advanced development.
Supports all capabilities available with the code.YesLow; simple, codeless development
Log Analytics Agent
Best for collecting files from on-premises and IaaS sources
File collection onlyNoLow
Logstash
Best for on-premises and IaaS sources, any source for which a plugin is available, and organizations already familiar with Logstash
Available plugins, plus custom plugin, capabilities provide significant flexibility.No; requires a VM or VM cluster to runLow; supports many scenarios with plugins
Logic Apps
High cost; avoid for high-volume data
Best for low-volume cloud sources
Codeless programming allows for limited flexibility, without support for implementing algorithms.

If no available action already supports your requirements, creating a custom action may add complexity.
YesLow; simple, codeless development
PowerShell
Best for prototyping and periodic file uploads
Direct support for file collection.

PowerShell can be used to collect more sources, but will require coding and configuring the script as a service.
NoLow
Log Analytics API
Best for ISVs implementing integration, and for unique collection requirements
Supports all capabilities available with the code.Depends on the implementationHigh
Azure Functions
Best for high-volume cloud sources, and for unique collection requirements
Supports all capabilities available with the code.YesHigh; requires programming knowledge

CCP (Public preview)

In example with CCP connector you can create two JSON files and deploy them with templates to Azure

See here for the JSON template

Labs

Here are some labs for the two section for Sentinel covered so far.

ModuleLab
Learning Path 4 – Create queries for Microsoft Sentinel using Kusto Query Language (KQL)Exercise 1 – Create queries for Microsoft Sentinel using Kusto Query Language (KQL)
Learning Path 5 – Configure your Microsoft Sentinel environmentExercise 1 – Configure your Microsoft Sentinel environment
Learning Path 6 – Connect logs to Microsoft SentinelExercise 1 – Connect data to Microsoft Sentinel using data connectors
Learning Path 6 – Connect logs to Microsoft SentinelExercise 2 – Connect Windows devices to Microsoft Sentinel using data connectors
Learning Path 6 – Connect logs to Microsoft SentinelExercise 3 – Connect Linux hosts to Microsoft Sentinel using data connectors

Closure

And again some things to remember:

The different data connections to Sentinel:

  • Solutions: As part of the Microsoft Sentinel solution, several data connections are installed together with pertinent information such analytics rules, workbooks, and playbooks.
  • Community connections: The Microsoft Sentinel community has created additional data connectors, which are available in the Azure Marketplace. The company that developed the community data connection is in charge of providing documentation.
  • Custom connectors: You can also build your own, custom connection if you have a data source that isn’t listed or isn’t presently supported.

You need to think about the following:

  • What data you need to monitor?
  • What source you logs come from?
  • What kind of data sources you have?
  • How do they log activities?
  • Methods for ingesting those logs

You need to Identity the following when designing Data connector:

  • What Azure subscription to use
  • Whare are the Information sources
  • What connectors for data is needed
  • How the connectivity to the network is built
  • Authentication methods used
  • Permissions that are required
  • What detailed configurations you will need

Why you should enable Data connector with Azure policies?

What information Microsoft 365 Defender and Microsoft Defender for Cloud can send to Microsoft Sentinel.

What are the methods you can use to connect custom threat intelligence (TI) connectors?

Link to main post

This image has an empty alt attribute; its file name is image-123.png
Author: Harri Jaakkonen

Leave a Reply

Your email address will not be published. Required fields are marked *