Intermediate

Azure AI Security

Azure provides Azure Machine Learning, Azure OpenAI Service, and Cognitive Services for AI workloads. Securing these requires Managed Identity, Private Endpoints, Key Vault, and comprehensive monitoring.

Azure Machine Learning Security

  1. Managed Identity

    Use system-assigned or user-assigned Managed Identities for Azure ML workspaces, compute clusters, and endpoints. This eliminates the need for stored credentials and enables automatic token management through Azure AD.

  2. Private Endpoints

    Deploy Azure ML workspaces with Private Endpoints to ensure all traffic stays within your virtual network. Configure private DNS zones to resolve workspace URLs to private IP addresses.

  3. Workspace Isolation

    Use separate Azure ML workspaces for different security classifications (development, staging, production). Apply Azure RBAC at the workspace level with custom roles for data scientists, ML engineers, and operators.

  4. Compute Security

    Configure compute clusters with no public IP, SSH disabled, and managed identity attached. Use compute instance schedules to automatically shut down idle resources, reducing attack surface and cost.

Azure Key Vault for AI

Secret Type Key Vault Configuration Access Pattern
API Keys Store as Key Vault secrets with expiration dates Azure ML references via workspace connection
Encryption Keys HSM-backed keys for data encryption at rest CMEK for storage accounts and compute disks
Certificates TLS certificates for inference endpoints Auto-rotation with Key Vault managed certificates
Connection Strings Database and storage connection strings Referenced by ML pipelines at runtime

Azure OpenAI Service Security

⚠
Data Handling: Azure OpenAI Service does not use customer data to train or improve models by default. However, you should explicitly verify content filtering settings, abuse monitoring configuration, and data retention policies for your subscription.
  • Network restrictions: Configure Azure OpenAI with Private Endpoints and disable public network access to prevent unauthorized API calls
  • Content filtering: Enable and configure content safety filters appropriate for your use case. Custom content filters can block specific categories of harmful content
  • Token-based authentication: Use Azure AD authentication instead of API keys for programmatic access. This provides identity-based auditing and conditional access policies
  • Rate limiting: Configure tokens-per-minute and requests-per-minute limits to prevent abuse and control costs

Azure Monitor for ML

Diagnostic Logs

Enable diagnostic logging for Azure ML workspaces to capture deployment events, training job metrics, and endpoint invocations. Send logs to Log Analytics for querying and alerting.

Microsoft Defender for Cloud

Enable Defender for Cloud to detect security misconfigurations in ML resources. Defender provides recommendations for hardening compute, storage, and networking components.

Azure Sentinel

Integrate ML security logs with Azure Sentinel (Microsoft Sentinel) for SIEM capabilities. Create custom analytics rules to detect threats specific to AI workloads.

Compliance Dashboards

Use Azure Policy and Regulatory Compliance dashboards to track ML resource compliance against NIST, ISO 27001, SOC 2, and HIPAA frameworks continuously.

💡
Next Up: In the next lesson, we explore multi-cloud AI security — cross-cloud identity federation, unified policy management, and multi-cloud governance strategies.