Senior Azure Databricks Platform Security Engineer

Job Title: Senior Azure Databricks Platform Security Engineer

Location: Navi Mumbai

Working Hours: 9:00 AM – 6:00 PM

Mode of Interview: Face-to-Face at Navi Mumbai

Headcount: 1 Position

Position Summary

The Senior Azure Databricks Platform Security Engineer is an individual contributor role responsible for the end-to-end security of the Azure Databricks platform and the Lakehouse data estate. The incumbent will harden Azure Databricks workspaces, govern Unity Catalog, seamlessly integrate Databricks with Azure tenant-level security controls, and build custom Microsoft Sentinel threat detections and automated playbooks for all Databricks workloads.

Requirements

Key Responsibilities

• Azure Databricks Platform Security & Unity Catalog Governance:
  

  • Harden Azure Databricks workspaces using VNet injection, Secure Cluster Connectivity (No Public IP), Private Link, Customer-Managed Keys (CMK), and Personal Access Token (PAT) / Service Principal governance.
    

  • Configure and govern Unity Catalog metastores, catalogs, schemas, external locations, storage credentials, and fine-grained row/column-level access controls.
    

  • Define and enforce cluster policies, init-script security controls, Azure Key Vault-backed secret scopes, and library allow-lists.
    

  • Audit access across ADLS Gen2, Delta Lake, and external data sources while enforcing encryption-at-rest (CMK) and Private Endpoint connectivity.
    

• Azure Cloud Tenant Security Controls:
  

  • Configure Microsoft Entra ID security controls supporting Databricks, including Conditional Access policies, Multi-Factor Authentication (MFA), and automated SCIM user/group provisioning.
    

  • Operate Azure Privileged Identity Management (PIM) for Databricks workspace and resource-group privileged roles, leading periodic access reviews.
    

  • Apply and enforce Azure Policy initiatives across Databricks deployments to ensure compulsory VNet injection, disabled public access, CMK usage, and active diagnostic logging.
    

• Sentinel Threat Detections & Response Automation:
  

  • Onboard complete Databricks diagnostic log sources (workspace, cluster, jobs, secrets, SQL endpoints, Unity Catalog) into Log Analytics and Microsoft Sentinel.
    

  • Design and build Sentinel Analytics Rules and Workbooks tailored to Databricks-specific threat vectors (PAT token abuse, init-script tampering, cluster-policy bypasses, and unauthorized Unity Catalog grant modifications).
    

  • Author Logic Apps playbooks to automate threat response actions, including instant token revocation, ITSM ticket creation, and Teams/email security alerts.
    

Candidate Profile & Qualifications

• Experience: 6 to 9 years of relevant experience in enterprise cloud security engineering, data platform security, and Azure infrastructure.
  

• Education: B.Tech or M.Tech in Computer Science, Information Technology, Cybersecurity, or a related technical field.
  

• Core Technical Competencies:
  

  • Deep technical expertise in Azure Databricks administration, workspace networking (VNet Injection, Private Endpoints), and Unity Catalog access controls.
    

  • Strong hands-on experience with Microsoft Entra ID (Conditional Access, SCIM, PIM) and Azure Policy management.
    

  • Demonstrated proficiency in Microsoft Sentinel, KQL (Kusto Query Language), Log Analytics, and Azure Logic Apps automation.
    

• Relevant Certifications:
  

  • AZ-500: Microsoft Azure Security Technologies
    

  • Databricks Certified Data Engineer Associate
    

  • SC-200: Microsoft Security Operations Analyst
    

  • AZ-700: Designing and Implementing Azure Networking Solutions