Databricks Platform Engineer

Databricks Engineer Must Have Technical/Functional Skills Key Responsibilities Platform & Infrastructure · Design, deploy, and manage Databricks workspaces across multiple Azure subscriptions and environments (dev, staging, production) · Implement Infrastructure-as-Code (IaC) solutions using Terraform for Databricks resources · Establish and maintain security best practices, including network isolation, RBAC, and data governance · Configure and optimize cluster management, including auto-scaling policies and cost optimization strategies · Manage Databricks workspace connectivity and networking (VNets, private endpoints, firewalls) DevOps & Automation · Build CI/CD pipelines using Azure DevOps for deploying Databricks notebooks, jobs, and configurations · Implement automated testing and validation of data pipelines · Create self-service deployment frameworks and templates for development teams · Establish monitoring, alerting, and logging solutions for Databricks environments · Automate workspace provisioning and deprovisioning processes Collaboration & Enablement · Partner with data engineers and analysts to understand platform requirements · Provide technical guidance and documentation for platform usage · Develop runbooks and operational procedures for incident response · Mentor team members on Databricks best practices and Azure DevOps tools Governance & Compliance · Implement access controls and audit logging across Databricks workspaces · Ensure compliance with organizational security and data governance policies · Monitor and manage workspace usage, costs, and resource utilization · Document architecture decisions and maintain platform documentation Roles & Responsibilities Required Qualifications · 7+ years of experience in platform engineering, infrastructure engineering, or DevOps · 2+ years of hands-on experience with Databricks infrastructure setup (or Apache Spark/Delta Lake environments) · 3+ years of experience with Azure cloud services and infrastructure · 2+ years of experience with Azure DevOps (Pipelines, Repos, Artifacts) · Strong proficiency in Infrastructure-as-Code tools (Terraform Enterprise) · Solid understanding of CI/CD principles and implementation · Experience with Linux/Unix environments and scripting (Bash, Python, or similar) · Familiarity with containerization technologies (Docker, Kubernetes) Strong knowledge of networking, security, and identity management in cloud environments TCS Employee Benefits Summary: Discretionary Annual Incentive. Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans. Family Support: Maternal & Parental Leaves. Insurance Options: Auto & Home Insurance, Identity Theft Protection. Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement. Time Off: Vacation, Time Off, Sick Leave & Holidays. Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing. Salary Range: $100,000-$117,000 a year Tata Consultancy Services