Azure Databricks Operation Lead

Job Description


Client is seeking an Azure Databricks operations lead . In this role, you will enable digital transformation for our clients in a global delivery model, research on technologies independently, recommend appropriate solutions and contribute to technology-specific best practices and standards. You will provide inputs for identifying best-fit architectural solutions for one or more projects; develop design of applications, provide regular support and guidance to project teams on complex coding and issue resolution with the objective of ensuring best-fit and high-quality technical solutions within the area of technology specialization and in compliance with guidelines, policies and norms of client. You will be part of a learning culture, where teamwork and collaboration are encouraged, excellence is rewarded, and diversity is respected and valued.



Required Qualifications:

• Candidate must be located within commuting distance of Plano, TX or be willing to relocate to the area. This position may require travel within the US
• Bachelor's degree or foreign equivalent required from an accredited institution. Will also consider three years of progressive experience in the specialty in lieu of every year of education
• At least 4 years of experience with Information Technology
• Proven years of hands-on experience in Azure, Databricks
• Proven hands-on experience in Azure services support (Azure Databricks, Data Pipeline, Azure Synapse, Symentic models)
• Proficiency in data engineering tools: Spark, Kafka, Airflow, SQL, Python
• Familiarity with vector search and semantic retrieval for retail use cases
• Expertise in data privacy and compliance in consumer-facing environments
• Good experience in end-to-end implementation of data warehouse and data marts
• Strong understanding of Data Warehousing concepts of Data Vault, Dimensional modelling, OLAP design
• Extensive experience with CI/CD platforms such as GitLab CI, GitHub Actions, Azure Pipelines, and Jenkins
• Experience designing and implementing complex solutions for distributed systems
• Experience in leading and mentoring teams

• Strong understanding of retail or CPG data domains
• Experience in Relational Modeling, Dimensional Modeling and Modeling of Unstructured Data
• Familiarity with log and monitoring solutions
• Experience in utilizing development containers, unit testing/code quality review/linting - standard best practice
• Good understanding of Agile software development frameworks
• Strong communication and Analytical skills
• Ability to work in teams in a diverse, multi-stakeholder environment comprising of Business and Technology teams
• Experience and desire to work in a global delivery environment
• Experience and detailed knowledge with Master Data Management, ETL, Data Quality, metadata management, data profiling, micro-batches, streaming data loads