Databricks Engineer

Job Description

1. Deliverables

Expected deliverables may include (but are not limited to):

· Updated Databricks architecture designs and diagrams

· Optimized data pipelines and notebook implementations

· AI/BI dashboards and semantic models

· Configured and functional Genie Rooms

· Databricks chatbot proof-of-concept or production implementation

· Technical documentation and runbooks

· Knowledge transfer sessions and walkthroughs

1. Roles and Responsibilities

Databricks Engineer / Developer

· Provide hands-on development, configuration, and optimization

· Serve as a technical advisor on Databricks architecture and AI enablement

· Collaborate with data engineering, analytics, and business stakeholders

· Ensure deliverables meet quality, security, and performance standards

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to View email address on click.appcast.io learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy:

Skills and Requirements

The Databricks Engineer / Developer must demonstrate strong hands-on experience with the following core capabilities:

· Proficiency in SQL, YAML, and Databricks Notebooks for data engineering, analytics, and automation

· Deep understanding and practical application of Medallion Architecture (Bronze, Silver, Gold patterns)

· Experience designing, managing, and optimizing Delta Lake tables

· Strong knowledge of Unity Catalog, including data governance, permissions, lineage, and security best practices

· Hands-on experience building and maintaining AI/BI Dashboards and Reports within Databricks

· Experience configuring and enabling Databricks Genie Rooms

· Practical experience with Natural Language to SQL capabilities and semantic modeling to support business users

· Strong proficiency in Python for data engineering, analytics, and AI workloads

3.2 Version Control, CI/CD, and Environment Management

The Engineer must have demonstrated experience with modern DevOps practices in Databricks environments, including:

· Implementing and managing Databricks Asset Bundles using GitHub

· Applying source control best practices for notebooks, jobs, and configuration

· Promoting and managing solutions across development, UAT, and production environments

· Supporting CI/CD pipelines for Databricks workloads, ensuring repeatability and governance · Experience with Databricks Agents / Agent Bricks

· Designing and implementing agentic workflows for AI-driven automation and decision-making

· Familiarity with retrieval-augmented generation (RAG) or AI orchestration patterns within Databricks