Data Bricks Lead Engineer

Job Title: Databricks Lead Engineer

Company: Quanta Systems Technologies LLP

Employment Type: Contract

Location: Washington DC

Experience: 10+ Years

Job Description:

We are seeking a Databricks Engineer to build of a new Databricks solution that ingests source files and constructs a semantic/serving layer that powers a Power BI dashboard. A semantic layer already exists; the new solution must conform to the same structure so that a replicated Power BI solution works seamlessly against it with no rework to reports, measures, or relationships. The engineer owns the Databricks data and semantic layer and partners with a dedicated Power BI developer/team, who own the dashboard build, to deliver the end solution.

Key Responsibilities

• Lead the design and delivery of file-ingestion pipelines in Azure Databricks (batch and/or streaming) that land and process incoming files into Delta Lake using a Medallion architecture (Bronze → Silver → Gold).
• Design the Gold/serving layer to replicate the existing semantic model exactly — matching table names, grain, column names, data types, keys, and relationships so the existing Power BI dataset can be repointed without redesign.
• Expose the model through Databricks SQL endpoints (Direct Query) so the Power BI developer can connect and validate the dashboard with no redesign.
• Collaborate with the Power BI developer/team as the data-side owner — align on the semantic contract (star schema, measures, RLS expectations), support their dashboard build, and resolve any data or model issues to land the end deliverable.
• Document the semantic mapping and work with the Power BI team to reconcile the replicated dashboard's output against the existing solution to prove parity.

Required Skills

• AWS Databricks, Delta Lake, Medallion architecture, Unity Catalog
• Python and strong SQL
• Databricks SQL endpoints and a working understanding of Power BI semantic modeling (star schema, DAX, incremental refresh, RLS) sufficient to define and hand off the semantic contract
• Databricks Certified Data Engineer (Professional preferred)

#Immediatehiring #DatabricksEngineer #Databricksjobs #DataEngineer #AzureDatabricks #AWSDatabricks #DeltaLake #MedallionArchitecture #UnityCatalog#PythonDeveloper #SQLDeveloper #hiringnow #ContractJobs #Raritan_NJ #QuantaSystemsTechnologies

Company’s Website :

Company’s LinkedIn Profile :

DM me or share your resume at [ View email address on click.appcast.io .]