Sr. Data Engineer-Azure Databricks
Scadea
Job Description
Job Description
Job Title: Sr. Data Engineer
Location: Houston, TX
Length: 6-10 months
Job Summary
We are seeking a skilled Data Engineer with extensive, hands-on expertise in Azure Databricks to design, develop, and manage our enterprise data platform. In this architecture, all core transformations will be built directly within Databricks using Databricks Workflows and Jobs. The ideal candidate must have deep experience with Unity Catalog, DevOps (CI/CD), and the ability to drive Synapse migrations into code-driven Databricks environments.
• Core Databricks Engineering: Design, build, and optimize end-to-end data pipelines entirely within Azure Databricks and Delta Lake using PySpark, SQL, or Scala.
• Orchestration & Automation: Build, schedule, and maintain complex production pipelines utilizing Databricks Workflows and Jobs as the primary orchestration engine.
• Data Legacy Migration: Lead and execute the migration of legacy data architectures into optimized Databricks solutions.
• Pipeline Conversion: Translate and refactor legacy business logic (such as converting historical Alteryx workflows) into clean, scalable Databricks notebook code.
• Data Governance: Implement and manage data access, security, schemas, and lineage across the platform using Databricks Unity Catalog.
• CI/CD & DevOps Automation: Develop, maintain, and automate Databricks workspace deployments and jobs using YAML and Classic CI/CD pipelines in Azure DevOps. Certifications (Highly Prioritized)
• Active Professional Certifications are strongly preferred, such as:
o Databricks Certified Data Engineer Associate / Professional
o Microsoft Certified: Azure Data Engineer Associate (DP-203)
o Microsoft Azure Fundamentals (AZ-900) or equivalent foundational Azure knowledge is required at a minimum.' • Alteryx Conversion: Familiarity with Alteryx workflows is a major plus, specifically to assist in reverse-engineering and converting legacy visual workflows into PySpark/SQL code. (Note: Candidates do not need to be Alteryx developers, but must be comfortable reading/migrating the ETL logic).
Vacancy posted 8 days ago
Similar jobs that could be interesting for youBased on the Sr. Data Engineer-Azure Databricks in Houston, TX vacancy
- ...Senior Data Engineer (Azure Databricks) Location: Houston, TX (Onsite) Duration: 6 10 Months Contract Interview: Virtual Technical Skills: Azure Databricks | Delta Lake | PySpark | SQL | Scala | Databricks Workflows | Databricks Jobs | Unity...DatabricksSeniorContract workLocal area
- ...team today. Position Specific Description As a Senior Data Engineer, you will be responsible for designing, building, and maintaining... ...-on with modern data technologies such as Amazon Redshift, Databricks, and cloud-based data processing frameworks to ensure...DatabricksSeniorFull timeFor contractorsWork experience placementLocal areaRelocation
$95k - $150k
...a collaborative environment? As an experienced Senior Data Engineer you will have the ability to share new ideas and collaborate... ...Design, develop and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks Write and tune PySpark / Spark SQL notebooks for large...DatabricksSeniorPermanent employment- ...Your Role The Data Services team is responsible for technical... ...the enterprise. The Data Engineer, Senior will report to the Sr Manager, Data Solutions /... ...with cloud platforms (Azure preferred) and services such... ...tools such as dbt, Snowflake, Databricks, Airflow, or Tidal. Solid...DatabricksSenior
Blue Shield of CA
Houston, TX2 days ago $80k - $133k
Job Family:Data Science & AnalysisTravel Required:Up to 10%Clearance Required:Ability... ...Will Do:Guidehouse seeks a Senior Data Engineer to design, develop, and optimize modern... ...-native data platforms leveraging AWS, Azure, Databricks, and other modern technologies.Design...DatabricksSeniorFull timeContract workFlexible hours$77k - $202k
...Description & SummaryAt PwC, our people in data and analytics engineering focus on leveraging advanced... ...Architect, GCP Data Engineer Microsoft Azure Solutions Architect, Azure Data Engineer... ...Associate, Snowflake Core, Snowflake Databricks Data Engineer Associate] is a plus-...DatabricksSeniorFull timeH1b$124k - $280k
...Description & SummaryAt PwC, our people in data and analytics engineering focus on leveraging advanced... ...Architect, GCP Data Engineer Microsoft Azure Solutions Architect, Azure Data Engineer... ...Snowflake Core, Snowflake Architect, Databricks Data Engineer Associate] is a plus-...DatabricksSeniorFull timeH1b- ...and AI is advancing Marathon Petroleum’s data ecosystem through trusted, governed and... ...innovation. The Senior Data Operations Engineer is a senior technical leader... ...emphasis on cloud-native platforms including Databricks, Azure, Cognite, streaming technologies, observability...DatabricksSeniorFull timeLocal area
$77k - $202k
Industry/SectorNot ApplicableSpecialismManaged ServicesManagement LevelSenior AssociateJob Description & SummaryAt PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics...DatabricksSeniorFull timeH1b- ...Quanex Quanex is looking for a Sr. Data Engineer to join our team located in Houston, Texas . Sr... ...architectures. Experience with cloud platforms (Azure preferred). Experience with Microsoft... ...to big data technologies (Spark, Databricks). Understanding of EDI or structured...DatabricksSeniorLocal areaRelocation
Quanex
Houston, TX5 days ago - Position Overview We are seeking a Senior Data Engineer to design, build, and operate cloud-... ...distributed processing frameworks (Spark, Databricks, Flink, Beam). Implement lakehouse/... ...patterns. Utilize cloud data services (Azure Data Lake/Databricks/Synapse; AWS S3/...DatabricksSenior
- ...Professional Technical Services Network as a contract Sr Data Engineer . This is a Houston based contract role with an... ...Experience with cloud-based data platforms (Azure preferred) Hands-on experience with Databricks, Spark, SQL, and Python Experience with Azure Data...DatabricksSeniorPermanent employmentContract work
- The Senior Data Engineer will design, implement, and sustain RectorSeal's Azure cloud data platform and medallion architecture to support the Data & BI team. This... ...lakehouse architectures and ELT/ETL workflows using Databricks, Lakeflow, Azure Data Factory, FiveTran, and...DatabricksSeniorFor contractors
$30 - $35 per hour
Data Engineer- DatabricksLocation: Remote - CST HoursEmployment Type: Full-Time ConsultantDuration: Project Based Contract Through... ...running smoothly. In this role, you’ll support a modern Databricks + Snowflake + Azure Data Factory (ADF) Enterprise Data Warehouse (EDW)...DatabricksFull timeContract workLocal areaRemote workShift work$100.79k - $160.26k
...s see what we can achieve. Together.SummaryThe Data Engineer, Solutions & Data role designs, builds, and operates... ...and implement data pipelines using Azure data technologies (e.g., Azure Data Factory, Azure Databricks, Azure Event Hubs, SSIS) to ingest, process, and...DatabricksFull timeWork at officeRemote workRelocationVisa sponsorshipRelocation package$103k - $170k
...depends on a trustworthy, well-governed data foundation that turns raw sensor... ..., decision-ready information. The Data Engineer, within the Data & AI team, designs, builds... ...linkage, and self-service analytics on our Azure, Databricks, and Snowflake stack. Success requires...DatabricksTemporary workWork at officeLocal areaRemote workFlexible hours- ...Stanford Climate Ventures is seeking a Data Engineer to join their Data & AI team in Houston, TX. In this role, you will design, build,... ...skills in Python and SQL, and experience with Apache Spark and Databricks. The position offers some hybrid work flexibility and a...Databricks
Stanford Climate Ventures
Houston, TX4 days ago - ...containerization and streaming technologies, and Python, .NET, Node, Java and SQL to design, implement and support full life-cycle data engineering projects for current and future Summary of Operations application and other Planning applications requiring large data...SeniorFull time
- ...Senior Data Engineer Location: Houston, TX Contract Length: 12+ Months Job Ref: 247245 Seeking... ...pipelines Strong experience with AWS, Azure, or GCP Experience with distributed... ...processing technologies such as Spark or Databricks Experience with data modeling, data architecture...DatabricksContract work
Strategic Staffing Solutions
Houston, TX2 days ago - ...DescriptionAs a Senior Lead Software Engineer at JPMorganChase within the... ...an integral part of an agile data engineering team. To enhance,... ...-native experience (AWS, Azure, or GCP)Ability to present... ...modern data platforms such as Databricks or SnowflakeDeep hands-on experience...DatabricksSeniorWork at office
- ...SCM Data, Innovation & analytics is seeking a Data modeler with strong Databricks expertise to design, build, and operationalize scalable Supply Chain data products that... ...processing. Hands‑on knowledge with prompt engineering Hands‑on experience in developing solutions using...Databricks
Integrated Resources
Houston, TX4 days ago - Job Family:Data Science & AnalysisTravel Required:Up to 10%Clearance... ...Do:Guidehouse seeks a Data Engineer to support the development,... ...and services in AWS and Azure.Assist with the implementation... ...data and agenciesExposure to Databricks, Spark, or large-scale data processing...DatabricksPermanent employmentFull timeInternshipWork at officeFlexible hours
- ...Overview Databricks, Python, Statistics (especially Monte Carlo simulation), building viz to support findings, along with communication with the lead data analytics and risk department would be key skills. Understanding the business (i.e., O&G) would be a plus Responsibilities...DatabricksPermanent employmentContract workLocal area
Quantum Technologies USA
Houston, TX2 days ago $99k - $232k
...Description & SummaryAt PwC, our people in data and analytics engineering focus on leveraging advanced... ...Architect, GCP Data Engineer Microsoft Azure Solutions Architect, Azure Data Engineer... ..., or Snowflake Core, Snowflake Databricks Data Engineer Associate] is a plus- Designing...DatabricksFull timeH1b- ...Experience : 8–10 years overall, with 3+ years of Azure experience. Required Skills Strong hands-on experience with Azure Data Factory (ADF) — pipelines, data flows,... ...retries. Strong experience with Azure Databricks — Notebooks, Jobs/Workflows, PySpark/Scala,...DatabricksTemporary work
- ...ADF Data Engineer Houston, TX, 77079 Hire Type: Contract Experience: 5 8 years in... ...engineering with at least 3+ years on Azure. Azure Data Factory: Pipelines, data... ...parameterization, error handling & retries. Azure Databricks: Databricks Notebooks, Jobs/Workflows,...DatabricksFull timeContract work
$120k - $155k
Join our team as a Data Solutions Engineer, where you will play a key role in designing and developing... ....• Use big data technologies like Azure Data Lake, Synapse Analytics, and Data... ...Storage• Azure Synapse Analytics, Azure Databricks, Apache Spark, Apache Airflow• Azure Logic...DatabricksFull timeFlexible hours- We are seeking a Python Software Engineer with Data Engineering experience to join a global team building a large‑scale Metering Data Platform... ..., production-grade applications rather than traditional Databricks notebook‑centric ETL development. The ideal candidate has a...Databricks
$118.7k - $218.6k
Position Summary Lead AI and Data Science Engineer II Drive the design and delivery of advanced analytics, artificial intelligence (AI... ..., clustering, and dimension reduction, in Apache Spark or Databricks environmentsExperience developing artificial intelligence (...Databricks- ...We are seeking an experienced Data Engineer to join our team on a contract basis. The ideal candidate will have 8-15 years of experience... ...supervision Experience with cloud-based data platforms (e.g. AWS, Azure, Google Cloud) Strong collaboration and communication skills,...SeniorHourly payContract work
Akeepo Technologies
Houston, TX1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Sr. Data Engineer-Azure Databricks. Be the first to apply!
Related searches
- aws data engineer Houston, TX
- big data devops engineer Houston, TX
- big data developer Houston, TX
- software data engineer Houston, TX
- data center engineer Houston, TX
- big data cloud engineer Houston, TX
- director data engineering Houston, TX
- big data engineer Houston, TX
- senior data integration developer Houston, TX
- data developer Houston, TX