Azure Databricks ETL Engineer
Diverse Lynx
Azure Databricks Etl Engineer
We are seeking an experienced Azure Databricks ETL Engineer with strong hands-on expertise in ETL development using Python and SQL to design, build, and optimize scalable data pipelines on Azure Databricks. The role focuses on delivering robust, high-performance data solutions leveraging Azure data ecosystem services for analytics, reporting, and downstream consumption.
Key Responsibilities:
- Design, develop, and maintain ETL ELT pipelines using Azure Databricks
- Build data transformation workflows using PySpark (Python) and SQL
- Ingest data from diverse Azure sources such as Azure Data Lake Storage (ADLS Gen2), Azure SQL, and external systems
- Implement and manage Delta Lake tables with appropriate partitioning and optimization
- Ensure data quality through validation checks, reconciliation, and error handling
- Optimize Spark jobs for performance, scalability, and cost efficiency
- Integrate Databricks pipelines with Azure Data Factory (ADF) or equivalent orchestration tools
- Collaborate with data architects, cloud engineers, and business stakeholders
- Support deployment, monitoring, and troubleshooting of production data pipelines
- Maintain technical documentation for ETL processes and data flows
- Follow Azure cloud security, governance, and compliance best practices
Mandatory Skills Qualifications:
- Azure Databricks
- Strong hands-on experience with Azure Databricks
- Proficiency in PySpark and Spark SQL
- Experience working with Delta Lake (ACID transactions, schema evolution)
- Hands-on experience with Azure Data Lake Storage (ADLS Gen2)
- ETL Data Engineering
- Strong Python scripting skills for ETL development
- Advanced SQL for data transformation and analytics
- Solid understanding of ETL concepts, data warehousing, and data modeling
- Experience processing large-scale, high-volume datasets
- Cloud Platform
- Good understanding of Azure data ecosystem
- Azure Data Factory (ADF)
- Azure SQL Synapse (good to have)
- Azure Key Vault (access secrets security best practices)
Good to Have Preferred:
- Experience with Azure Synapse Analytics
- Exposure to streaming pipelines (Structured Streaming, Event Hub)
- CICD exposure using Azure DevOps
- Experience with monitoring and logging (Azure Monitor, Log Analytics)
- Understanding of data security, RBAC, and Azure governance
- Experience working in Agile Scrum delivery models
Soft Skills:
- Strong analytical and problem-solving abilities
- Good communication and stakeholder coordination skills
- Ability to work independently and within distributed teams
- High ownership, attention to detail, and quality focus
Education:
- Bachelors or Masters degree in Computer Science, Information Technology, or a related field
Vacancy posted 3 hours ago
Similar jobs that could be interesting for youBased on the Azure Databricks ETL Engineer in United States vacancy
- ...Role : QA Engineer Type : Contract Location : Remote Profile of Skillset... ...Testing data solutions, including ETL, transactional and analytical models... ...Data source experience - specifically in Azure Databricks, Azure Analysis Services and SQL ServerDatabricksContract workRemote work
Exaways Corporation
United States1 day ago - ...seeking a detail-oriented and experienced ETL Engineer to join their data engineering team.... ...of cloud computing technology to include Azure, Oracle, Google, etc. ~Minimum 4-6... ...data technologies (e.g., Spark, Hadoop, Databricks) Experience with transformer-based models...Databricks
NewGen Technologies (Maryland)
Herndon, VA5 days ago $30 - $40 per hour
...ETL Quality Engineer Location: Irving, TX // Onsite Pay Rate: $30-40/Hr We are looking for... .... ~ Experience working in Microsoft Azure environments, such as Azure Data Factory... ...-solving skills. ~ Exposure to Databricks is preferred. Diverse Lynx LLC...DatabricksDiverse Lynx
Irving, TX2 days ago- ...ETL Engineer Job Description: We are currently seeking a highly skilled and experienced ETL engineer with a strong background... ...DB, and other related technologies. Experience with Azure Data Factory, Azure Databricks, and Python is also highly desirable....DatabricksWork at office3 days per week
ShiftCode Analytics
New York, NY20 hours ago - ...Feitong Buke is seeking a results-driven ETL Developer in the United States with deep expertise in Oracle Data Integrator... ...ETL pipelines, optimizing SQL queries, and working with Azure Data Factory and Databricks. Strong communication and teamwork skills are essential. #J...Databricks
Feitong Buke
New York, NY16 hours ago - ...Role : Senior Azure ETL Developer Location : Local of Richmond VA or within 50... ...services such as Azure Data\nFactory Azure Databricks Azure Data Lake Storage Azure SQL Azure... .... Work with data architects data engineers business\nanalysts and business teams...DatabricksLocal area
Kaav Inc.
Richmond, VA4 days ago - ...Azure Cloud Operations Engineer We are looking for an Azure Cloud Operations Engineer. This is a 6-12+... ...storage / ADLS2, Azure Data Factory, Azure Databricks) Create and deploy all the “... ...SQL/Oracle/etc.) Create and deploy ETL code required to move data through various...DatabricksContract workRemote work
Software Technology Inc
United States20 hours ago - ...Design and implement scalable ETL/ELT pipelines using Azure Data Factory. Build automated data... ...integrating Power BI with Azure Synapse or Databricks Experience building self-service... ...Power BI Data Analyst (PL-300) Microsoft Azure Data Engineer (DP-203)...DatabricksRemote work
The Dignify Solutions, LLC
United States1 hour ago - ...Senior Azure Data & Integration Engineer (ADF, SSIS, APIs, iPaaS) 3 Key Consulting has been retained to... ...fault-tolerant data pipelines using ETL/ELT frameworks to support ingestion,... ...and evaluate tools such as Snowflake, Databricks, and Airflow, providing...DatabricksPermanent employmentH1bShift work
3 Key Consulting
Calabasas, CA20 hours ago $40 - $45 per hour
...for an Application Development Engineer I. Must have 3-5 years of... ...Python/Unix Shell Scripting and ETL experience. 100% Remote. Must... ...cloud platforms (such as AWS or Azure), and specific technologies (like... ...Confluence, Git, JuypterHub, Databricks. Additional Skills: Migration...DatabricksContract workRemote workShift workZEEKTEK
New York, NY1 day ago- ...tasks. A degree of creativity and latitude is required. Typically reports to a supervisor or manager. Requirement: Working on cloud experience, deploying containerized applications, building data pipelines using GCP DataProc, DataFlow/Azure Databricks using pySpark....Databricks
Samprasoft
Dallas, TX1 day ago - ...Hatigen Consulting Services is looking for a highly skilled Microsoft Fabric Engineer. The successful candidate will have strong expertise in Azure Data Factory, Azure Databricks, and modern data platforms. Key responsibilities include designing and maintaining data pipelines...Databricks
Hatigen Consulting Services
New York, NY4 hours ago - ...Job Summary: We are looking for a highly skilled Azure Data Engineer to design and implement modern data solutions using the Azure cloud... ...in Azure Data Lake (Gen1/Gen2) , Data Factory , Databricks , Synapse Analytics , and Power BI , with a strong understanding...Databricks
Macpower Digital Assets Edge
Rockville, MD3 days ago - ...Job Title: Azure Automation Engineer Location: Montvale, NJ (On-Site) /Hybrid Job Description: Essential Skills: Automation Engineer... ...Environments (ASE), SQL Database, SQL Managed Instance, Azure Databricks, Synapse Analytics, Azure Functions, Logic Apps, WebApps,...Databricks
Vytwo
Prosper, TX5 days ago - ...Interlink Cloud Advisors is looking for a Data Engineer / Azure Fabric Engineer Consultant to help clients build modern, scalable data platforms... ...and Azure services (e.g., Data Factory, Synapse, Databricks). You'll integrate data from operational systems, APIs, and...DatabricksRemote work
Interlink Cloud Advisors
United States1 day ago - ...Sr Data Quality Engineer - Azure Focus Chicago, IL (loop/downtown) - 2 days work from home (WFH), 3 days onsite Summary: This... ...quality and reliability across data pipelines in our Azure, Databricks, SQL Server/Azure SQL and Medallion Architecture (Bronze,...DatabricksWork from home
1872 Consulting
Chicago, IL20 hours ago - ...impact. About the Role We’re looking for a (Senior) BI & Data Engineer to support global claims reporting for a leading reinsurance... ...organisation. You’ll work at the intersection of Power BI and Azure Databricks, building scalable data models and high-impact dashboards...Databricks
Ascent Software
New York, NY1 day ago - ...-have (Advanced-to-Expert) testing experience in the following: Desired testing experience (in order of importance): Azure Databricks Lakehouse Power BI dashboard UI automation testing Machine Learning Entity Framework SQL programming experience...Databricks
RIT Solutions
Englishtown, NJ20 hours ago - ...Title: Azure Data Architect Location: Remote... ...insights by building robust ETL/ELT pipelines and maintaining... ...ecosystem, leveraging Snowflake and Databricks to ensure our data infrastructure... ...data transformation, feature engineering, and stream processing....DatabricksRemote work
Anveta
United States1 day ago - ...Azure Data Engineer – Dynamics 365 & Power BI Hybrid Environment in Troy MI Overview / Summary The Data Engineer is responsible... ...systems including ERP and eCommerce platforms ~ Exposure to Databricks or similar tools is a plus Soft Skills Ability to...DatabricksShift work
HTC Global Services
Troy, MI20 hours ago - ...Azure Databricks Engineer Hello, Greetings! I have an exciting full-time job opportunity for an Azure Databricks Engineer role in Kennesaw, GA... ...solutions using Databricks for large-scale data processing (ETL/ELT) and analytics. Strong, practical experience using dbt...DatabricksFull time
JConnect Infotech
Kennesaw, GA20 hours ago - ...We are looking for an experienced SAP-Azure Data Engineer to join our team. The ideal candidate will... ...on leveraging Azure Synapse Analytics, Databricks, and other cloud-native technologies to... ...data integration, data modeling, and ETL/ELT processes, ensuring data quality and...DatabricksRemote work
Agentic Dream
New York, NY1 day ago - ...ContractStaffingRecruiters.com is seeking a leader with strong Databricks experience to join their fast-moving team... ...will involve designing and developing ETL processes to transform various forms of... ..., leveraging cloud technologies such as Azure SQL. Strong communication skills and the...Databricks
ContractStaffingRecruiters.com
Branford, CT4 hours ago - ...Job title AZURE DATA LEAD Job summary Design Build Test Deploy... ...Microsoft SSIS Azure Data Factory Databricks Spark Domain Skills-... ...including Azure Cloud. Build ETL pipelines using workflow management... ...years of experience in data engineering data warehousing or related field...DatabricksShift work
3B Staffing LLC
Chicago, IL4 days ago - ...Databricks Engineers / Architects We are looking for experienced Databricks Engineers / Architects... ...in data pipeline development (ETL/ELT) Solid understanding of data engineering... ...Experience with cloud platforms (AWS / Azure / GCP) Knowledge of performance...DatabricksRemote work
ISHIR
United States15 hours ago - ...Spark Engineer Opportunity They have been trying to hire offshore for a while but quality of candidates... ...Experience Spark Engineer working mostly in Azure and Databricks working to establish new data pipelines for ETL process. Needs candidates as senior as he can get...DatabricksRemote workShift work
Software Technology Inc
United States1 day ago - ...Job Description: Strong hands-on experience with Azure Data Factory, Databricks and Azure Cloud services Good knowledge in Blob Storage... ...Azure Data Factory (ADF) pipelines to automate end to end ETL workflows and enabling high performance data movement through...Databricks
Prophecy Technologies
Dallas, TX20 hours ago $204.85k - $265.1k
...Senior Systems Engineer, Data Management Our field sales professionals... ...platform (e.g., Snowflake, Databricks, BigQuery, Redshift, Cloudera,... ..., data lakes/lakehouse, and ETL/ELT concepts (staging, modeling... ...one major public cloud (AWS, Azure, or GCP) and one or more leading...DatabricksFull timeDell
Worcester, MAa month ago$60 - $61.76 per hour
.... GBaMS ReqID:10656355 Data Engineer - GMI XMX Engineering Ops Job Description... ...hands-on experience in: o Azure Data Factory (ADF) o Azure Databricks o PySpark / Spark SQL o SQL (... ...supported: o Production-grade ETL/ELT pipelines o Large-scale...DatabricksLocal areaVarite
San Jose, CA2 days ago- ...years overall experience in data engineering or related fields. • 3–5 years hands-on experience with Databricks and Spark. • Strong... ...techniques. • Experience with ETL processes, data modeling, and... ...performing data operations within Azure environment and can work in a...Databricks
Novia Infotech
Raleigh, NC20 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Azure Databricks ETL Engineer. Be the first to apply!
Related searches
- azure infrastructure architect United States
- azure data engineer United States
- azure specialist United States
- devops engineer azure United States
- azure infrastructure engineer United States
- azure security engineer United States
- cloud engineer azure United States
- azure solution architect United States
- azure developer United States
- azure data architect United States