Senior Data Engineer/ Databricks, PySpark, Python
EPAM Systems
We are looking for an experienced Senior Data Engineer / Databricks Engineer to join a project focused on collecting, processing, and analyzing vehicle telemetry data for a major European automotive manufacturer. The platform operates at a significant scale, processing tens of terabytes of telemetry data in Azure Databricks and running approximately 200 Databricks jobs. The solution is currently in the production maintenance and evolution phase and is deployed across multiple regions, including Europe, North America, and China, with an additional deployment in China planned in the near future. A key focus of the role will be ensuring that the system can scale reliably while keeping infrastructure and operational costs predictable. Responsibilities Monitor the health, stability, and performance of the production data platform Investigate and resolve performance issues and production incidents Analyze Databricks workloads and identify performance bottlenecks Optimize PySpark workloads and Databricks jobs for performance, reliability, and cost efficiency Support and improve approximately 200 existing Databricks jobs and associated data pipelines Prepare the platform for increasing data volumes and workloads as additional vehicle platforms are onboarded Design and implement scaling strategies with a strong focus on predictable infrastructure and operational costs Track and optimize Databricks and Azure resource consumption Improve platform observability, monitoring, alerting, and operational processes Ensure consistency across multiple regional deployments in Europe, North America, and China Identify opportunities for technical improvements, automation, and reduction of operational overhead Requirements 3+ years of hands-on experience in data engineering Expertise in Azure Databricks Proficiency in PySpark and distributed data processing Advanced skills in Python development Understanding of performance optimization for large-scale Spark workloads Background in operating and troubleshooting production data platforms Knowledge of monitoring, observability, and capacity planning combined with cloud cost optimization Ability to analyze existing systems, identify bottlenecks, and propose pragmatic improvements English proficiency at an Upper-Intermediate level (B2) or higher Nice to have Familiarity with Microsoft Azure services and cloud infrastructure Experience building and maintaining CI/CD pipelines using GitHub Actions Exposure to large-scale telemetry, IoT, or automotive data Background in managing Databricks environments with a large number of scheduled jobs and pipelines Showcase of multi-region or geographically distributed cloud deployments
- We are looking for a Senior Data Software Engineer to join a team that builds and maintains... ...data exchange solutions, Databricks-based data sharing... ...services and integrations using Python Develop and support data pipelines using PySpark and Apache Airflow (MWAA) Deploy...DatabricksSeniorPython
- ...разработки масштабируемых конвейеров в Databricks с использованием Python и PySpark. Вы будете работать с бизнес- и... ...Требуется минимум 3 года опыта в data engineering, знание Python, Databricks и... ...английским языком. Предпочтительно senior/tech lead уровень; удаленная...DatabricksSeniorPythonRemote job
$55 - $62 per hour
...Overview: Req ID: 394117 NTT DATA strives to hire... ...Client is currently seeking a Senior Data Engineering Support Engineer to join their... ...expert-level proficiency in Databricks and PySpark. In this role, you will be... ...frameworks and scripts in Python/PySpark to automate data...DatabricksSeniorPythonTemporary workRemote workFlexible hours- ...0567Reference26-01842Senior PySpark and Delta Data Engineer Job Summary This is the deep build role. The Senior PySpark and Delta Data Engineer... ...Advanced SQL and strong Python software engineering fundamentals... ...orchestration through Databricks Workflows, Airflow, or an equivalent...DatabricksSeniorPythonContract work
- ...Job Family : Data Science Consulting Travel Required :... ...Guidehouse is seeking a Data Engineer to join our Defense and Security... ...sources tools such as SQL, Python, and PySpark to formulate or recognize... ...quality of data ingested into Databricks Organizing and leading...DatabricksSeniorPython
$110k - $130k
...We are looking for an experienced Senior Data Engineer with strong expertise in Databricks, Python, PySpark, Apache Spark, and large-scale data engineering. The ideal candidate will be responsible...DatabricksSeniorPythonFull time- ...Senior AWS Data Engineer | Onsite, Washington DC Area | TS/SCI Required About... ...Develop processing jobs in Python, Spark, and SQL that are... ...and SQL, plus working Spark/PySpark experience ~ Track record... ...orchestration Exposure to Databricks, Snowflake, or Informatica...DatabricksSeniorPythonFull timeFor contractors
- ...Senior Data Engineer – PySpark & Databricks Location: Charlotte, North Carolina (Hybrid)// Need only locals Experience: 10+ Years Job Type: Contract... ...strong hands-on expertise in PySpark, Databricks, Python, and SQL to design, develop, and maintain scalable...DatabricksSeniorPythonContract workLocal area
- ...investments in its enterprise data and AI capabilities.... ...an opportunity for a senior-level engineer to help shape the... ...including Databricks, Unity Catalog, AI-ready... ...Delta Lake, Spark, and PySpark Creating robust data... ...PySpark Delta Lake Python SQL Azure Data...DatabricksSeniorPython
- ...Responsibilities & Skills End-to-End Data QA Source-to-Target Mapping... ...Schema Drift Validation Pyspark Python Azure Databricks Delta Lake DLT Devops... ...Science, Statistics, Mathematics, Engineering, Business Analytics. Preferred: Master...DatabricksSeniorPython
- We are seeking a Senior Data Software Engineer with Databricks to lead the design, development, and maintenance of... ...Databricks, Delta Lake and Apache Spark (PySpark) Skills in dbt and Databricks... ...Catalog Advanced SQL and strong Python skills Solid data modeling skills,...DatabricksSeniorPython
- ...Our partner is looking for a Senior Data Engineer based in the United States.... ...data loaders using Go and Python. Design and evolve... ...idempotent data pipelines using Databricks to process billions of... ...with Databricks, including PySpark, Delta Lake, and Unity Catalog...DatabricksSeniorPythonFull timeFlexible hours
- ...Strong PySpark experience is required. Candidates must be visa-... ...local to Texas Job Title: Data Engineer – PySpark Location: Dallas... ...hands-on experience with Python and PySpark . ~ Extensive... ...Experience with technologies such as Databricks, Delta Lake, Hive, or...DatabricksPythonLocal area
- ...Platform Manager is a senior technical... ...scalable enterprise data platforms within a... ...driving enterprise data engineering, analytics,... ...architecting scalable Azure Databricks solutions, and... ...pipelines using PySpark, Scala, SQL, and Azure... ...in SQL, Python/PySpark, Scala, and...DatabricksSeniorPythonRemote work
$110k - $156k
...About the Company The Senior Developer, Data Engineering will lead the design,... ...with Microsoft Fabric or Databricks. ~ Proven experience leading... ...platform team. ~ Strong Python skills for data engineering... ...development. ~ Proficiency in PySpark, Pandas, or Polars...DatabricksSeniorPython$60 - $65 per hour
...help build analytics that engineers can trust. This role focuses on designing data pipelines and... ...qualifications Advanced SQL Python plus data analysis and... ...Tooling experience with Databricks and AWS Observability and... ...frameworks such as Pandas / PySpark Machine learning or...DatabricksSeniorPythonHourly pay- ..., and governance mechanisms across enterprise data platforms. Key Responsibilities Develop... ...handling. Required Technical Skills Data Engineering Python SQL PySpark Scala Data Platforms Databricks Delta Lake Integration REST APIs Git...DatabricksSeniorPython
- ...Description Job Description Sr. Data Engineer for Cincinnati OH Will... ...cloud technologies like PySpark, Spark SQL, Azure Data Lake... ...Azure Analytical Services, Databricks and Azure Data Explorer. Will... ...applications. using C#.NET, Python, VB.NET, SQL, KQL, PowerShell...DatabricksSeniorPython
$11k - $20k
...Job Description Job Description Job Title: Senior Databricks Data Engineer Location-Type: Onsite, Santa Monica, CA (office relocation... ...experience in production environments. •Strong proficiency in Python (PySpark) and Spark SQL for large-scale data transformation. •...DatabricksSeniorPythonPermanent employmentWork at officeImmediate startRelocation- ...solutions to our clients world-wide. Job Title: Senior Analytic Data Engineer (Python/Power BI) Location: Remote Duration: 6 Months... ...tools (e.g., Apache Spark(Dataproc), GCP Tools, Databricks, PySpark). 3 Years of experience building ETL and Data Integration...DatabricksSeniorPythonTemporary workRemote workShift work
- ...Advisory.KPMG is currently seeking a Sr. Associate, Data Science Engineer to join our Consulting practice.Responsibilities:Translate... ..., deployment, and MLOps, with hands-on expertise in Python, PySpark, SQL, Databricks, and Snowflake, while partnering with clients and...DatabricksSeniorPythonH1bLocal area
- ...Job Description Job Description . Job Title: Senior Azure Data Engineer Location: Redmond, WA 98052 Work Setup:... ...engineering and strong hands-on expertise in Python, SQL, Apache Spark, PySpark, and Azure Databricks . The candidate will be responsible for developing...DatabricksSeniorPython
- ...revenue-generating software and data engineering solutions for clients. With... ...We are looking for a Senior Databricks Data Engineer to design, build... ..., you will leverage PySpark, SQL, and Unity Catalog to... ...Spaces. Familiarity with Python API development using FastAPI...DatabricksSeniorPythonFull timeRemote work
$140k - $155k
...Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of... ...and experienced Senior Data Engineer to join our... ...and business users. Python/SQL Server/Snowflake/Databricks application development... ..., ETL, CI/CD. Plus: PySpark, Financial systems/capital...DatabricksSeniorPythonFull timeTemporary workFlexible hours$197.3k - $225.1k
Data Engineer 4 (Python, PySpark, AWS) Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems... ..., and cloud-based data warehousing platforms including Databricks and Snowflake Share your passion for staying on top...DatabricksPythonFull timePart timeInternshipH1bLocal area- ...Description Job Description Senior Architect Data Engineering Role Overview We are... ...modern stack centered on Databricks, dbt, and Apache Airflow.... ...processing patterns using PySpark, Delta Live Tables (DLT),... ...: Elite proficiency in Python (PySpark) and advanced SQL...DatabricksSeniorPython
$77.6k - $176k
...Job Number: R0252244 Databricks Data Engineer, Senior The Opportunity: As a senior Databricks Data Engineer, you will be hands-on designing... ..., and Unity Catalog ~3+ years of experience writing Python and PySpark code for ETL scripts and custom transformations ~...DatabricksSeniorPythonFull timeContract workPart timeLocal areaRemote work$146.74k - $233.66k
...Job Description: Senior Manager, Data Engineering Drive Operational Excellence. At Ashurst Perkins Coie... ...-on technical excellence across Databricks, dimensional modeling, pipelines,... ...Advanced, working knowledge of SQL, Python, and PySpark for data engineering and...DatabricksSeniorPython$129k - $192k
...The Team The Ads Data Engineering team's purpose is to design... ...is growing. As a Senior Ads Data Engineer, you... ...own that platform on Databricks: making the existing pipelines... ...with Apache Spark (PySpark and/or Scala),... ...~ Strong SQL and Python (or Scala) skills for...DatabricksSeniorPythonFull timeSummer workRemote workFlexible hours- ...Job Duties:-• Experience: 8–10 years in Data Engineering and Data Analysis. • Informatica... ...design, development, and optimization. • PySpark Development: Advanced skills in PySpark... ...• Programming Skills: Proficiency in Python and Kafka for streaming and batch data...SeniorPython
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer/ Databricks, PySpark, Python. Be the first to apply!
- staff data engineer United States
- junior data engineer United States
- bi data engineer United States
- senior cloud data engineer United States
- IT data engineer United States
- big data cloud engineer United States
- junior big data engineer United States
- data engineering intern summer United States
- data engineer analytics United States
- sr data engineer United States







