Senior Databricks Data Engineer
Prophecy Technologies
Role Overview: This role focuses on designing and developing robust, scalable ETL/ELT pipelines, with a strong emphasis on Databricks and PySpark. The successful candidate will be responsible for building both batch and real-time streaming data ingestion and transformation frameworks, implementing advanced data architectures, and optimizing data processing workflows for performance and efficiency. Key Responsibilities:
- Design and develop scalable ETL/ELT pipelines utilizing Databricks and PySpark.
- Build and maintain batch and real-time streaming ingestion frameworks.
- Develop reusable ingestion and transformation frameworks to ensure consistency and efficiency.
- Implement the Medallion architecture (Bronze, Silver, Gold layers) for data organization.
- Develop incremental and Change Data Capture (CDC)-based ingestion pipelines.
- Design and implement real-time streaming pipelines using technologies like Kafka and Structured Streaming.
- Optimize Spark jobs, SQL queries, and streaming pipelines for enhanced performance.
- Implement Delta Lake-based ingestion and transformation frameworks.
- Tune partitioning, caching, and Spark execution strategies to maximize throughput.
- Strong proficiency in SQL and data modeling.
- Extensive experience with cloud platforms and distributed systems.
- Familiarity with CI/CD pipelines and DevOps practices.
- Expertise in MySQL.
- Proficiency with Databricks.
- Strong skills in PySpark.
- Minimum 10 years of overall work experience.
- Minimum 5 years of experience specifically as a Data Engineer.
- 8-10+ years of relevant experience is required.
Vacancy posted 6 hours ago
Similar jobs that could be interesting for youBased on the Senior Databricks Data Engineer in Washington DC vacancy
- ...solutions provider in Arlington, Virginia, is seeking a senior-level professional in Data Engineering. The role involves managing system inventories,... ...engineering and visualization, with hands-on experience using Databricks and GitHub. This remote-first position requires the...DatabricksSeniorRemote work
Nyla Technology Solutions
Arlington, VA4 days ago - ...requirements discussion with business analyst and business users Python/SQL Server/Snowflake/Databricks application development and system design Develop and maintain data models and schemas to support data integration and analysis Implement data quality and...DatabricksSenior
Saxon Global
Washington DC2 days ago - ...Fire Up is seeking a Senior Data Engineer in Arlington, Virginia, to enhance data pipelines for analytical purposes. The role involves creating... ...SQL, strong Python skills, and proficiency with tools like Databricks and Workday. We offer competitive benefits, including...DatabricksSenior
Fire Up
Arlington, VA1 day ago - ...Senior Informatica Data Engineer Randstad is seeking a Senior Informatica Data Engineer for a client in Washington, DC. This role focuses on... ...aligned with business goals, leveraging advanced tools like Databricks, PySpark, and SQL for high-performance data operations....DatabricksSenior
Samprasoft
Washington DC2 days ago $180k - $200k
A global investment firm in Washington, DC is looking for a Data Engineer to create robust data pipelines for their ML team. The ideal candidate... ...and maintain scalable data solutions, work with tools like Databricks, and lead projects involving AI technologies. This position...DatabricksSenior- ...Master Data Management Architect Lead the design, development, and deployment of... ...integration pipelines leveraging modern data engineering tools and practices. Develop,... ...data processing solutions using Kafka, Databricks, Snowflake, Azure Data Factory (ADF), and...DatabricksSeniorFlexible hours
Saxon Global
Washington DC2 days ago - ...candidate to join our talented Team. Job Title: Senior IT Data Engineer Locations: NW Washington, DC Job Description:... ...and big data systems including: Amazon Aurora, Snowflake, Databricks, Amazon Redshift. Ability to use infrastructure as code...DatabricksSenior
Ampcus
Washington DC3 days ago $116.9k - $243.1k
A prominent technology firm is seeking a Senior Data Engineer to enhance their data platforms using Databricks and AWS. This role involves optimizing data infrastructure, collaborating with top professionals in an agile environment, and requires 5+ years of experience in...DatabricksSeniorRemote job- iTech Media is looking for a Senior Data Engineer in Washington D.C. to support mission-critical programs focused on data analytics and integration... ...possess experience with enterprise analytics tools like Databricks and Tableau, and must have an active DoD Secret clearance....DatabricksSeniorRemote job
$110k - $180k
iTech Media is looking for a skilled Data Engineer to support a DoD data platform and analytics environment. The candidate will design,... ...4 years of experience in data engineering, strong skills in Databricks, PySpark, Python, and SQL. The role offers a competitive salary...DatabricksSenior- Lumiture is seeking a Senior Data Engineer to join its innovative team in Washington, DC. You will design and build a robust data analytics platform in Databricks to support product analysis and enhance data management across the organization. Ideal candidates will have...DatabricksSeniorFlexible hours
- Makpar is looking for a Senior Data Engineer in Washington, D.C. This role involves building and orchestrating data pipelines using Databricks, designing ETL processes, and translating business needs into SQL queries. A Bachelor’s or Master’s degree specializing in computer...DatabricksSenior
$125k - $180k
SMX is seeking a Senior Data Engineer to join our team in Arlington, Virginia, supporting a Federal customer. The ideal candidate will have... ...analytics solutions, particularly using Palantir Foundry and Databricks. Responsibilities include leading a team of developers,...DatabricksSenior$130k - $165k
Capital Technology Group is seeking a Data Engineer to join their team in Washington, DC. The role involves designing, building, and maintaining... ...in data engineering, a Bachelor's degree, and expertise in Databricks and AWS. The position offers a competitive salary between $13...DatabricksSeniorRemote job- ...Systems & Solutions in Arlington, Virginia is looking for a Senior Data Engineer to support a Modernized Data Science Platform. The ideal... ...pipelines and models using platforms like Palantir Foundry or Databricks. This role demands technical leadership and hands-on...DatabricksSenior
- IntelliTech seeks a Senior Data Engineer to support mission-critical programs focused on data integration and analytics. The ideal candidate... ...with Python, SQL, and enterprise analytics platforms like Databricks and Tableau. Responsibilities include designing data pipelines...DatabricksSeniorRemote job
- DataTrek in Arlington, Virginia is seeking a Senior Analytics Engineer for a one-year contract. This role focuses on data engineering, building dashboards, and BI solutions... ...over 3 years of experience and expertise in Databricks, Palantir Foundry, SQL, and Python....DatabricksSeniorContract work
- iTech Media is seeking a Senior Data Engineer to support data integration and analytics in secure federal environments. You will design and... ...candidate has strong experience with Python, SQL, and tools like Databricks and Tableau, along with an active DoD Secret clearance....DatabricksSenior
- ...Senior Data Engineer Location: Hybrid in Washington, DC Number of Openings: 1 Interview Type: Webcam Nearby states candidates only... ...reporting while building the future-state Azure Synapse and Databricks Lakehouse. Key Responsibilities: Legacy...DatabricksSeniorLocal area
InterSources
Washington DC1 day ago - ...years of experience. Medicaid/Medicare data structures (MMIS, EDI 837/835, claims... ...(Bronze/Silver/Gold) using Azure Databricks (PySpark/SQL). Lead transition of legacy... ...technology or service projects. Full system engineering lifecycle. ITIL practices. End...DatabricksSeniorLocal area
Macpower Digital Assets Edge
Washington DC2 days ago $100k - $160k
...JOB TITLE: Senior Data Engineer (Secret Clearance Required) LOCATION: DC Metro Area, Fayetteville NC, Tampa FL DURATION: Permanent... ...Experience with distributed data/computing tools including Spark, Databricks, Hadoop, Hive, AWS EMR, or Kafka Experience with data...DatabricksSeniorHourly payPermanent employmentFull timeUrsus Inc
Washington DC4 days ago- ...Makpar has an exciting opportunity for a Senior Data Engineer to join our growing team.his role requires expertise in Databricks, SQL, and ETL pipelines, with strong experience building scalable data frameworks to support enterprise decision-making. We are seeking...DatabricksSeniorContract workFor subcontractorWork from homeFlexible hours
iQuasar
Washington DC13 days ago - EY is seeking a Senior Consultant Data Engineer in Arlington, Virginia, to design and build cloud-based analytics solutions leveraging Databricks. The ideal candidate will have over 5 years of experience in data engineering, strong expertise in cloud data solutions, and...DatabricksSenior
- Analytica is seeking a Technical Lead - Data Engineering in Bethesda, MD. This hybrid role requires a senior engineer with 8+ years of experience in building and maintaining... ...ETL/ELT pipelines and strong expertise in Databricks and AWS. Candidates should have a Bachelor’s...DatabricksSenior
$77.92k - $125.73k
...What We Can Achieve Together The Senior Data Engineer focuses on the development of analytics-oriented data models and data ingestion pipelines... ...complex problems. Desired Experience Experience using Databricks for data ingestion and curation. Experience with Workday financial...DatabricksSeniorLocal areaFlexible hoursFire Up
Arlington, VA1 day ago- ...combine functional and technical expertise in cloud engineering, data management, cybersecurity and emerging... .... Makpar has an exciting opportunity for a Senior Data Engineer. This role requires expertise in Databricks, SQL, and ETL pipelines , with strong experience...DatabricksSeniorStart working todayFlexible hours
Makpar Corporation
Washington DC3 days ago - ...Senior Data Engineer General Information Requisition # 683 Locations USA-VA-Arlington Posting Date 03/20/2026 Security... ...used in analytics development. ~ Experience with Databricks, PostgreSQL, or comparable enterprise analytics platforms....DatabricksSeniorFull timeWork at officeLocal areaRemote work
Elder Research
Arlington, VA1 day ago - ...next-generation solutions that fuse AI, data, and modern engineering to power intelligent experiences... ...meaningful impact. We’re looking for a Senior Data Engineer to play a pivotal role... ...a robust data analytics platform in Databricks that will support a skilled team of data...DatabricksSeniorImmediate start
- ...General Job Description: Designs, develops, and supports enterprise data integration and analytics environments supporting operational... ...Apache Spark, Python, SQL or relational databases, Hadoop, Databricks and data pipeline management in Spark for batch and streaming....DatabricksSenior
- ...Senior Data Engineer (Labs) Ignite Digital enables national security agencies to accelerate decisions, elevate operational outcomes and... ..., or sustainment organizations. Experience with Snowflake, Databricks, or other data AI / analytic platforms. Experience working...DatabricksSeniorFor contractorsWork at office1 day per week
Ignite Digital Services
Washington DC1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Databricks Data Engineer. Be the first to apply!
Related searches
- staff data engineer Washington DC
- data visualization developer Washington DC
- data science developer Washington DC
- senior data center engineer Washington DC
- sr information security engineer Washington DC
- IT data engineer Washington DC
- junior big data engineer Washington DC
- entry level big data engineer Washington DC
- data engineer contract Washington DC
- aws data engineer Washington DC