Senior Data Engineer - Spark / Scala
Astyra Corporation
Job Description
Job Description
Astyra’s client is seeking a hands-on Senior Data Engineer to support a large-scale data and technology modernization initiative. This position will focus on modernizing legacy ETL workloads, developing Spark-based data pipelines, and ensuring data accuracy and consistency as workloads transition to modern platforms.
The ideal candidate has deep hands-on experience with Spark and Scala and a strong background designing, building, optimizing, and validating enterprise data pipelines.
- Design, build, and maintain scalable data pipelines using Spark and Scala
- Support the migration of legacy ETL workloads into a modern Spark-based data framework
- Develop and enhance data transformation and processing solutions
- Perform data comparison, reconciliation, and validation between legacy and target platforms
- Identify and troubleshoot data discrepancies throughout the migration process
- Develop data quality checks and controls to ensure completeness and accuracy
- Optimize Spark jobs and pipelines for performance, scalability, and reliability
- Work closely with data engineers, architects, and other technical teams throughout the modernization effort
- Support testing, deployment, troubleshooting, and ongoing maintenance of production data pipelines
- Contribute to broader feature engineering and data modernization initiatives
- 7+ years of experience in data engineering or a related technical discipline
- Strong hands-on experience designing and developing enterprise data pipelines
- Advanced experience with Apache Spark
- Strong development experience with Scala
- Experience with ETL development, modernization, and migration initiatives
- Strong understanding of data transformation, data quality, and reconciliation
- Experience comparing and validating data across legacy and modernized systems
- Experience developing and supporting production data-processing workloads
- Familiarity with modern CI/CD and software development practices
- Strong troubleshooting and problem-solving skills
- Experience migrating legacy ETL platforms to Spark-based architectures
- Experience supporting large-scale technology or data modernization initiatives
- Experience within financial services or other highly regulated environments
- Python experience
- Exposure to Generative AI or AI-assisted development and modernization tools
- Previous experience within a large enterprise engineering environment
#AC
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer - Spark / Scala in Richmond, VA vacancy
$133k - $172k
...Senior Data Engineer Job Description CoStar Group (NASDAQ: CSGP) is a leading global provider... ...Data Vault), plus proficiency in Python, Scala, Java, or C#/.NET ~ Production... ...or lakehouse platform (e.g., Databricks/Spark, Snowflake, BigQuery, Redshift, Synapse...SeniorHourly payFull timeWork at officeWork from homeMonday to ThursdayCoStar Group
Richmond, VA3 hours ago$229.9k - $262.4k
...Senior Lead Data Engineer (Bank Tech) Do you love building and pioneering in the technology space?... ...Utilize programming languages like Java, Scala, Python and Open Source RDBMS and... ...(MapReduce, Hadoop, Hive, EMR, Kafka, Spark, Gurobi, or MySQL) ~4+ year experience...SeniorFull timePart timeInternshipLocal areaCapital One
Richmond, VA22 hours ago- Position DescriptionWe are seeking a Senior Data Engineer to lead the design and implementation of... ...pipelines using Azure Databricks (PySpark/Spark SQL) to ingest, transform, and process... ...Databricks, including Spark (PySpark/Scala), Delta Lake, and cluster/job optimization...SeniorWork at office
- ...Position Description We are seeking a Senior Data Engineer to lead the design and implementation of... ...pipelines using Azure Databricks (PySpark/Spark SQL) to ingest, transform, and process... ...Databricks, including Spark (PySpark/Scala), Delta Lake, and cluster/job optimization...SeniorWork experience placementWork at office
$144.2k - $164.6k
...customer needs. We are seeking Data Engineers who are passionate about... ...data drives decisions. As a Senior Data Engineer, you will design... ...technologies such as Python, Spark, Snowflake, Databricks and AWS... ...programming languages like Java, Scala, Python and Open Source RDBMS...SeniorFull timePart timeInternshipLocal area- ...REQUIREMENT Job Title: Senior Data Engineer Location: Richmond, VA Duration: 12 Months Pls share profile... ...will have hands-on experience with AWS, Snowflake, Python, Spark, CICD pipelines, DataOps , and a strong understanding of credit...SeniorContract work
Maintec Technologies
Richmond, VA3 days ago $140k - $170k
Senior Data Engineer Chicago, VA Hybrid Full Time $140k - $170k Join a growing, data-driven organization that is investing heavily in its modern... ...environment Strong expertise with PySpark and Apache Spark Advanced SQL development and query optimization skills Experience...SeniorFull time- Sr Data Engineer Role: Sr Data Engineer MoI: 2 Video Notes from Manager: Candidates must be located in San Antonio, TX, VA, DC, NY- will work... ...building a code framework using Pyspark and debugged using spark logs for performance tuning. The expected output is a highly performant...SeniorContract workTemporary workRemote work
- ...Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are... ...is looking for an experienced Lead Data Engineer to design, build, and optimize scalable data... ...high-volume data processing solutions using Spark, Kafka, Hadoop, EMR, or equivalent...SeniorLocal area
Tiger Analytics Inc.
Richmond, VAa month ago - Senior Data Engineer Fully Remote•United States Job Type Full-time Description Overview Tanaq Technical Services (TTS), a division of St... ..., S3, Redshift, Data Factory ETL/ELT frameworks: Airflow, Spark, Databricks Data modeling, schema design, and optimization...SeniorFull timeContract workLocal areaRemote work
$90k - $200k
...DescriptionCapTech Machine Learning Engineers are responsible for... ...and implementing data-driven solutions for... ...solutions using Python, Scala, or similar languages.... ...Proficiency leveraging SQL, Spark, NoSQL, and/or cloud... ...level: Mid-Senior LevelIndustry: Management...Work at officeRemote workVisa sponsorshipWork visaFlexible hours- ...Data Engineer Anywhere Type: Contract Category: Data Industry... ...Remote Our client seeks a senior Data Engineer to help launch... ...using PySpark or Apache Spark, Snowflake, and AWS Glue.... ...proficiency with PySpark or Scala/Spark, batch processing, and...Hourly payContract workFor contractorsLocal areaRemote work
Eliassen Group
Richmond, VA4 days ago $128.01k - $152.18k
...businesses to manage risks with confidence.Job DescriptionAI-First Data Strategy for P&C Insurance Carrier | Lead Innovation in Property & Casualty Insurance Data SolutionsWe are seeking a Senior Data Engineer to spearhead our transition to a modern, first, AI-first data...SeniorFull timeWork at officeLocal area$229.9k - $262.4k
...Sr. Lead Data Engineer (Enterprise Platforms Technology) Do you love... ...technologies. As a Capital One Senior Lead Data Engineer, you'll have... ...including Python, SQL, Scala, or Java ~4+ years of experience... ..., Hadoop, Hive, EMR, Kafka, Spark, or MySQL) ~4+ year experience...SeniorFull timePart timeInternshipLocal areaCapital One
Richmond, VA3 days ago- ...Senior Data Engineer The Data Services team is responsible for technical design and end-to-end delivery of complex data-driven solutions and data products for the enterprise. The Senior Data Engineer will report to the Sr Manager, Data Solutions / Manager. This role...SeniorFull timePart timeWork at officeLocal areaWork from homeHome office2 days per week
Blue Shield Of California
Richmond, VA4 days ago - 1. Job Title : Data Engineer 2. Job Summary : Develop unit test case for each module Conduct/ guide conducting of unit and integration testing... ...& Responsibilities : - Should have 5-7 years exp in the Spark Scala Java AWS implementations - Develop and deploy distributed...Second jobShift work
- ...Description Job Description Astyra’s client is seeking a hands-on Senior Data Engineer with strong Generative AI experience to help build a data... ...utilities or automation solutions Familiarity with Spark or other distributed data-processing technologies...Senior
$131.1k - $243.5k
...Job Description Join Lockheed?Martin's Data & AI Enablement organization and... ...company's AI transformation. As a lead data engineer, you will design, build, and productize... ...skills in Python and SQL; proficient with Spark (PySpark/Scala) and deep experience with AI-assisted...Full timeTemporary workWork experience placementFlexible hours$124k - $280k
...Data And Analytics Engineering Senior Manager At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into...SeniorPwC (US)
Richmond, VA2 days ago$110k - $120k
...Job Description Job Description Data Engineer Richmond, VA 23219 (hybrid) Pay: $110,000-120,000 Role Summary The Senior Data Engineer is a hands-on expert and technical... ...technologies such as Kafka, Kinesis, or Spark Streaming Perform on-the-fly data...Senior- ...to meet you! ABOUT THE ROLE We are looking for a Senior Data Engineer to design and maintain data pipelines and automation systems... ...Upper-intermediate English level. NICE TO HAVES - Scala experience. - FinOps Foundation Certification (...SeniorWork at officeWork from homeVisa sponsorshipWork visa
$82.6k - $162.8k
...transform finance. Work You'll Do As a OneStream Senior Developer, you will be responsible for:... ...consolidation, reporting, and financial data management requirements.Build and... ..., Information Systems, Computer Science, Engineering, or Business4+ years of experience implementing...Senior$107.7k - $179.5k
...secure success.Work You’ll DoAs a Security Engineer III on the mission-focused technology and... ..., and maintain secure platform and data engineering solutions within the Jupiter/... ...development From entry-level employees to senior leaders, we believe there’s always room to...SeniorLocal area- ...Virgina Key Responsibilities: •Design, develop, and maintain data pipelines leveraging Python, Spark/PySpark, and cloud-native services. •Build and optimize... ...•Collaborate with data scientists, analysts, platform engineers, and product teams to enable reliable, high-quality...
$90k - $200k
...a list that spans across the country.Job DescriptionCapTech Data Engineering consultants enable clients to build and maintain advanced data... ...sponsorship. #LI-LM1 SummaryType: Full-timeFunction: ConsultingExperience level: Mid-Senior LevelIndustry: Management ConsultingSeniorWork at officeRemote workVisa sponsorshipWork visaFlexible hours$130k - $170k
...States of America)Please review the following job description:Senior Engineer/Platform Leader accountable for designing, building, and operating... ...management, retrieval and knowledge-grounding patterns, data access patterns, CI/CD automation, evaluation/testing, and observability...SeniorFull timePart timeShift workDay shift- ...Job Title: Data Engineer - Spark & Real-Time Data Processing Location: Richmond, VA Duration: 6 Months Role Overview We are seeking... ...Streaming) Proficiency in Python (PySpark) and/or Scala Experience building batch and streaming ETL pipelines...
Artech
Henrico, VA4 days ago - ...Data Engineer (763232) Requirements: Strong Data engineering fundamentals. Utilize Big data frameworks like Spark/Databricks. Training LLMs with structed and unstructured data sets. Understanding of Graph DB. Experience with Azure Blob...
RIT Solutions, Inc.
Richmond, VA4 days ago - ...Looking For Data Engineer Top Skills Python Spark Scala AWS Snowflake Required Skills Basic Qualification Additional Skills Candidate must be your W2 Employee Interview Process Additional Keywords Degree Requirements Certification...
Samprasoft
Henrico, VA3 hours ago - ...Data Engineer Location: McLean, VA, Richmond, VA, Wilmington, DE, and Boston (Need candidate to work onsite from any location, 3 days... ...% Frontend) Primary focused will be on Python. Top Skills Python AWS Build Spark job using EMR Snowflake...
Samprasoft
Richmond, VA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer - Spark / Scala. Be the first to apply!
Related searches
- data engineer contract Richmond, VA
- big data cloud engineer Richmond, VA
- big data developer Richmond, VA
- finance data engineer Richmond, VA
- software data engineer Richmond, VA
- data engineer intern Richmond, VA
- data engineer machine learning Richmond, VA
- senior data quality engineer Richmond, VA
- data visualization developer Richmond, VA
- entry level data engineer Richmond, VA