Data Engineer

Description:

Seattle, WA (Onsite) | Local candidates only as client interview will be in person.


Day to Day Job Duties:
Perform end-to-end datastore migrations from on-premises DataLake environments to AWS-hosted Lakehouse platforms as part of the migration factory team.
Refactor and migrate existing data pipelines, including data extraction logic, transformation processes, and job scheduling.
Execute large-scale data transfers while ensuring data integrity, completeness, reliability, and consistency between source and target platforms.
Translate and modernize legacy SQL and Apache Spark-based processing logic for Snowflake and Apache Iceberg environments.
Analyze existing data usage patterns, business requirements, and downstream consumption to support the development and delivery of reusable data products.
Design and build data reconciliation and validation frameworks to verify data accuracy during and after migration.
Collaborate with business stakeholders, application teams, data owners, and technical teams to perform validation and obtain migration sign-off.
Act as a technical liaison between migration, data engineering, application, infrastructure, and business teams throughout the migration lifecycle.
Troubleshoot data pipeline, data transformation, performance, and migration-related issues and implement appropriate technical solutions.
Optimize data pipelines and processing workloads to improve performance, scalability, and operational efficiency.
Apply core data engineering concepts including Slowly Changing Dimensions (SCD Type 2), schema evolution, partitioning, clustering, normalization and denormalization, natural and surrogate keys, and data quality controls.
Work with structured and semi-structured data formats including JSON, Avro, and Parquet.
Follow established Software Development Life Cycle (SDLC), source-control, testing, deployment, and Continuous Integration/Continuous Deployment (CI/CD) practices.
Adapt to new data technologies, migration tools, engineering standards, and workflows as required by the program.
Collaborate effectively with geographically distributed and global delivery teams.
Basic Qualifications:
Minimum 3+ years of hands-on software development and/or data engineering experience, including coding, development, troubleshooting, and implementation of data-related solutions.
Minimum 3+ years of experience working with SQL, including development, analysis, query troubleshooting, and performance optimization.
Minimum 3+ years of hands-on programming experience using Python and/or Java for data processing, application development, automation, or integration.
Minimum 3+ years of experience developing or supporting ETL/ELT data pipelines, including data extraction, transformation, loading, and pipeline troubleshooting.
Experience developing or supporting distributed data-processing solutions using Apache Spark.
Experience working with data engineering concepts including SCD Type 2, schema evolution, partitioning, clustering, normalization versus denormalization, natural versus surrogate keys, and data quality frameworks.
Experience working with one or more data and integration technologies including Kafka, ANSI SQL, FTP, Apache Spark, Hadoop, Snowflake, Apache Iceberg, and Sybase IQ.
Experience working with structured and semi-structured data formats including JSON, Avro, and Parquet.
Experience working within established SDLC and CI/CD processes, including source control, testing, deployment, and release practices.
Familiarity with containerized application environments and Kubernetes.
Demonstrated ability to troubleshoot technical issues, communicate effectively with stakeholders, collaborate across global teams, and take ownership of assigned deliverables.


Travel:
No specific travel requirement has been identified for this position.


Degree:
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field, or equivalent work experience.


Nice to Have:
Experience performing large-scale data platform or datastore migration initiatives.
Experience migrating data workloads from on-premises environments to cloud-based platforms, particularly AWS.
Experience working with Lakehouse architectures.
Experience with Snowflake and Apache Iceberg-based data platforms.
Experience working within the financial services industry.
Experience supporting complex migration programs involving multiple business, technology, and global delivery teams.


Enable Skills-Based Hiring No ICIMS RR ID #
Additional Details

• Skill Category : Regular
• ICIMS RR ID # : NA
• Client Name : Goldman Sachs
• Engagement Type : T&M Competitive 1-3
• Vertical : BFS
• Must-Have Primary Skill : Application Management-Development-Data Bricks, ADLS (Azure Data Lake Store), PySpark, Azure SQL, Python, SSIS
• Primary Skill: Yrs Experience : Expert (5+ Years Experience)
• RC- Domain : Data & Analytics
• RC- Subdomain : 25 - Big Data, Data Science & Analytics (Apache Hadoop HDFS, Base/Hive/Pig/Mahout/Flume/Scoop/MapReduce/Yarn, Cloudera HD, NoSQL, R, Spark/Shark/Milb, MATLAB)
• RC- Role : Data Specialist/Analyst
• RC- Experience Level : III
• RC- Geo Tier : US-4
• RC- Exception Required : TBD
• COVID-19 Vaccine Required? : (No Value)
• Client-Accepted Visa Types : (No Value)