Data Engineer

Job Description

Job Description

We are seeking a Data Engineer to support the design, development, optimization, and maintenance of cloud-based data pipelines, platforms, and data stores used for large-scale analytics.

The ideal candidate will have hands-on experience building scalable ETL/ELT workflows, working with cloud-based data technologies, and transforming structured and unstructured data into reliable, analysis-ready formats. This role will collaborate closely with analysts, engineers, developers, and other stakeholders to develop data solutions that support reporting, analytics, modeling, and visualization.

Key Responsibilities

• Assist in designing, developing, and maintaining scalable ETL/ELT pipelines to ingest, transform, and deliver structured and unstructured data from multiple cloud-based sources.
• Build and optimize data storage solutions , including data warehouses, data lakes, and lakehouses, to support large-scale analytics, reporting, and business intelligence.
• Perform backend SQL and NoSQL database testing , including data validation, post-deployment testing, troubleshooting, and identification of ingestion or schema issues.
• Collaborate with analysts, engineers, developers, and stakeholders to translate business and technical requirements into data models, workflows, and pipeline specifications.
• Develop scripts and automation to transform diverse data types into usable, analysis-ready formats.
• Support the scaling, monitoring, maintenance, and operation of cloud-based data platforms.
• Validate data quality, integrity, and reliability throughout the data pipeline lifecycle.
• Troubleshoot data processing, pipeline, and storage issues as needed.

Required Qualifications

• 2+ years of experience in Data Warehouse development.
• Bachelor’s degree.
• Experience analyzing and processing aggregated data from multiple sources in cloud-based environments .
• Experience developing and maintaining scalable data stores that provide large datasets in formats suitable for business analysis.
• Hands-on experience with SQL and Python for retrieving, parsing, transforming, and processing structured and unstructured data.
• Experience developing scalable ETL or ELT workflows using cloud-based technologies for reporting and analytics.
• Experience with Cloudera and/or Hadoop technologies for data ingestion, transformation, and processing.
• Experience using generative AI coding tools to support software or data engineering development.
• Ability to perform backend SQL and NoSQL database testing , including post-deployment and data validation testing.
• Ability to obtain and maintain a Public Trust or Suitability/Fitness determination , based on project requirements.
• Must be a U.S. Citizen or Lawful Permanent Resident (Green Card Holder).

Preferred Qualifications

• Experience working in collaborative, cross-functional engineering environments.
• Experience with AWS Glue, AWS Athena, and PostgreSQL .
• Experience with distributed data and computing technologies such as Spark, Databricks, Hive, AWS EMR, or Kafka .
• Experience developing real-time data processing and streaming applications .
• Experience implementing NoSQL solutions using MongoDB or Cassandra .
• Experience with data warehousing technologies such as AWS Redshift, MySQL, or Snowflake .
• Experience with UNIX/Linux , including basic commands and shell scripting.
• Experience working within Agile engineering environments .