Data Engineer

We’re looking for an experienced Data Engineer to design, build, and maintain scalable data platforms and pipelines that power analytics, reporting, machine learning, and AI applications. This is a hands‑on engineering role for someone who enjoys solving complex data problems, working with large datasets, and building reliable systems that make data accessible and useful across the organization. What You’ll Do Design, build, and maintain scalable ETL/ELT data pipelines Develop data ingestion and transformation processes across structured, semi-structured, and unstructured data sources Build and optimize data warehouses, data lakes, and lakehouse architectures Develop reliable batch and real‑time/streaming data pipelines Design data models that support analytics, reporting, operational applications, and AI/ML use cases Integrate data from APIs, databases, SaaS platforms, files, and third‑party systems Improve data quality, reliability, observability, and performance Build automated processes for data validation, testing, monitoring, and error handling Optimize queries, storage, compute, and pipeline performance Implement appropriate data security, governance, access controls, and privacy standards Partner with data scientists, AI/ML engineers, analysts, software engineers, and business stakeholders Support data infrastructure through CI/CD, infrastructure automation, and modern DataOps practices Troubleshoot complex production data issues and identify opportunities to improve platform scalability and reliability What We’re Looking For 4+ years of professional Data Engineering or related experience Strong proficiency with SQL Strong programming skills in Python Experience designing and building production‑grade ETL/ELT pipelines Experience with modern cloud data platforms such as Snowflake, Databricks, BigQuery, Redshift, or Microsoft Fabric Hands‑on experience with AWS, Azure, or GCP Experience with data transformation and orchestration technologies such as dbt, Airflow, Dagster, Prefect, or similar Experience working with relational and NoSQL databases Understanding of dimensional modeling, data warehousing, lakehouse architectures, and distributed data processing Experience with technologies such as Spark, Kafka, or similar distributed/streaming platforms Familiarity with Git, CI/CD, Docker, and modern software engineering practices Strong understanding of data quality, lineage, governance, security, and observability Ability to translate business and technical requirements into scalable data solutions Nice to Have Experience building data infrastructure for AI/ML and Generative AI applications Experience with vector databases, embeddings, or unstructured data pipelines Experience with real‑time event‑driven architectures Experience with Terraform or other Infrastructure‑as‑Code tools Experience with Kubernetes Experience implementing data catalogs, lineage, and governance platforms Experience supporting high‑volume or highly distributed data environments What Success Looks Like You build data systems people can trust. Your pipelines are reliable, scalable, observable, and designed with downstream users in mind. You understand that strong data engineering isn't just about moving data from one place to another—it's about creating a foundation that allows analytics, applications, and AI systems to operate effectively at scale. #J-18808-Ljbffr Prodigy Resources