Senior Data Engineer

Required Skills & Experience

Bachelor's degree in Computer Science, Data Engineering, Engineering, or a related field. \n5+ years of data engineering experience. \nStrong Python and SQL skills. \nHands-on experience building production data pipelines in AWS. \nExperience with AWS data technologies such as S3, Glue, Lambda, RDS/Aurora, Redshift, Athena, Kinesis or equivalent technologies. \nExperience with ETL/ELT architecture and orchestration frameworks. \nStrong knowledge of relational and non-relational database design. \nExperience implementing automated data-quality processes. \nExperience building APIs or data services. \nUnderstanding of data governance, lineage, security and access-control principles.

Job Description

Insight Global is seeking a Senior Data Engineer to design and build the cloud data foundation supporting Off-Prem. The platform depends on large volumes of operational, engineering, analytical and contextual information. This role will develop the pipelines, storage architecture, data models, APIs and quality controls required to make that information securely and reliably available applications and AI capabilities. The ideal candidate combines modern AWS data engineering expertise with a strong understanding of data quality, time-series information and scalable data architectures.\nDesign and implement the a cloud data architecture. \nBuild scalable ingestion, transformation, storage and serving pipelines on AWS. \nDevelop pipelines for operational, historical, engineering, application and contextual data. \nDesign data models optimized for analytical applications and AI consumption. \nEstablish appropriate storage patterns across relational, object, time-series and analytical data stores. \nDevelop APIs and services enabling tools and agents to retrieve data consistently. \nEstablish automated data-quality validation, reconciliation and monitoring. \nImplement metadata management, lineage, cataloging and data-governance practices. \nDesign appropriate mechanisms for data segregation and access control. \nOptimize data pipelines for scalability, performance, reliability and cost. \nPartner with operational subject-matter experts to validate the meaning and quality of source data. \nSupport migration of relevant information from document-based repositories into structured cloud-hosted data services. \nCollaborate with AI/ML engineers to create reliable datasets, feature pipelines, retrieval mechanisms and knowledge sources. \nEstablish reusable data integration patterns that accelerate onboarding of additional platform use cases and sites.\n\n$65-$75