Senior Data Engineer

Senior Data EngineerSenior Data Engineer (8+ years) will be responsible for designing, building, and maintaining data pipelines that extract data from source systems, transform it through the medallion architecture layers, and prepare it for consumption by the analytics layer. This role will work closely with the Power BI Developer to ensure data is properly structured, documented, and accessible for reporting and analytics.Technical Skills - Required Skill Area:PythonSQLAWS S3 / Data Lake ArchitecturePySpark / AWS GlueDelta Lake / ParquetAWS LambdaAWS DMS / CDC PatternsREST API IntegrationGit / Version ControlDatabricksMicrosoft Fabric Data ProductsGMP experience is a preferred.Location: Resource will need to work from Indianapolis office at least 50% of the capacity. Expenses will be covered.Key Responsibilities:Phase 1 Deliverables:Following EDB standard, design and establish AWS S3 bucket structure for data lake (Bronze/Silver/Gold zones) with Red CCI security controlsBuild new data pipelines to extract data from the selected SaaS-based HSE systemsEnhance/extend data pipelines using the dataset from AWS EDB Asset related data products and digital solutionsBuild data ingestion pipeline for Manufacturing Execution System via REST API (Lambda-based extraction)Data pipeline to SaaS based tools to integrate data into the data warehouseImplement CDC pipeline for LabVantage LIMS using AWS DMS from Oracle database (or Azure Data Factory)Develop Bronze-to-Silver transformations using AWS Glue or Azure Data Factory depending on data domainsConfigure AWS Glue Data Catalog with appropriate metadata and Red CCI classification tagsPhase 2 Deliverables:Connect to EDB marketplace for enterprise reference dataBuild Silver-to-Gold transformations creating batch-centric data productsImplement data quality checks and monitoring dashboardsDevelop orchestration workflows using AWS Step Functions