IT Data Engineer - BB4352

TechData Service Company, LLC is looking for an IT Data Engineer for an onsite contract role in Cambridge, MA . This position supports Discovery R&D and data science initiatives by building Databricks-based, scalable data platforms and automated scientific pipelines. You will work close to research and data teams to help ensure data workflows are reproducible, governed, and ready for downstream analytics. How you’ll make an impact Design, develop, and maintain scalable data pipelines using Databricks , Apache Spark , and cloud-native technologies. Build and optimize ETL/ELT processes across structured, semi-structured, and unstructured datasets. Create data ingestion frameworks for research , laboratory , clinical , and external scientific datasets . Implement data quality , validation , monitoring , and governance practices. Support data lakehouse architecture and participate in data platform modernization efforts. Develop and maintain Databricks notebooks , workflows , Delta Live Tables , and Databricks Jobs . Apply Medallion Architecture (Bronze, Silver, Gold) to manage the data lifecycle. Manage Delta Lake environments, optimizing performance , scalability , and cost . Integrate Databricks with cloud-native services and enterprise applications. Deploy, configure, and support the Seqera Platform (formerly Nextflow Tower ). Develop and maintain Nextflow pipelines for bioinformatics, genomics, imaging, AI/ML, and scientific computing workloads. Integrate Seqera workflows with AWS infrastructure and compute environments. Support containerized workflows using Docker and Kubernetes . Design and implement cloud-based data solutions on AWS , including S3 and data lifecycle policies. Build Infrastructure-as-Code solutions using Terraform or CloudFormation . Implement security controls and access management aligned with enterprise IT standards. Partner with data scientists, researchers bioinformaticians, and stakeholders to clarify data requirements and support workflow automation. Troubleshoot pipeline failures, performance issues, and workflow bottlenecks; contribute to platform roadmaps and continuous improvement. SOPs, and knowledge articles. What you bring Bachelor’s degree in Computer Science, Information Technology, Data Engineering, Bioinformatics, or a related technical field. 5+ years of experience in data engineering, cloud engineering, or analytics platform development. 3+ years hands‑on experience with Databricks and Apache Spark . 2+ years experience with Seqera Platform (Nextflow Tower) and Nextflow workflows. Experience supporting scientific research , life sciences , pharmaceutical , biotech , or healthcare environments preferred. Strong familiarity with: Databricks Lakehouse Platform , PySpark and Spark SQL , Delta Lake and Delta Live Tables , Databricks Workflows , and Unity Catalog . Experience with SQL and Python , plus bioinformatics workflow automation . Hands‑on work with Docker and container orchestration via Kubernetes , including high‑performance computing environments. AWS required : S3 , IAM , EC2 , VPC , Lambda . Infrastructure-as-Code with Terraform or CloudFormation , plus cloud monitoring and logging tools. Practical experience across data lake and lakehouse architectures, ETL/ELT frameworks, data modeling, data cataloging, and governance. Experience with API integrations , data quality frameworks , and development practices using GitHub or GitLab . Knowledge of CI/CD pipelines and tools such as Jenkins or GitHub Actions , along with Agile and DevOps methodologies. Compensation and role details Location: Cambridge, MA (onsite) Job type: contract Pay: USD 70 - 95 per hour Experience: minimum 2 years Benefits Dental insurance Health insurance Application questions Where are you currently located? What hourly rate are you seeking on a W2? #J-18808-Ljbffr TechData Service Company, LLC