Data Engineer

Data Engineer

Location: Minnetonka Mills, MN

Duration: Contract/ w2

Rate: $45.00 - $50.00

Job Description

Data Engineer with strong hands-on experience in Azure Cloud, Azure Databricks, PySpark, Azure Data Factory (ADF), Delta Lake, and Unity Catalog. The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and cloud-based data solutions, with expertise in ETL/ELT processing, data modeling, performance tuning, and data governance. The role requires proficiency in Python, SQL, GitHub, and CI/CD practices, along with experience working with large-scale datasets in enterprise environments. This is an individual contributor role that demands strong analytical and problem-solving skills, the ability to work independently, and ownership of end-to-end solution delivery while collaborating effectively with cross-functional teams and business stakeholders.

Expertise in: Azure Databricks, PySpark, Delta Lake Azure Data Factory (ADF) and ADLS Gen2 Experience working on large-scale, cloud-native data platforms (Azure preferred) Exposure to AI/LLM or agentic frameworks (Databricks AI/ML, Codex, Claude, etc.) Strong experience building REST or Graph QL APIs Experience with Kubernetes (AKS preferred) for API deployment Understanding of API security, contracts, error handling, and performance Strong understanding of data governance, security, and CI/CD practices

Responsibilities & Technical Skills

• Design and build ETL/ELT data pipelines for ingestion, transformation, and loading
• Ensure seamless data integration across multiple systems and applications
• Build and optimize batch + streaming pipelines
• Implement scalable cloud data architecture (Azure; AWS/GCP exposure a plus)
• Develop and orchestrate ADF pipelines, triggers, and scheduling
• Performance tuning, cost optimization, and transformation improvements
• Monitoring & observability through Splunk / Azure Monitor
• Collaborate with stakeholders, analysts, and data science teams
• Provide documentation, reports, and translate data into actionable insights
• Ability to deliver efficiently in a fast-changing environment

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination. All applicants will be evaluated solely on the basis of their ability, competence, and their proven capability to perform the functions outlined in the corresponding role. We promote and support a diverse workforce across all levels in the company.