Data Engineer II

Is this request tied to a specific business initiative/project?
Yes. This request is tied to modernization and optimization of analytics data assets that are related to the customer digital journeys and platform reliability. The resource will help refactor and optimize existing Redshift and sandbox data work into scalable, production-grade analytics data products using Databricks and PySpark
Top Must-Have Skillsets
1. Expert-level PySpark and Databricks experience, including large-scale data processing workloads.
2. Advanced SQL expertise, especially with high-volume queries in AWS Redshift environments.
3. Experience refactoring legacy SQL/PySpark logic into optimized, modular pipelines.
4. Ability to architect reliable, scalable, production-grade analytics data products with strong governance standards.
Nice-to-Have Skillsets
1. Experience in travel, e-commerce, logistics, or other fast-paced enterprise-scale environments.
2. Familiarity with behavioral analytics, conversion funnels, attribution logic, and performance monitoring metrics.
3. Experience standardizing data product outputs, including schema enforcement, metadata documentation, and lowercase/underscore naming conventions.
4. Ability to transition experimental sandbox work into production-ready assets for long-term support.
5. Experience working with transactional log sources and complex analytics data structures.
Are there any other key project details that suppliers should know?
Communication and partnership skills are important, as the person will work with the core analytics team and help create assets that can be handed off to broader engineering and offshore teams for long-term maintenance.

Position Overview
We are seeking an expert Senior Analytics Engineer for a short-term contract to accelerate the modernization and optimization of our data platform footprint. Our team sits at the critical intersection of customer digital journeys, platform reliability, and operational performance telemetry.
Your primary mission will be to dive into our existing Redshift and sandbox data environments, untangle complex data structures, and use Databricks and PySpark to refactor, optimize, and formally architect these datasets into highly efficient, scalable, and production-grade Analytics Data Products. The ideal candidate requires minimal onboarding, possesses deep technical mastery of log-parsing and clickstream data, and can implement strict data governance standards from day one.
Core Responsibilities
Data Product Architecture: Transform ad-hoc sandbox queries, clickstream event data, and massive transactional log sources into structured, reliable, downstream analytics data assets.
Pipeline Refactoring & Optimization: Convert legacy SQL/Redshift logic into highly optimized, modular PySpark pipelines within Databricks, drastically reducing compute times and improving query efficiency.
Log Parsing & Behavioral Analytics: Standardize the ingestion and structural mapping of diverse semi-structured log data (e.g., shopping cart logs, API error events, system performance logs) to build precise conversion funnel and system reliability metrics.
Implement Data Governance: Enforce rigorous technical standards across all data products, ensuring strict schema enforcement, detailed metadata documentation, and standardized lowercase/underscore naming conventions across all outputs.
Production Blueprinting: Partner with the core analytics team to transition experimental sandbox work into robust, production-ready data assets that can easily be handed off to broader engineering and offshore teams for long-term maintenance.
Technical Requirements & Qualifications
Expert-Level PySpark & Databricks: 5+ years of hands-on experience designing, optimization, and maintaining large-scale data processing workloads in Databricks using PySpark (DataFrames, Structured Streaming, Delta Lake).
Advanced SQL & Cloud Data Warehousing: Deep expertise in writing and optimizing complex, high-volume queries in AWS Redshift or Postgres environments.
Analytics Domain Context: Strong understanding of how data structures must be optimized specifically for analytics use cases, including building clean attribution logic, conversion funnels, and performance monitoring metrics.
Immediate Autonomy: A self-starter mentality capable of auditing unfamiliar database schemas, reverse-engineering existing logic, and delivering clean, production-ready code with minimal business context.
Preferred Skills
Prior experience working in complex, fast-paced, enterprise-scale environments (travel, e-commerce, or logistics industries is a major plus).

EEO:

"Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of - Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans."