Lead Data Engineer

Experience: 10+ Years

Job Type: W2 Contract Only

Job Summary We are seeking a Senior Data Engineer with 10+ years of experience in designing, developing, and supporting scalable data solutions. The ideal candidate will have strong experience with Python, SQL, ETL/ELT, cloud platforms, data pipelines, data warehousing, and distributed data processing .

• 10+ years of experience in Data Engineering, ETL, data integration, or related technologies .
• Strong hands-on experience with Python and advanced SQL .
• Experience designing and developing scalable ETL/ELT data pipelines .
• Strong understanding of data warehousing, data lakes, data modeling, and data integration concepts.
• Experience with Apache Spark / PySpark for large-scale data processing.
• Hands-on experience with Databricks or similar modern data platforms.
• Experience working with cloud platforms such as AWS, Azure, or Google Cloud Platform .
• Experience with relational databases such as SQL Server, Oracle, PostgreSQL, MySQL, Snowflake, or similar .
• Experience developing data ingestion and transformation processes from databases, APIs, files, and other sources.
• Strong knowledge of data quality, validation, reconciliation, and error handling .
• Experience with workflow orchestration tools such as Airflow, Azure Data Factory, AWS Glue, or similar .
• Experience with Git, CI/CD, and automated deployment processes .
• Strong troubleshooting and performance optimization skills.
• Experience working in Agile/Scrum environments .

• Design, develop, and maintain enterprise-scale data pipelines and integration solutions .
• Develop complex data transformations using Python, SQL, Spark, and PySpark .
• Build data ingestion processes from structured and unstructured data sources.
• Design and maintain data warehouse and data lake solutions.
• Optimize SQL queries, Spark jobs, and data pipelines for performance and scalability.
• Implement data validation, quality checks, monitoring, and reconciliation processes.
• Develop and manage data workflows using orchestration tools.
• Work with cloud storage, databases, APIs, and enterprise data platforms.
• Support production data pipelines and troubleshoot data-processing issues.
• Implement CI/CD practices for data engineering code and deployment.
• Collaborate with data architects, analysts, developers, QA teams, and business stakeholders.
• Participate in technical design, code reviews, sprint planning, and other Agile activities.
• Document data flows, technical solutions, data mappings, and operational processes.
• Mentor junior and mid-level data engineers and contribute to data engineering best practices.

• Experience with Delta Lake / Lakehouse architecture .
• Knowledge of AWS S3, Azure Data Lake, ADLS, or Google Cloud Storage .
• Experience with Snowflake or other cloud data warehouses.
• Knowledge of Kafka, Event Hubs, or other streaming technologies .
• Experience with Terraform or Infrastructure as Code .
• Knowledge of Docker and Kubernetes.
• Experience with data governance, metadata, lineage, and security.
• Exposure to real-time and near-real-time data processing.

• Strong understanding of enterprise data integration, reporting, analytics, and data warehouse environments .
• Experience working with large-volume transactional and analytical data.
• Understanding of business requirements, data mapping, data transformation, and downstream reporting needs.
• Experience working with cross-functional technical and business teams.

For applications and inquiries, contact:View email address on us.fitly.work