Lead Data Engineer

Openkyber

Position: Senior Data Engineer

Experience: 10+ Years

Employment Type: Contract W2

Job Summary
We are looking for an experienced Senior Data Engineer with 10+ years of experience in designing, developing, and maintaining scalable data pipelines and enterprise data platforms. The ideal candidate should have strong expertise in Python, SQL, PySpark, ETL/ELT, cloud platforms, data warehousing, and modern data engineering technologies.
Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines for large and complex datasets.

Develop data processing solutions using Python, PySpark, and SQL.

Build and optimize data pipelines using tools such as AWS Glue, Informatica, Azure Data Factory, Databricks, or equivalent technologies.

Design and implement data warehouses, data lakes, and lakehouse solutions.

Work with cloud platforms such as AWS, Azure, or Google Cloud Platform.

Develop data ingestion and transformation processes from structured and unstructured data sources.

Perform data profiling, validation, cleansing, and quality checks.

Optimize SQL queries, Spark jobs, and data pipelines for performance and scalability.

Work with databases including Oracle, PostgreSQL, SQL Server, Teradata, Snowflake, BigQuery, or Redshift.

Implement CI/CD processes and version control using Git, Jenkins, GitHub Actions, or Azure DevOps.

Collaborate with Data Architects, Business Analysts, DevOps Engineers, and application teams.

Monitor production pipelines, troubleshoot failures, and resolve data-related issues.

Ensure data security, governance, lineage, and compliance standards are followed.

Participate in architecture discussions, code reviews, technical documentation, and mentoring of junior engineers.

Required Skills

10+ years of experience in Data Engineering or related fields.

Strong hands-on experience with Python, PySpark, and SQL.

Strong understanding of ETL/ELT concepts and data pipeline development.

Experience with AWS, Azure, or Google Cloud Platform cloud environments.

Strong experience with Databricks / Spark and distributed data processing.

Experience with Snowflake, BigQuery, Redshift, Teradata, Oracle, or PostgreSQL.

Experience with data lakes, data warehouses, and lakehouse architecture.

Knowledge of Kafka or other streaming technologies is preferred.

Experience with Git and CI/CD tools.

Strong troubleshooting, analytical, and problem-solving skills.

Excellent communication and collaboration skills.

Preferred Skills

Experience with Informatica PowerCenter/IICS/IDMC, Talend, AWS Glue, ADF, or similar ETL tools.

Experience with Airflow, Control-M, or other workflow orchestration tools.

Knowledge of Terraform and cloud infrastructure.

Experience with Delta Lake / Delta Live Tables.

Knowledge of data governance, metadata management, and data quality frameworks.

Experience working in Agile/Scrum environments.

For applications and inquiries, contact:View email address on us.fitly.work

Apply

Vacancy posted 2 days ago

Similar jobs that could be interesting for youBased on the Lead Data Engineer in Atlanta, GA vacancy

Lead Data Engineer
Robert Half is working with a client in the Atlanta area seeking a Senior Lead Data Engineer.Lead the design, development, and optimization of enterprise data platforms and data warehouse solutionsDrive the migration and modernization of legacy data environments to SnowflakeArchitect...
Suggested
Robert Half
Atlanta, GA
1 day ago
Lead Data Engineer
Honeywell is accelerating its transformation from industrial automation to full autonomy and the data that powers that future starts here. As a Lead Data Engineer on our Industrial AI & Data Platforms team, you will architect and own the data foundations that enable physical...
Suggested
Permanent employment
Temporary work
Relocation package
Flexible hours
Honeywell
Atlanta, GA
2 days ago
Sr. Lead Data Engineer
$205.32k
...USAJob Family GroupEngineering / Product DevelopmentJob ProfileSr Lead Data EngineerManagement LevelSr Manager - Non People LeaderFlexible... ..., LLCSr. Lead Data EngineerJob Description: Sr. Lead Data Engineer positions offered by Cox Automotive Corporate Services, LLC (Atlanta...
Suggested
Full time
Work at office
Local area
Remote work
Flexible hours
Cox Enterprises
Atlanta, GA
2 days ago
Lead Data Engineer
$134.5k - $265.1k
Position Summary Deloitte is seeking a Lead Data Engineer- Databricks to support the design, build, and delivery of modern data and analytics solutions for clients across industries. In this role, you will help teams translate business needs into scalable data platform...
Suggested
Local area
Visa sponsorship
Deloitte
Atlanta, GA
38 minutes ago
Lead Data Engineer
...comprehensive electrical procurement, Ampacity specializes in expert-engineered, field-ready solutions that accelerate project timelines and... ...America. About this Role Ampacity is looking for a Lead Data Engineer to lead the rebuild and modernization of our data...
Suggested
Full time
Work at office
Local area
Quanta Services
Atlanta, GA
2 days ago
Lead Data Engineer
...uphold organizational standards. Develops business models to represent requirements, such as use cases, business process flows, and data flow diagrams.Negotiates agreements and commitments. Facilitates communication between business units and Information Technology from...
Work at office
Synovus
Atlanta, GA
4 days ago
Lead Data Engineer - United States
...Atlanta, GeorgiaCompany: Georgia-PacificCareer Field: Information Systems & TechnologyJob Number: 191597Apply: JobAs a Lead Data Platform Engineer on the Georgia Pacific Data Platform team, you will help deliver trusted, high-quality data across hundreds of systems and...
Work at office
Worldwide
Flexible hours
Georgia-Pacific
Atlanta, GA
3 days ago
Lead Machine Learning / Data Science Engineer
...the country.Job DescriptionCapTech Machine Learning Engineers are responsible for designing and implementing data-driven solutions for our clients, with a specific... ...and our clients.Specific responsibilities for the Lead Machine Learning Engineer position include:...
Work at office
Remote work
Visa sponsorship
Work visa
Flexible hours
CapTech
Atlanta, GA
3 days ago
Lead Consultant - Data Architecture & Engineering
...impact that is felt by all. See where your commitment could take you.Who are we looking for?GHD is seeking a Lead Consultant - Data Architecture & Engineering to join our Americas Business Advisory practice. This role is a senior seller‑doer position for a technically...
Work at office
Local area
3 days per week
GHD
Atlanta, GA
4 days ago
Lead Data Engineer
...Job Summary Join us as a Lead Data Engineer on the Corporate Data Technology Team. Become an integral part of an agile team dedicated to enhancing, building, and delivering trusted market‑leading technology products in a secure, stable, and scalable way. As a core...
Shift work
Cooper Lighting Solutions
Atlanta, GA
4 days ago
Lead Data Engineer
...Lead Data EngineerChoosing Capgemini means choosing a company where you will be empowered to shape your career in the way you'd like,... ...technology and build a more sustainable, more inclusive world.Data engineers are responsible for building reliable and scalable data...
Full time
Local area
Capgemini
Atlanta, GA
2 days ago
Lead Data Engineer
$200k
...Lead Data Engineer - Remote 100% Remote (U.S. Based) Up to $200,000 Base Salary U.S. Citizens & Green Card Holders Only I'm partnering with a rapidly growing healthcare technology company that is looking to hire a Lead Data Engineer to help drive the evolution...
Remote work
Harnham
Atlanta, GA
14 hours ago
Lead Data Engineer
...Lead Data EngineerStrategically architect, design, develop industry-specific solutions in support of core enterprise functions based... ...s and market best practices. Strong understanding of software engineering principles, databases and data modeling/design techniques; data...
Syntricate Technologies
Atlanta, GA
4 days ago
Senior Lead Data Engineer
...Senior Lead Data EngineerWe are seeking a highly skilled and motivated Senior Lead Data Engineer to join our dynamic team. The ideal candidate will possess extensive knowledge of AWS AI Services and will be responsible for designing, building, and maintaining scalable...
Diverse Lynx
Atlanta, GA
3 days ago
Lead Data Engineer
100% Onsite Role - Onsite interview in MD Bachelor degree in Computer Science, Data Science, Engineering, or a related field 15+ years of experience in software or data engineering, with at least 4 years focused on ML systems or MLOps in a production environment. Demonstrated...
Openkyber, LLC
Atlanta, GA
4 days ago
Lead Data Engineer
...Job Title: Data Engineer ( DATA ENGINEERING | DATA MODELING | PERFORMANCE) Location: Remote & hybrid, alternating weeks at the nearest office Duration: 4 months+ contract Number of Positions: 2 roles Responsibilities: DATA ENGINEERING | DATA MODELING | PERFORMANCE Senior...
Contract work
Work at office
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Job Title: Sr. Data Engineer Location: Portland, OR Onsite Job Type: Contract Experience: 14+ Years Mandatory Interested candidates are required to include their LinkedIn URL, ****@*****.***, and cell phone number along with their...
Contract work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Senior Data Engineer Remote prefer candidates form Charleston, SC Long Term W2 The Senior Data Engineer, leads the design and implementation of robust data solutions across multiple domains, driving technical excellence and scalability. This role mentor others, shape best...
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Role: Senior/Lead Data Engineer Location: Remote The engineer will be responsible for platform reliability, monitoring, incident management, automation, performance optimization, security, and operational support for data and Generative AI workloads....
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...you to discuss about this opportunity in details. Role 1: Sr. Data Engineer with Databricks, Python, Salesforce Automation Role 2: Sr. Data... ...source system analysis. Thanks & Regards, OpenKyber LEAD TALENT ACQUISITION SPECIALIST Direct Number:- Fax : Email: Website...
Long term contract
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...adventure where you can push the limits of what's possible. As a Lead Software Engineer at OpenKyber within the Corporate Technology - Consumer and... ...the value realized by automation Implements and manages data solutions using Snowflake, including data modeling, performance...
Work at office
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Role Name: Lead Data Engineer Location: Pittsburgh, PA OR Lake Mary, FL Duration : 12 Months Must-have skills: SQL, Oracle, Snowflake, Collibra, Financial Domain Expertise. Good-to-have skills: Data Modeling, Banking Structures, AI, Data Analytics...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Job Title: Senior Data Engineer Location: Covington, KY Long term contract with Extension At our OpenKyber's place, we love to use data to create incredible personalized customer experiences and develop solutions that help our customers live the lives they want. As part...
Long term contract
Worldwide
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...has a client in Minneapolis, MN that is seeking a hybrid Senior Data Engineer to join their team. Summary: The client is looking for a... ...and collaboration skills and ability to act as a lead and work independently The pay range is the lowest to highest...
Hourly pay
Contract work
Flexible hours
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Job Title: Senior Data Engineer (Snowflake,DBT & DataStage) Remote W2 only We are seeking a Senior Data Engineer with strong experience in Snowflake, AWS, DataStage, and dbt/ELT tools to build and maintain scalable data warehouse solutions and data pipelines. Pointers...
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...details about the role. Position: Senior Technology Consultant - Data / AI Native Engineer Location: Arlington, VA || New York, NY || St. Louis, MO (... ...strong Software Engineering practices and has experience leading complex enterprise data initiatives. The role requires...
Long term contract
Local area
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Required Qualifications: Bachelor's degree in Information Systems, Information Technology, Computer Science, Engineering or equivalent experience Strong experience with SQL and PL/SQL development Experience working with relational databases such as SQL Server...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Data Engineer Job Description We are looking for a Senior Data Engineer with 8+ years of experience building and supporting enterprise data platforms. The ideal candidate will have strong hands-on expertise with Microsoft Fabric, Azure, ETL/ELT, SQL, Python, and modern...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Role: Senior Data Engineer Location: Dallas, TX (Onsite) 6-12 Months Responsibilities Design and Implement scalable solutions for ever-increasing data volumes, using big data/cloud technologies like Pyspark, Kafka, etc. Collaborate with cross-functional...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Job Title: AWS Glue Data Engineer (14+ Years Only) Location: Fort Mill SC / New York, NY / Florham Park, NJ (Hybrid) We are seeking a highly skilled AWS Data Engineer to design, develop, and optimize large-scale data pipelines and ETL workflows on AWS. The ideal...
Openkyber
Atlanta, GA
2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!