Lead Data Engineer
Openkyber
Position: Senior Data Engineer
Experience: 10+ Years
Employment Type: Contract W2
We are looking for an experienced Senior Data Engineer with 10+ years of experience in designing, developing, and maintaining scalable data pipelines and enterprise data platforms. The ideal candidate should have strong expertise in Python, SQL, PySpark, ETL/ELT, cloud platforms, data warehousing, and modern data engineering technologies.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines for large and complex datasets.
- Develop data processing solutions using Python, PySpark, and SQL.
- Build and optimize data pipelines using tools such as AWS Glue, Informatica, Azure Data Factory, Databricks, or equivalent technologies.
- Design and implement data warehouses, data lakes, and lakehouse solutions.
- Work with cloud platforms such as AWS, Azure, or Google Cloud Platform.
- Develop data ingestion and transformation processes from structured and unstructured data sources.
- Perform data profiling, validation, cleansing, and quality checks.
- Optimize SQL queries, Spark jobs, and data pipelines for performance and scalability.
- Work with databases including Oracle, PostgreSQL, SQL Server, Teradata, Snowflake, BigQuery, or Redshift.
- Implement CI/CD processes and version control using Git, Jenkins, GitHub Actions, or Azure DevOps.
- Collaborate with Data Architects, Business Analysts, DevOps Engineers, and application teams.
- Monitor production pipelines, troubleshoot failures, and resolve data-related issues.
- Ensure data security, governance, lineage, and compliance standards are followed.
- Participate in architecture discussions, code reviews, technical documentation, and mentoring of junior engineers.
Required Skills
- 10+ years of experience in Data Engineering or related fields.
- Strong hands-on experience with Python, PySpark, and SQL.
- Strong understanding of ETL/ELT concepts and data pipeline development.
- Experience with AWS, Azure, or Google Cloud Platform cloud environments.
- Strong experience with Databricks / Spark and distributed data processing.
- Experience with Snowflake, BigQuery, Redshift, Teradata, Oracle, or PostgreSQL.
- Experience with data lakes, data warehouses, and lakehouse architecture.
- Knowledge of Kafka or other streaming technologies is preferred.
- Experience with Git and CI/CD tools.
- Strong troubleshooting, analytical, and problem-solving skills.
- Excellent communication and collaboration skills.
Preferred Skills
- Experience with Informatica PowerCenter/IICS/IDMC, Talend, AWS Glue, ADF, or similar ETL tools.
- Experience with Airflow, Control-M, or other workflow orchestration tools.
- Knowledge of Terraform and cloud infrastructure.
- Experience with Delta Lake / Delta Live Tables.
- Knowledge of data governance, metadata management, and data quality frameworks.
- Experience working in Agile/Scrum environments.
For applications and inquiries, contact:View email address on us.fitly.work
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer in Atlanta, GA vacancy
$205.32k
...USAJob Family GroupEngineering / Product DevelopmentJob ProfileSr Lead Data EngineerManagement LevelSr Manager - Non People LeaderFlexible... ..., LLCSr. Lead Data EngineerJob Description: Sr. Lead Data Engineer positions offered by Cox Automotive Corporate Services, LLC (Atlanta...SuggestedFull timeWork at officeLocal areaRemote workFlexible hoursCox Enterprises
Atlanta, GA4 days ago- Inspire Brands is hiring a Lead Data and AI Engineer for the Enterprise Data Organization to design, build and manage data pipelines (Data ingestion, data transformation, data distribution, quality rules, data storage etc.) using cutting edge AI solutions (Generative AI...SuggestedWorldwide
Inspire Brands
Atlanta, GA4 days ago - ...An enterprise technology company is seeking a Technology and Data Engineer 5 for a hybrid opportunity in Raleigh, North Carolina . This role is centered on strengthening Infrastructure as Code security across cloud environments and supporting secure software delivery practices...SuggestedFull timeContract workWork at officeLocal area
Openkyber
Atlanta, GA2 days ago - ...Description: Senior Data Engineer - Data Platforms Role Overview We are seeking a Senior Data Engineer to design, build, and maintain scalable data engineering solutions that power analytics, reporting, and data products across the organization. This role works closely...Suggested
Openkyber
Atlanta, GA2 days ago - ...We are seeking a Data Engineer with experience in projects emphasizing complex data solutions. This person will be responsible for supporting data transformation and distribution to end consumers. Candidate will assist with requirements analysis, design/develop process...SuggestedFor contractors
Openkyber
Atlanta, GA2 days ago - ...Data Engineer and Lead with Denodo Exp. At Newark NJ Denodo Development (3+ years) Most critical skill Virtual Data Models View development and data virtualization Joins, relationships, transformations, aggregations, unions Query optimization and...
Openkyber
Atlanta, GA2 days ago - ...Senior AWS Tech Lead Data Engineer Start ASAP (2-4 WEEKS) Duration 12 months Remote role prefer SO CA, or CA or west coast. But location does not matter. Can live anywhere in USA. Don't need architects or Big Data engineers. We Need AWS Lead Data engineer who can do hands...Immediate startRemote work
Openkyber
Atlanta, GA2 days ago - ...NAVA Software is looking for a Lead Data Engineer Details: Lead Data Engineer Location: Houston TX Monday to Thursday onsite Duration: 6-12 months Position Overview The Lead Data Engineer is responsible for defining, implementing, and overseeing enterprise-wide data architecture...
Openkyber
Atlanta, GA2 days ago - ...Job Title: Data Engineer ( DATA ENGINEERING | DATA MODELING | PERFORMANCE) Location: Remote & hybrid, alternating weeks at the nearest office Duration: 4 months+ contract Number of Positions: 2 roles Responsibilities: DATA ENGINEERING | DATA MODELING | PERFORMANCE Senior...Contract workWork at officeRemote work
Openkyber
Atlanta, GA2 days ago - ...Position- Lead Data Engineer Duration-Contract Location- tlanta, GA 10 + Years JD Must Have - Python, SQL, ETL/ELT, Airflow, Github, CI/CD, GCP/AWS/Azure 1) Strategically architect, Design, develop industry-specific solutions in support...Contract workImmediate start
Syntricate Technologies
Atlanta, GA1 day ago - ...comprehensive electrical procurement, Ampacity specializes in expert-engineered, field-ready solutions that accelerate project timelines and... ...America. About this Role Ampacity is looking for a Lead Data Engineer to lead the rebuild and modernization of our data platform...Work at officeLocal areaWork from home
Quanta Services
Atlanta, GA3 days ago - As a Lead Data Platform Engineer on the Georgia Pacific Data Platform team, you will help deliver trusted, high-quality data across hundreds of systems and thousands of data assets. You will own, deliver, operate, and help architect the Enterprise Data Lake, Ingestion Framework...Work at officeWorldwideFlexible hours
- Job Summary Join us as a Lead Data Engineer on the Corporate Data Technology Team. Become an integral part of an agile team dedicated to enhancing, building, and delivering trusted market‑leading technology products in a secure, stable, and scalable way. As a core technical...Shift work
- Data Engineering Lead (Marketing) Position Description (General role information, job purpose, main objectives of the role) Location: Atlanta, GA Duration: FULL TIME / C2H Mode: Hybrid ( 2 days a Week) This Data Engineering Lead (Marketing) position is part of Client...Full timeFor contractorsWork experience placement2 days per week
- Lead enterprise data engineering for a Databricks-based lakehouse and streaming analytics platform at Norfolk Southern Corp . In this hybrid role in Atlanta, GA , you will set technical direction, modernize ingestion and transformation patterns, and help guide steady-state...Work at officeRemote workShift workWeekend work
- ...Senior Data Engineer Remote prefer candidates form Charleston, SC Long Term W2 The Senior Data Engineer, leads the design and implementation of robust data solutions across multiple domains, driving technical excellence and scalability. This role mentor others, shape best...Remote work
Openkyber
Atlanta, GA2 days ago - ...Role : AWS Lead Data Engineer (3 Open Positions) Location : Newark City, NJ (Hybrid) Contact In person interview is required at Newark City, NJ (Hybrid) Required Qualifications: Bachelor's or Master's degree in Computer Science, Engineering, or a related...
Openkyber
Atlanta, GA2 days ago - ...Seeking a data engineer for hybrid contract role in Westmont IL. If interested , share your word document resume, work authorization, expected... ...enterprise data for AI/agentic consumption. Track record of leading workstreams and driving technical decisions across teams as a...Hourly payContract workFor contractorsMonday to FridayShift work2 days per week3 days per week
Openkyber
Atlanta, GA2 days ago - ...Role: Senior Data Engineer Location: Dallas, TX (Onsite) 6-12 Months Responsibilities Design and Implement scalable solutions for ever-increasing data volumes, using big data/cloud technologies like Pyspark, Kafka, etc.Collaborate with cross-functional teams to understand...
Openkyber
Atlanta, GA2 days ago - ...Required Qualifications: Bachelor's degree in Information Systems, Information Technology, Computer Science, Engineering or equivalent experience Strong experience with SQL and PL/SQL development Experience working with relational databases such as SQL Server...
Openkyber
Atlanta, GA2 days ago - ...Job Title: Senior Databricks Engineer Location: 100% Remote Interview Process: Video (2 Rounds) Job Description: Lead/Senior Databricks Data Engineer Position: Lead Databricks Data Engineer / Databricks Architect Experience: 10+ Years Location: Remote...Second jobRemote work
Openkyber
Atlanta, GA2 days ago - ...Position: Senior Data Engineer Location: Boston, MA / Providence RI (Hybrid) Job Description: Required Qualifications 8... ...accountability, urgency, and continuous improvement. Experience leading technical initiatives and influencing teams without relying...
Openkyber
Atlanta, GA2 days ago - ...Description: Senior Data Engineer - Data Platforms Role Overview We are seeking a Senior Data Engineer to design, build, and maintain scalable data engineering solutions that power analytics, reporting, and data products across the organization. This role works closely...Worldwide
Openkyber
Atlanta, GA2 days ago - ...Top Skills: Big Data exp: Spark, Nifi pipelines, data bricks Java, SQL, and PL/SQL ETL Development (They use SSIS)... ...to be available for onsite if needed) Job Overview: Leads software engineering projects and creates the development and delivery of enhanced...
Openkyber
Atlanta, GA2 days ago $180k - $190k
...Location: Tustin, CA Salary: $180,000.00 USD Annually - $190,000.00 USD Annually Description: Lead Data Engineer (EDO Platform Engineering Lead) Location: Irvine, CA Salary: $180,000-$190,000 About the Role We are seeking an experienced Lead Data Engineer...Work at officeImmediate startOpenkyber
Atlanta, GA2 days ago- ...Lead Data Engineer - Gen AI Location: West Sacramento, CA Hybrid (2 3 days onsite per week) Employment Type: Contract Duration: 12 36 months About the Role We are seeking a Lead Developer Data, Business Intelligence & Generative AI to provide hands-on...Contract work2 days per week3 days per week
Openkyber
Atlanta, GA2 days ago - ...We have an opening for Senior Data Engineer at Jersey City, NJ . Kindly apply with your updated resume if you are interested. Please find the job details below: Position: Senior Data Engineer Location: Jersey City, NJ (In-Person Interview) Duration:...Contract work
Openkyber
Atlanta, GA2 days ago - ...RESPONSIBILITIES: OpenKyber has a client seeking a Lead Data Engineer in Fort Lauderdale, FL to drive the design and delivery of modern Customer 360, Identity Graph, and AI-enabled data platform solutions. Summary: This role will lead the development of scalable...Hourly payContract work
Openkyber
Atlanta, GA2 days ago - ...Support the design and implementation of the data platform, including environment setup and... ...and unstructured sources. Establish engineering standards for code review, testing,... ...Demonstrated experience as a technical lead or senior authority on data platforms, including...For contractorsLocal area
Openkyber
Atlanta, GA2 days ago - ...you to discuss about this opportunity in details. Role 1: Sr. Data Engineer with Databricks, Python, Salesforce Automation Role 2: Sr. Data... ...source system analysis. Thanks & Regards, OpenKyber LEAD TALENT ACQUISITION SPECIALIST Direct Number:- Fax : Email: Website...Long term contract
Openkyber
Atlanta, GA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!
