Lead Data Engineer

Openkyber

Senior Data Engineer Job Description Job Title: Senior Data Engineer
Experience: 12+ Years

Job Type: W2 Contract
Job Summary
We are seeking a highly experienced Senior Data Engineer with 12+ years of experience in designing, developing, and maintaining scalable data platforms and data pipelines. The ideal candidate will have strong hands-on expertise in Python, SQL, Spark, Databricks, cloud technologies, ETL/ELT, data warehousing, and data integration .
Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines for enterprise applications.

Develop complex data transformation and processing workflows using Python, SQL, and Apache Spark .

Build and optimize data pipelines using Databricks and PySpark .

Design and implement data solutions on AWS, Azure, or Google Cloud Platform cloud platforms.

Develop data ingestion processes from databases, APIs, files, and other structured/unstructured sources.

Implement data quality, validation, reconciliation, and monitoring processes.

Design and optimize data warehouses, data lakes, and lakehouse architectures .

Work with relational and NoSQL databases to support data engineering requirements.

Perform performance tuning and optimization of SQL queries, Spark jobs, and data pipelines.

Implement orchestration using Apache Airflow, Azure Data Factory, AWS Glue, or similar tools .

Develop reusable frameworks and components for data ingestion and transformation.

Integrate data engineering solutions with CI/CD pipelines and DevOps processes.

Implement data security, access controls, encryption, and governance best practices.

Troubleshoot production data pipelines and resolve data processing issues.

Collaborate with data architects, analysts, data scientists, application developers, and business stakeholders.

Participate in Agile/Scrum ceremonies and contribute to technical design and architecture discussions.

Mentor junior and mid-level data engineers and provide technical guidance.

Required Skills

12+ years of experience in Data Engineering, ETL, or related data technologies.

Strong hands-on experience with Python and SQL .

Extensive experience with Apache Spark / PySpark .

Strong experience with Databricks and Delta Lake.

Experience building enterprise-scale ETL/ELT pipelines .

Strong knowledge of data warehousing concepts, dimensional modeling, and data lake architectures.

Experience with one or more cloud platforms: AWS, Azure, or Google Cloud Platform .

Experience with databases such as SQL Server, Oracle, PostgreSQL, MySQL, Snowflake, or similar .

Experience with workflow orchestration tools such as Airflow, Azure Data Factory, AWS Glue, or similar .

Strong understanding of batch and near-real-time data processing.

Experience with Git, CI/CD, Jenkins, Azure DevOps, or GitHub Actions .

Strong understanding of data quality, data validation, and performance optimization.

Excellent analytical, troubleshooting, and communication skills.

Preferred Skills

Experience with Azure Data Lake, AWS S3, ADLS, or Google Cloud Storage .

Experience with Snowflake or other modern cloud data warehouses.

Knowledge of Kafka, Event Hubs, or other streaming technologies .

Experience with Terraform or Infrastructure as Code .

Knowledge of Docker and Kubernetes.

Experience implementing data governance, metadata management, and lineage .

Experience working with large-scale distributed data environments.

Knowledge of Medallion Architecture (Bronze, Silver, Gold) .

Experience with real-time/streaming data pipelines.

Experience in designing highly available and fault-tolerant data platforms.

For applications and inquiries, contact:View email address on us.fitly.work

Apply

Vacancy posted 2 days ago

Similar jobs that could be interesting for youBased on the Lead Data Engineer in Atlanta, GA vacancy

Sr. Lead Data Engineer
$205.32k
...USAJob Family GroupEngineering / Product DevelopmentJob ProfileSr Lead Data EngineerManagement LevelSr Manager - Non People LeaderFlexible... ..., LLCSr. Lead Data EngineerJob Description: Sr. Lead Data Engineer positions offered by Cox Automotive Corporate Services, LLC (Atlanta...
Suggested
Full time
Work at office
Local area
Remote work
Flexible hours
Cox Enterprises
Atlanta, GA
5 days ago
Lead Data Engineer
...Senior Data Engineer Malvern, PA (Hybrid -3 day's onsite): Duration: Long term uties Build out an extensive transaction stitching application to helps track the journey of money flowing through a client's account over the lifetime of the funds. Skills Advanced...
Suggested
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Seeking a data engineer for hybrid contract role in Westmont IL. If interested , share your word document resume, work authorization, expected... ...enterprise data for AI/agentic consumption. Track record of leading workstreams and driving technical decisions across teams as a...
Suggested
Hourly pay
Contract work
For contractors
Monday to Friday
Shift work
2 days per week
3 days per week
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Data Engineer IV Location: Remote (PST Preferred) Duration: 7-Month Contract (Through April 16, 2027) Employment Type: Contract | 40 Hours... ...rate: $110/hr. - $118/hr. on W2 About the Opportunity A Global Leading Technology Company is seeking an experienced Data Engineer IV...
Suggested
Contract work
Immediate start
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Top Skills: Big Data exp: Spark, Nifi pipelines, data bricks Java, SQL, and PL/SQL ETL Development (They use SSIS)... ...to be available for onsite if needed) Job Overview: Leads software engineering projects and creates the development and delivery of enhanced...
Suggested
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Role Summary: Seeking a Tech Lead to lead a team of developers in designing, developing, and delivering scalable AI/Data solutions. The role involves technical leadership,... ...Expertise in AI/LLM technologies: RAG, Prompt Engineering, Agentic Workflows, Vector Databases,...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Senior Data Engineer Location: Remote Employment: 6-Month Contract Experience: 4+ Years Job Summary: We are looking for a Senior Data Engineer with strong hands-on experience in dbt Core, dbt Cloud, Fivetran, and Snowflake . The candidate will design, develop, deploy,...
Contract work
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...We are seeking a highly experienced Senior AWS Data Engineer with AI/GenAI experience to design, develop, and support scalable cloud-based data platforms and intelligent data solutions. The ideal candidate should have extensive experience in AWS data services, Python,...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Required Qualifications: Bachelor's degree in Information Systems, Information Technology, Computer Science, Engineering or equivalent experience Strong experience with SQL and PL/SQL development Experience working with relational databases such as SQL Server...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Job Title: Data Engineer ( DATA ENGINEERING | DATA MODELING | PERFORMANCE) Location: Remote & hybrid, alternating weeks at the nearest office Duration: 4 months+ contract Number of Positions: 2 roles Responsibilities: DATA ENGINEERING | DATA MODELING | PERFORMANCE Senior...
Contract work
Work at office
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...We have an opening for Senior Data Engineer at Jersey City, NJ . Kindly apply with your updated resume if you are interested. Please find the job details below: Position: Senior Data Engineer Location: Jersey City, NJ (In-Person Interview) Duration:...
Contract work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Title: W2 Sr. Data Engineer Location: Onsite ( Covington, KY) Visa- NO OPT/CPT/TN & H1B The Role: This Senior Data Engineer role requires 5+ years of technology experience, including at least 3 years developing and deploying data solutions. The primary...
H1b
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...An enterprise technology company is seeking a Technology and Data Engineer 5 for a hybrid opportunity in Raleigh, North Carolina . This role is centered on strengthening Infrastructure as Code security across cloud environments and supporting secure software delivery practices...
Full time
Contract work
Work at office
Local area
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Description: Senior Data Engineer - Data Platforms Role Overview We are seeking a Senior Data Engineer to design, build, and maintain scalable data engineering solutions that power analytics, reporting, and data products across the organization. This role works closely...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Data/Database Administration - Data Engineer - Montvale, NJ - Contract W2/C2C - $55-65 per HR Join our team as a Data Engineer in a dynamic contract role based in Montvale, NJ. This position involves managing data platforms, supporting application operations, and troubleshooting...
Contract work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...you to discuss about this opportunity in details. Role 1: Sr. Data Engineer with Databricks, Python, Salesforce Automation Role 2: Sr. Data... ...source system analysis. Thanks & Regards, OpenKyber LEAD TALENT ACQUISITION SPECIALIST Direct Number:- Fax : Email: Website...
Long term contract
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
$180k - $190k
...Location: Tustin, CA Salary: $180,000.00 USD Annually - $190,000.00 USD Annually Description: Lead Data Engineer (EDO Platform Engineering Lead) Location: Irvine, CA Salary: $180,000-$190,000 About the Role We are seeking an experienced Lead Data Engineer...
Work at office
Immediate start
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Role : Sr. Data Engineer with strong SQL and Databricks experience Location : Irving, TX (Hybrid 3 days' Work from Office) Type : Contract Client Interview in-person in Irving, TX. Seeking a Senior Data Engineer with deep expertise in SQL Server, Azure SQL, and Databricks...
Contract work
Work at office
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Position: Senior Data Engineer Location: Boston, MA / Providence RI (Hybrid) Job Description: Required Qualifications 8... ...accountability, urgency, and continuous improvement. Experience leading technical initiatives and influencing teams without relying...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Role: Senior Data Engineer Location: Dallas, TX (Onsite) Duration: 6-12 Months Responsibilities: Design and Implement scalable solutions for ever-increasing data volumes, using big data/cloud technologies like Pyspark, Kafka, etc. Collaborate...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Job Description: We are seeking an experienced Senior Data Engineer to design, develop, and optimize scalable data platforms that power enterprise analytics, machine learning, and AI-driven solutions. The ideal candidate will bring deep expertise in modern data engineering...
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Role : Sr. Data Engineer Location: Pasadena, CA 100% Onsite Work Arrangement: Hybrid 3 Days/Week Onsite Candidate Requirement: Local candidates only within 50 Miles. Job Summary We are seeking an experienced Data Engineer with strong hands-...
Local area
3 days per week
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Description: Senior Data Engineer - Data Platforms Role Overview We are seeking a Senior Data Engineer to design, build, and maintain scalable data engineering solutions that power analytics, reporting, and data products across the organization. This role works closely...
Worldwide
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...details about the role. Position: Senior Technology Consultant - Data / AI Native Engineer Location: Arlington, VA || New York, NY || St. Louis, MO (... ...strong Software Engineering practices and has experience leading complex enterprise data initiatives. The role requires...
Long term contract
Local area
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
Minimum 10-12 years of hands-on engineer for Amazon Flink and Kafka streaming data engineer For applications and inquiries, contact:****@*****.***
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...Senior Data Engineer Remote prefer candidates form Charleston, SC Long Term W2 The Senior Data Engineer, leads the design and implementation of robust data solutions across multiple domains, driving technical excellence and scalability. This role mentor others, shape best...
Remote work
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
$65 - $70 per hour
...Title: Senior Data Engineer Location: Miramar, FL Duration: 12 months Compensation: $65-$70/hr Work Requirements: , Holders or Authorized to Work in the U.S. Overview The Data Engineer designs, builds, and maintains scalable data pipelines and infrastructure that...
Local area
Flexible hours
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
...We are seeking a Data Engineer with experience in projects emphasizing complex data solutions. This person will be responsible for supporting data transformation and distribution to end consumers. Candidate will assist with requirements analysis, design/develop process...
For contractors
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
$40 - $50 per hour
...job summary: We are seeking an experienced Databricks Data Engineer to join our high-performing Data & AI practice in Phoenix, AZ. In this role, you will be responsible for designing and deploying enterprise-grade Lakehouse solutions on Google Cloud Platform (Google Cloud...
Hourly pay
Contract work
Temporary work
Work experience placement
Openkyber
Atlanta, GA
2 days ago
Lead Data Engineer
$67 per hour
...Role : Senior Data Engineer(Medicaid or Healthcare Experience(Required)) Client: Government of Nebrasaka(NeDHHS) Work Type : Remote Duration : 12 Months Max Payrate : 67$ Per Hour OpenKyber is a fast-growing Systems Integrator based...
Hourly pay
Local area
Remote work
Openkyber
Atlanta, GA
2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!