Data Engineer -Iceberg
American IT Systems
Data Engineer
Client: Elevance Health
Location: Atlanta, GA
Rate: $60 65/hr on C2C
Experience: 8-12+ years
Job Description
We are seeking an experienced Data Engineer with strong hands-on experience in SQL, Python, SSIS, Airflow, Apache Spark, Apache Iceberg, and Databricks . The ideal candidate will be responsible for designing, developing, and optimizing scalable data pipelines and data processing solutions in a cloud-based environment.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Python, SQL, SSIS, Airflow, and Spark.
- Develop complex SQL queries, stored procedures, data transformations, and performance optimization solutions.
- Build and maintain data pipelines and workflows using Apache Airflow , including scheduling, dependencies, monitoring, retries, and failure handling.
- Develop and optimize Apache Spark/PySpark applications for large-scale data processing.
- Work with Databricks to develop, deploy, and optimize data engineering workloads.
- Work with Apache Iceberg tables and modern data lake/lakehouse architectures.
- Migrate and modernize legacy SSIS ETL workflows into cloud-based and Spark/Databricks-based data pipelines.
- Implement data ingestion, transformation, validation, and integration processes from multiple sources.
- Optimize Spark jobs, SQL queries, data partitioning, and storage strategies for performance and scalability.
- Implement data quality checks, error handling, logging, monitoring, and pipeline recovery mechanisms.
- Collaborate with data architects, developers, analysts, and business teams to understand requirements and deliver reliable data solutions.
- Follow best practices for data security, governance, performance, and operational reliability .
Required Skills
- Strong hands-on experience with SQL
- Strong programming experience with Python
- Hands-on experience with Apache Spark / PySpark
- Experience with Databricks
- Strong experience with Apache Airflow
- Hands-on experience with SSIS
- Experience with Apache Iceberg
- Strong understanding of ETL/ELT and data pipeline development
- Experience working with large-scale datasets and distributed data processing
- Strong troubleshooting and performance-tuning skills
- Good understanding of modern data lake/lakehouse architectures
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Data Engineer -Iceberg in Atlanta, GA vacancy
$160.06k - $183.33k
...USAJob Family GroupEngineering / Product DevelopmentJob ProfileLead Data EngineerManagement LevelManager - Non People LeaderFlexible Work... ...Services, LLCLead Data EngineerJob Description: Lead Data Engineer positions offered by Cox Automotive Corporate Services, LLC (Atlanta...SuggestedFull timeWork at officeRemote workFlexible hours- Inspire Brands is hiring a Lead Data and AI Engineer for the Enterprise Data Organization to design, build and manage data pipelines (Data ingestion, data transformation, data distribution, quality rules, data storage etc.) using cutting edge AI solutions (Generative AI...SuggestedWorldwide
- ...uphold organizational standards. Develops business models to represent requirements, such as use cases, business process flows, and data flow diagrams.Negotiates agreements and commitments. Facilitates communication between business units and Information Technology from...SuggestedWork at office
- Honeywell is accelerating its transformation from industrial automation to full autonomy and the data that powers that future starts here. As a Lead Data Engineer on our Industrial AI & Data Platforms team, you will architect and own the data foundations that enable physical...SuggestedPermanent employmentTemporary workRelocation packageFlexible hours
$116k - $145k
THE POSITIONOur roster has an opening with your name on itWe are looking for a Data Engineer to join our growing data engineering team and help build the pipelines and infrastructure that power analytics, machine learning, and business decision-making across the company...SuggestedTemporary workLocal areaWorldwide- Who You’ll Work WithAt Slalom, we co-create custom software, data, and cloud products with clients who are ready to accelerate their... ...organizations that drive true change.Our Data Architecture & Engineering team sees data as the foundation for decision-making, artificial...Temporary workLocal area
- We are looking for a Data Engineer to join an opportunity in Atlanta, Georgia. In this role, you will design and support reliable data solutions that enable efficient reporting, analytics, and downstream business insights across cloud-based platforms. The ideal candidate...
- ..., and government agencies, a list that spans across the country.Job DescriptionCapTech Machine Learning Engineers are responsible for designing and implementing data-driven solutions for our clients, with a specific focus on building and deploying scalable machine learning...Work at officeRemote workVisa sponsorshipWork visaFlexible hours
$89k - $148k
Job Family:Data Science & AnalysisTravel Required:Up to 10%Clearance Required:Ability to Obtain Public TrustWhat You Will Do:Guidehouse seeks a Data Engineer with some AI experience to support in building, optimizing, and maintaining data pipelines, AI solutions, and analytics...Full timeFlexible hours- Tätigkeitsbereich:IT/TelekommunikationFachabteilung:Data & TechnologyGesellschaft:Mercedes-Benz USA, LLCStandort:Mercedes-Benz USA, LLC... ...customer problems using data.We are seeking a Principal Data Engineer to serve as a senior technical leader for enterprise data engineering...Local areaFlexible hoursWeekend work
$55 - $60 per hour
We are looking for a Data Engineer (GCP) for a remote, contract opportunity. This is a W-2 contract directly with TEKsystems (we are not able to support sponsorship for this particular role_.Top Skills:• Strong GCP/SQL (expert-level querying, performance tuning, optimization...Contract workTemporary workRemote work$77.5k - $176k
Data EngineerThe Opportunity:Ever-expanding technology like IoT, machine learning, and artificial intelligence means that there’s more... ...unstructured data available today than ever before. As a data engineer, you know that organizing data can yield pivotal insights when it...Full timeContract workPart timeWork at officeLocal areaRemote work- ...of Business Analysts, Project Managers, Information Architects, Data Analysts, and/or Database Administrators to deliver quality... ...knowledge of code quality, CI/CD, and DevOps practices in data engineering Working knowledge of and experience in development of AI/ML pipelines...Work at officeRemote workWork from home
- ABOUT THIS POSITIONWaystar is seeking a highly skilled Senior Data Engineer to join our Business Data & Architecture team and help advance our enterprise Data Modernization Program on Google Cloud Platform (GCP).This is a senior individual contributor role focused on designing...Full timeLive outLocal areaFlexible hours
- ...status quo, and are driven to succeed. About the Role:As a Senior Engineer on the Systematic Trading Technology Team, you will collaborate... ...using the latest technology, optimizing trading workflows, data modeling/analytics, and building applications/interfaces that not...Full timeWork at officeFlexible hours
- As a Senior Data Engineer, you will be part of a high-performing global team delivering advanced AI and data solutions for Honeywell’s industrial customers, with a focus on IoT and real-time data processing. In this role, you will design and implement scalable data architectures...Permanent employmentTemporary workFlexible hours
$35 - $45 per hour
DescriptionKforce has a client that is seeking a remote Senior Data Engineer to join their team.Summary:We are seeking a Senior Data Engineer to support the design, development, and optimization of enterprise data platforms and large-scale data solutions. This role will...Remote work- ...are and the impact you could have on the food service industry. There's a seat at our table for you...Position Summary:The Senior Data Engineer guides the development of GFS’ Data Platform consisting of the data lake, analytics, enterprise data sandboxes, and certified...Full timeWork at officeMonday to Friday
- ...ideal candidate is a problem-solver who enjoys working on complex data systems and is passionate about data quality. You thrive in... ...confidently.About the RoleWe're seeking a talented Senior Data Engineer to join our Enterprise Architecture team in a cross-cutting role...
$80k - $133k
Job Family:Data Science & AnalysisTravel Required:Up to 10%Clearance Required:Ability to Obtain Public TrustWhat You Will Do:Guidehouse seeks a Senior Data Engineer to design, develop, and optimize modern data platforms, pipelines, and cloud-based analytics solutions. The...Full timeContract workFlexible hours- ...Lead Online Data EngineerWith a career at The Home Depot, you can be yourself and also be part of something bigger.Position Purpose:... ...most qualified and suitable candidate for the position.Online Data Engineer teams at The Home Depot translate business requirements and...Work experience placementNight shift
Home Depot
Atlanta, GA2 days ago $153.58k
...-Apply NowDuties: Responsible for the technical leadership for data ingestion and processing engagements. Support all our businesses... ...solutions, in a cloud environment, using a vast range of data engineering and cloud technologies. Work on tasks independently and seek guidance...Full timePart timeWork experience placementLocal areaRemote workFlexible hours- #CareersJC-1489187Data Engineer Onshore Key ResponsibilitiesDesign develop and maintain scalable data pipelines for batch and realtime data processingGather and analyze business and data requirements from customer stakeholdersBuild optimize and support ETLELT workflows...
- ...that is felt by all. See where your commitment could take you.Who are we looking for?GHD is seeking a Lead Consultant - Data Architecture & Engineering to join our Americas Business Advisory practice. This role is a senior seller‑doer position for a technically deep...Work at officeLocal area3 days per week
- ...damaging the planet. We aim to create software that delivers real-time data to the entire supply chain that allows producers to get better... ...Job Summary We are seeking a highly skilled and motivated Data Engineer to join our dynamic team at Speria MTech. The ideal candidate...Casual workRemote workFlexible hours
Speria
Atlanta, GA3 days ago - ...focused on our customers' needs. RESPONSIBILITIES: Work with data scientist team to migrate the analytical data and projects to... ...Accounts. REQUIRED SKILLS: ~3+ years of professional data engineering or data wrangling experience ~3+ years working with Python...Full timeContract workTemporary workFor contractorsWork experience placementLocal areaRemote workFlexible hours
Visionaire Partners
Atlanta, GA4 days ago - ...fulfill the position’s purpose. If possible, please include the percentage of time spent on each key responsibility. Advanced Data Engineering and Solution Design (80%) Architect and implement scalable data pipelines to process and integrate structured and unstructured...
ProArch
Atlanta, GA1 day ago - ...Data EngineerLocation: Remote - Atlanta, GA Preferred - Local Candidate Visa's: USC, GC, GC - EADRequired Skills:7 years of SSIS experience5+ years of Snowflake experienceSQL ServerSolid experience with Stored ProceduresProficiency with TriggersProficiency with CDC -...Local areaRemote work
ShiftCode Analytics
Atlanta, GA3 days ago - ...Jr. Data EngineerWe are seeking a Jr. Data Engineer to join our team. This role involves working in a regulated, production-critical enterprise environment. The position requires close collaboration with business stakeholders, strong delivery capabilities, and the ability...
Apex Systems
Atlanta, GA3 days ago - ...Data EngineerLocation: US-GA-Atlanta (Sandy Springs)FLSA: Exempt#LI-HybridJob OverviewThe Data Engineer leads complex data engineering projects and drives innovation in data architecture and automation.Job ResponsibilitiesOversee the end-to-end delivery of large-scale...Contract workTemporary workLocal area
Safe-Guard Products International, LLC
Atlanta, GA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer -Iceberg. Be the first to apply!
Related searches
- data engineer analytics Atlanta, GA
- principal data engineer Atlanta, GA
- senior cloud data engineer Atlanta, GA
- data center engineer Atlanta, GA
- director data engineering Atlanta, GA
- data engineer graduate Atlanta, GA
- big data devops engineer Atlanta, GA
- azure data engineer Atlanta, GA
- data engineer manager Atlanta, GA
- sr data engineer Atlanta, GA