Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Technology Lead | Big Data - Data Processing | Spark

Ipolarity LLC

Job title - Technology Lead

Work location Atlanta, GA-30313

Is it Hybrid, onsite or remote position Full Onsite

Please confirm if the candidate can work from any Infosys DC office Nope, Client Preferred location is Atlanta

Tentative Start date 19-Oct

Contract duration greater than 6 months

Vendor rate ####

Minimum years of experience needed in the required skills- 5-8 years of experience

Minimum over all work experience required - 8-10 years of experience

Domain - CPG

Any certification required Good to Have

Mandatory skills required - Pyspark, Spark, Azure Synapse, Scala. SQL

Complete job description Support and maintain scalable Azure data platforms built on ADLS, Azure Synapse, Spark, ADF, and Delta Lake. Monitor and troubleshoot batch and incremental data pipelines, resolve production issues, and optimize jobs to ensure performance, reliability, and data availability. Maintain data models across the Bronze, Silver, and Gold layers to support reporting and analytics. Enforce security, governance, and operational best practices while mentoring engineers, maintaining support documentation, and driving continuous service improvement.

Interview mode - In person/Virtual - Virtual

How many rounds of interview Two Rounds

Project Code: COKE DnA IQ Support

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Technology Lead | Big Data - Data Processing | Spark in Atlanta, GA vacancy
  •  ...Job Description: Technology Lead Data Management (Talend & Snowflake) Job Title: Technology Lead...  ...and real-time ETL/ELT integration processes using the Talend platform. Snowflake...  ...using Talend Data Integration / Big Data platform for building enterprise ETL... 
    Big data
    Contract work

    Argyllinfotech

    Atlanta, GA
    1 day ago
  •  ...Experient, we provide experience, technology, and transformation...  ...a Senior Snowflake Data Engineer to join our development...  ...used by our client to process business critical order transaction...  ...~ Knowledge of big data technologies (Hadoop, Spark, Kafka) and data visualization... 
    Big data

    Experient

    Atlanta, GA
    3 days ago
  • $100k - $135k

     ...SQL and Apache SparkETL/Data Pipeline DevelopmentBig Data technologies: Hadoop, Spark, Kafka, Hive, HDFS,...  ...warehouse and large-scale data processing experienceExperience...  ...scalable solutions for Big Data processing and...  ...into technical solutions.Lead and mentor junior data... 
    Big data

    Tata Consultancy Services

    Atlanta, GA
    3 days ago
  •  ...We are seeking an experienced Technology Lead / Azure Data Engineer with strong hands-on expertise in Azure Synapse Analytics and...  ...candidate will have a deep understanding of Apache Spark, distributed data processing, Data Lake architecture, and modern data engineering... 
    Suggested
    Contract work

    2T Consulting

    Atlanta, GA
    a month ago
  •  ...Senior Data EngineerSkill - Last Used - Experience in Years/...  ...to the fast-paced world of Big Data technology and has experience in building...  ...transforming the way Client captures, processes, stores and visualizes data....  ...including Hadoop, Spark, and related frameworks.Programming... 
    Big data

    Omni Inclusive

    Atlanta, GA
    2 days ago
  •  ...Job Title: Lead Data Engineer Work Location: Newark, New Jersey 07068...  ...CloudFormation, S3, Athena, Glue, EMR/Spark, RDS, Redshift, DynamoDB, Lambda, Step...  ...Knowledge of ETL/ELT, data modeling, and big data technologies. ~ Familiarity with data warehousing... 
    Big data
    Long term contract
    Contract work

    Openkyber

    Atlanta, GA
    2 days ago
  •  ...Azure Data EngineerSeeking an Azure Data...  ...and customers, and leading the design and hands...  ...private cloud technologies, our Data Engineer...  ...perspective on storage, big data platform...  ...data ingestion, data processing, and analytical...  ..., SQL, ADLS/Blob, Spark/SparkSQL, Databricks... 
    Big data
    Work experience placement

    Software Technology Inc

    Atlanta, GA
    2 days ago
  •  ...Description Job Description Data Scientist (5–10 Years...  ...tests to assess the impact of process or product changes. 10. Data...  ...audiences. Familiarity with big data and distributed computing frameworks (e.g., Hadoop, Spark) is a plus. Awareness of data... 
    Big data

    4P Consulting Inc.

    Atlanta, GA
    a month ago
  •  ...Role : AWS Lead Data Engineer (3 Open Positions) Location...  ...with advancements in AWS technologies and data engineering...  ...Redshift, S3, Lambda, EMR/Spark, Kinesis, Athena, RDS, API...  ...experience with ETL/ELT processes, data modeling, and big data frameworks. ~ Demonstrated... 
    Big data
    Work at office

    Openkyber

    Atlanta, GA
    2 days ago
  • $150k - $170k

     ...be submitted Role: Senior Data Engineer Salary: $150,000-1...  .... Data Engineering & Processing Develop ETL/ELT pipelines using...  .... Optimize performance for Spark workloads and Redshift queries...  ...Tableau, Quicksight, Looker, big data and Realtime BI experience... 
    Big data
    Local area
    Remote work

    LHH

    Atlanta, GA
    3 days ago
  •  ...Statistics, or equivalent professional experience Experience building data processing pipelines and large scale machine learning platforms with experience in big data technologies like Spark, SQL, Snowflake/Hadoop, etc Experience operationalizing distributed... 
    Big data
    Long term contract
    Contract work

    Openkyber

    Atlanta, GA
    1 day ago
  •  ...OpenKyber is hiring a Lead Snowflake Software Developer...  ...experience with a strong focus on data engineering, analytics,...  ...data modeling, ETL/ELT processes, database management, and...  ...Strong experience with Apache Spark and Azure, with proficiency in big data processing, analytics... 
    Big data
    Contract work

    Openkyber

    Atlanta, GA
    1 day ago
  •  ...product/business partners, data engineers, data...  ...Augmentation & Services team. Lead the entire product...  .... Knowledge of Big Data Technologies: Experience with big...  ...technologies (e.g., Hadoop, Spark, Kafka) can help in...  ...and optimizing data processing. User Experience (UX... 
    Big data

    RIT Solutions

    Atlanta, GA
    1 day ago
  •  ...Databricks Data EngineerAre you fascinated by information technology and its role in innovative business solutions...  .... Develop scalable processing solutions using PySpark, Python, Spark SQL, SQL, Delta Lake, and...  ...Engineering, ETL/Data Warehouse, or Big Data experience, including... 
    Big data
    Work at office
    Local area
    Remote work

    CGI

    Atlanta, GA
    7 hours ago
  •  ...Lead Software Engineer (Big Data, Spark, AWS, Full Stack) Location: Rockville, MD or Tysons Corner, VA Work...  ...Hive, and Trino ~ Experience processing large-scale datasets (TB/PB scale)...  ...Design Patterns ~ Data Storage Technologies (Relational & NoSQL) ~ Performance... 
    Big data

    Openkyber

    Atlanta, GA
    2 days ago
  • $84 per hour

     ...Job Title: Data Engineer Cloud Data & Cybersecurity Analytics...  ...maintain AWS Glue ETL pipelines to process, transform, and integrate...  ...solutions using Python, SQL, and Spark . Support data governance...  ...cloud-based data engineering and big data processing. ~ Strong... 
    Big data
    Contract work
    H1b
    Local area

    Openkyber

    Atlanta, GA
    1 day ago
  •  ...candidate will have a strong Data Engineering background with...  ...extensive experience working with Big Data technologies . Day-to-Day...  ...established development practices, processes, and reporting standards. The...  ...technologies. Experience with Spark for large-scale batch processing... 
    Big data
    Contract work
    Work at office
    Immediate start
    Remote work
    3 days per week

    Openkyber

    Atlanta, GA
    2 days ago
  •  ...Senior AWS Tech Lead Data Engineer Start ASAP (2-4 WEEKS...  ...'t need architects or Big Data engineers. We...  ...a strong focus on AWS technologies About the Role: We are...  ...scalable data pipelines and processing systems. The candidate...  ...such as Apache Spark, Hadoop, or Kafka. Experience... 
    Big data
    Immediate start
    Remote work

    Openkyber

    Atlanta, GA
    1 day ago
  •  ...seeking an experienced Senior Data Engineer to design, develop,...  ..., and distributed data processing technologies. This role will play a critical...  ...ecosystems leveraging Databricks, Spark, Delta Lake, Snowflake,...  ...data ecosystems. Big Data & Streaming Solutions:... 
    Big data

    Openkyber

    Atlanta, GA
    2 days ago
  •  ...an experienced Senior Data Engineer to join a large...  ...maintain scalable ETL/ELT processes and data pipelines to...  ...in Information Technology with significant experience...  ...Azure . Experience with big-data or distributed-data technologies such as Spark, Kafka, Hadoop, or NoSQL... 
    Big data
    Permanent employment
    Contract work
    Visa sponsorship

    Openkyber

    Atlanta, GA
    2 days ago
  •  ...CTG is seeking to fill a Data Engineer - AI &...  ...batch and real-time data processing pipelines. Create...  ...with Databricks, Apache Spark, or similar cloud data...  ...Language Processing (NLP) technologies is a plus. Excellent...  ...cloud data platforms, or big data development.... 
    Big data
    Remote work

    Openkyber

    Atlanta, GA
    1 day ago
  •  ...00% Remote Interview Process: Video (2 Rounds) Job...  ...: Job Description Lead/Senior Databricks Data Engineer Position: Lead...  ...Databricks, Apache Spark, PySpark, SQL, Delta...  ...workload optimization Big Data Apache...  ...Engineering, Information Technology, or related field. 10... 
    Big data
    Second job
    Remote work

    Openkyber

    Atlanta, GA
    2 days ago
  •  ...Top Skills: Big Data exp: Spark, Nifi pipelines, data bricks Java, SQL...  ...needed) Job Overview: Leads software engineering...  ...Managers in implementation of processes per the evolving needs of the...  ...information systems, Information Technology, Computer Science,... 
    Big data

    Openkyber

    Atlanta, GA
    2 days ago
  •  ...Position: Big Data Engineer Length: 24+ months...  ...Unity Catalog, Apache Spark (Spark SQL / PySpark),...  ...proficiency with these technologies and the ability to...  ...architecture principles and help lead initiatives from...  ...manipulating, processing, and extracting value... 
    Big data
    Shift work

    Stellent IT LLC

    Atlanta, GA
    1 day ago
  • $10,214.44 - $10,678.1 per month

     ...78.10 monthly SUMMARY The Data & Analytics Unit is responsible...  ...ETL (Extract, Transform, Load) processes and tools, data modeling, and data...  ...practices. Proven experience with big data technologies and frameworks (e.g., Hadoop, Spark, Kafka). Deep understanding... 
    Big data

    Openkyber

    Atlanta, GA
    2 days ago
  •  ..., and acquirers trust our technology to safeguard more than one...  ...transactions each year. The Data Science Team within...  ...Research, etc.) to improve processes, best practices and...  ...overfit, etc.); Knowledge of Big Data technologies such as Spark, Hadoop and related; Proficiency... 
    Big data
    Contract work
    Temporary work
    Work at office
    Remote work
    Atlanta, GA
    9 days ago
  •  ...Seeking a data engineer for hybrid contract...  ...skills/MCP Job Processing Framework What this...  ...data engineering and Big Data development,...  ...in PySpark/Spark for distributed data...  .... Track record of leading workstreams and driving...  ...with streaming technologies such as Kafka, Spark... 
    Big data
    Hourly pay
    Contract work
    For contractors
    Monday to Friday
    Shift work
    2 days per week
    3 days per week

    Openkyber

    Atlanta, GA
    2 days ago
  • OverviewThe Infosys Data and Analytics (DNA)...  ...with cutting-edge technologies, collaborate with industry...  ...elicitation process by documenting assigned...  ...ExperiencePyspark, Spark, Scala, Azure...  ...Skill 1Technology|Big Data - Data Processing...  ...2Career RoleTechnology Lead - US: 153311BR
    Big data
    Full time
    Temporary work
    Relocation

    Infosys Technologies

    Atlanta, GA
    4 days ago
  •  ...Description 5-10 Years This role is for someone with advanced data analysis skills and a firm grasp of the scientific method....  ...familiar with a variety of Machine Learning algorithms and big data processing frameworks. In addition to building machine learning and other... 
    Big data

    4P Consulting Inc.

    Atlanta, GA
    a month ago
  • $157.1k - $212.6k

     ...customers leverage emerging technologies that directly impact their...  ...a managed petabyte-scale data warehouse for fast SQL...  ...data; Amazon EMR a managed big data platform for Spark Hive and Presto workloads;...  ...real-time data streaming and processing; and Amazon OpenSearch a... 
    Big data
    Full time
    Local area
    Flexible hours

    Amazon

    Atlanta, GA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Technology Lead | Big Data - Data Processing | Spark. Be the first to apply!