Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Real-Time Data Engineer - Spark Pipelines (Charlotte)

Apex Systems

Job#: 3040573 Job Description: Big Data Engineer Location: Charlotte, North Carolina (On-site) Role Overview We are seeking a Big Data Developer/Engineer to join a dynamic team. This role involves the end-to-end design and build process of near-real-time and batch data pipelines. The position requires a self-starter who can work with minimal supervision and adapt to new technologies within an Agile development environment. Key Responsibilities Design and build near-real-time and batch data pipelines. Perform data ingestion from various sources including message queues, file shares, REST APIs, and relational databases. Handle data formats such as JSON, CSV, and XML. Develop and optimize solutions using Spark Structured Streaming. Conduct performance tuning for Spark, MapReduce, or SQL jobs. Work with source code and version control systems like Git and Bitbucket. Collaborate with a team of diverse skill sets to comprehend customer requests and provide effective solutions. Utilize Jenkins for continuous integration and manage JAR dependencies. Required Qualifications Technical Skills: Proficiency in SQL, with experience in one or more of the following: MySQL, HIVE, Impala, or Spark SQL. Experience with Hadoop/Big Data ecosystems and distributed systems such as CDP, HDP, Cloudera, or Hortonworks. Working knowledge of Spark, Sqoop, Kafka, MapReduce, and NoSQL databases like HBase or SOLR. Programming experience in at least one of the following: Scala, Python, PHP, or Shell Scripting. Experience with Elastic Search and Kibana. Proficiency with the Linux operating system. Understanding of the Software Development Life Cycle (SDLC) and Agile methodologies. Everforth Apex is a world-class IT services company that serves thousands of clients across the globe. When you join Everforth Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRateds Best of Staffing® in Talent Satisfaction in the United States and Great Place to Work® in the United Kingdom and Mexico. Everforth Apex uses a virtual recruiter as part of the application process. Click here for more details. By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Everforth Apex and its affiliates, and contracted partners. Frequency varies for text messages. Message and data rates may apply. Carriers are not liable for delayed or undelivered messages. You can reply STOP to cancel and HELP for help. You can access our privacy policy at Everforth Apex Benefits Overview: Everforth Apex offers a range of supplemental benefits, including medical, dental, vision, life, disability, and other insurance plans that offer an optional layer of financial protection. We offer an ESPP (employee stock purchase program) and a 401K program which allows you to contribute typically within 30 days of starting, with a company match after 12 months of tenure. Everforth Apex also offers a HSA (Health Savings Account on the HDHP plan), a SupportLinc Employee Assistance Program (EAP) with up to 8 free counseling sessions, a corporate discount savings program and other discounts. In terms of professional development, Everforth Apex hosts an on-demand training program, provides access to certification prep and a library of technical and leadership courses/books/seminars once you have 6+ months of tenure, and certification discounts and other perks to associations that include CompTIA and IIBA. Everforth Apex has a dedicated customer service team for our Consultants that can address questions around benefits and other resources, as well as a certified Career Coach. You can access a full list of our benefits, programs, support teams and resources within our ‘Welcome Packet’ as well, which an Everforth Apex team member can provide. Everforth Apex Systems is an equal opportunity employer. We do not discriminate or allow discrimination on the basis of race, color, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Everforth Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you require an accommodation under the Americans with Disabilities Act to participate in an interview with a virtual recruiter or to use our website for a search or application, please contact our Benefits Department at [email protected] or View phone number on click.appcast.io. Please note that this contact information is strictly to be used for medical ADA accommodations and that no other inquiries will be answered. UnitedHealthcare creates and publishes the Transparency in Coverage Machine-Readable Files on behalf of Everforth Apex Systems. © 2026 Everforth, Inc. All rights reserved. Everforth Apex Systems is part of the Commercial Segment of Everforth, Inc.

NYSE: EFOR

4400 Cox Road Suite 200 Glen Allen, Virginia 23060 Everforth Apex is an equal opportunity employer. We do not discriminate or allow discrimination on the basis of race, color, religion, creed, sex (including pregnancy, childbirth, breastfeeding, or related medical conditions), age, sexual orientation, gender identity, national origin, ancestry, citizenship, genetic information, registered domestic partner status, marital status, disability, status as a crime victim, protected veteran status, political affiliation, union membership, or any other characteristic protected by law. Everforth Apex will consider qualified applicants with criminal histories in a manner consistent with the requirements of applicable law. If you have visited our website in search of information on employment opportunities or to apply for a position, and you require an accommodation in using our website for a search or application, please contact our Benefits Department at [email protected] or View phone number on click.appcast.io. (Do not submit resumes or solicit consultants to this email address). UnitedHealthcare creates and publishes the Transparency in Coverage Machine-Readable Files on behalf of Everforth Apex. #J-18808-Ljbffr Apex Systems

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Real-Time Data Engineer - Spark Pipelines (Charlotte) in Charlotte, NC vacancy
  • Data Systems Engineer (AWS, Snowflake, RedShift, Python, Scala, Hadoop, Spark, Kafka, Hive, API, Handling, API Development...  ..., Batch Data Pipelines) in Charlotte, NC API...  ...NC Duration: Full-Time ONSITEposition (no...  ...build process of Near-Real Time and Batch Data... 
    Pipeline
    Full time
    Local area
    Immediate start
    Remote work

    DBA Web Technologies

    Charlotte, NC
    more than 2 months ago
  • Capgemini is looking for a GCP Python Lead Data Engineer in Charlotte or New York City. The ideal candidate...  ...include developing data pipelines and ETL workflows, collaborating with...  ...troubleshooting data issues. We offer a full-time position with comprehensive health benefits... 
    Pipeline
    Full time

    Capgemini

    Charlotte, NC
    3 days ago
  • Capgemini is looking for a GCP Python Lead Data Engineer in Charlotte, NC. This role involves designing and...  ...hands-on experience in building data pipelines for analytics and AI/ML applications....  ...benefits package including paid time off, medical coverage, and retirement... 
    Pipeline

    Capgemini

    Charlotte, NC
    3 days ago
  •  ...Role: Lead Data Engineer with Gen AI Location: Charlotte, NC Duration: Long Term Contract Key Responsibilities Design...  ...(SoRs) by building reliable data pipelines, APIs, and connectors across...  ...analytics, feature extraction, and real-time agent decisioning. Implement observability... 
    Pipeline
    Long term contract

    Realtech

    Charlotte, NC
    1 day ago
  • Vanguard is seeking a Data Engineer in Charlotte, North Carolina to develop ETL pipelines and support our analytics organization. The role entails troubleshooting software, designing database systems, and translating specifications into actionable code. The ideal candidate... 
    Pipeline

    Vanguard

    Charlotte, NC
    3 days ago
  • Gravity IT Resources is seeking a Senior Data Engineer for a 2 to 3 year hybrid contract in Charlotte, NC. In this technical leadership role, you will design and deploy complex data pipelines while leading the development team. We're looking for someone with 8 to 15 years... 
    Pipeline
    Contract work
    H1b
    Visa sponsorship

    Gravity IT Resources

    Charlotte, NC
    2 days ago
  • Capgemini seeks a GCP Python Data Engineer based in the U.S. (Charlotte preferred). This role involves designing and developing scalable data pipelines using GCP services, optimizing ETL/ELT workflows, and collaborating with cross-functional teams. The ideal candidate... 
    Pipeline
    Flexible hours

    Capgemini

    Charlotte, NC
    3 days ago
  •  ...Charlotte, NC | Onsite 2 to 3 year contract |...  ...looking for a Senior Data Engineer for a Fortune 150...  ...role with real scope, real scale,...  ...deployment of complex data pipeline components Provide...  ...and maintain real-time, event-driven data...  ...to have: PySpark, Spark, SQL, Hadoop, CI/... 
    Pipeline
    Contract work
    H1b
    Visa sponsorship

    Gravity IT Resources

    Charlotte, NC
    2 days ago
  •  ...Overview Big Data Engineer – Charlotte, North Carolina (On-site). Role Overview: We are seeking...  ...design and build process of near-real-time and batch data pipelines. The position requires a self-...  ...Develop and optimize solutions using Spark Structured Streaming. Conduct... 
    Pipeline

    Apex Systems

    Charlotte, NC
    4 days ago
  • $77.6k - $176k

     ...Spark Data Engineer, Senior Ever-expanding technology like IoT, machine learning, and artificial...  ...the industry. You’ll deploy and develop pipelines and platforms that organize and make disparate...  ...demonstration of our values. Full‑time and part‑time employees working at... 
    Pipeline
    Full time
    Contract work
    Part time
    Local area

    Booz Allen Hamilton

    Charlotte, NC
    1 day ago
  • $100.96k - $151.44k

     ...About the Role Data Engineer - GE08AE. We’re determined to...  ...assets and build data pipelines that align with modern...  ...developing code that enables real‑time modeling solutions to...  ...databases, Hadoop/Spark, cloud data sources,...  ..., IL, Hartford, CT or Charlotte, NC) 3 days a week (Tuesday... 
    Pipeline
    Temporary work
    Work at office
    3 days per week

    The Hartford

    Charlotte, NC
    5 days ago
  • $100k - $110k

     ...Cloud & DevOps, Data, and Software Engineering, servicing an array...  ...efficient data pipelines and analytics...  ...if filled within Charlotte, NC is $100K - $...  ...Apache Hadoop, Spark, and Hive. Implement...  ...and provide timely resolutions. Stay...  ...data modeling, and real‑time data... 
    Pipeline
    Temporary work
    Flexible hours

    Synechron

    Charlotte, NC
    2 days ago
  • Senior Data Engineer Location: Charlotte, NC, Tallahassee, FL, Springfield, IL, Lansing, MI, Alpharetta, GA...  ...solutions using Azure, Databricks, Spark, SQL, DBT, Python, and Airflow. This...  ...scale data processing frameworks and pipelines. Develop and maintain reusable data... 
    Pipeline

    TechniPros

    Charlotte, NC
    4 days ago
  •  ...Job Click Here Senior Data Engineer | Onsite | Charlotte, NC | H1B Sponsorship...  ...leadership role with real scope, real scale,...  ...deployment of complex data pipeline components Provide...  ...and maintain real‑time, event‑driven data ingestion...  ...to have: PySpark, Spark, SQL, Hadoop, CI/CD... 
    Pipeline
    Contract work
    H1b
    Visa sponsorship

    Gravity IT Resources

    Charlotte, NC
    3 days ago
  • $135.04k - $202.56k

    Sr Staff Data Engineer - GE07DE We’re determined to make...  ...frameworks and tools (e.g., Spark, Snowflake),...  ...with DevOps / DataOps pipelines, cloud platforms (AWS...  ...office (Hartford, CT or Charlotte, NC) 3 days a week (Tuesday...  ...both batch and near‑real‑time environments, leveraging... 
    Pipeline
    Temporary work
    Work at office
    3 days per week

    The Hartford

    Charlotte, NC
    1 day ago
  •  ...leading staffing and consulting firm is seeking a skilled Data Engineer in Charlotte, NC. This role requires strong expertise in Apache Spark, Python, and AWS data services to design and develop scalable data pipelines. You will collaborate with teams to ensure data... 
    Pipeline

    Insight Global

    Charlotte, NC
    4 days ago
  •  ...Position: GCP Data Engineer Client Location: Charlotte, NC Duration: Fulltime Visa Sponsorship: No Visa sponsorship...  ...Experience in Data Engineering ETL Pipelines Batch and Streaming along with SQL...  ...package to all regular, full‑time employees. In the U.S. and Canada,... 
    Pipeline
    Full time
    Local area
    Visa sponsorship

    Smart Caliber Technology Inc

    Charlotte, NC
    2 days ago
  •  ...Job is located in Charlotte, NC - Onsite Hybrid Day 1 Job Description: GCP Python Lead Data Engineer We are seeking a highly skilled GCP...  ...developing batch and real-time data pipelines, working with large-scale...  ...Experience with Dataproc (Spark/PySpark) for large-scale processing... 
    Pipeline
    Full time
    Local area

    Capgemini

    Charlotte, NC
    2 days ago
  •  ...IDR is seeking a Sr Data Engineer to join one of our top clients in Charlotte, NC as a long term opportunity...  ...- Apply today! At this time, we are only able to...  ...models, building automated pipelines, and establishing data quality...  ...with high visibility, real ownership, and a modern... 
    Pipeline

    IDR, Inc.

    Charlotte, NC
    2 days ago
  • $108.8k - $136k

     ...healthcare navigation as a Data Engineer II (Analytics Engineer...  ...patterns Monitor pipeline health, troubleshoot incidents...  ...to this job at any time Preferred...  ...practices Familiarity with Spark/Databricks and cloud data...  ...NYC, Denver, CO, and Charlotte, NC area) Salary NewYork... 
    Pipeline
    Local area
    Flexible hours

    Judi Health

    Charlotte, NC
    1 day ago
  •  ...Aurora Experience with AWS data warehousing tools such...  ...and refining data pipelines within AWS Proficient in handling both real-time and batch data processing...  ...and Languages: Python, Spark, PySpark, and Pandas Infrastructure...  ...Lead Developers (Data Engineer, Software Engineer,... 
    Pipeline

    Brooksource

    Charlotte, NC
    1 day ago
  •  ...services company to hire a Data Engineering Associate Manager. **...  ...Associate Manager in Charlotte, NC, and the job...  ...for batch or real-time consumption patterns....  ...high-performance data pipelines and integrated solutions...  ...platforms, including Hadoop, Spark, Snowflake, and... 
    Pipeline

    Inclusively

    Charlotte, NC
    5 days ago
  •  ...well-known Financial Services Company is looking for a Data Pipeline Engineer in Charlotte, NC or Irving, TX (Hybrid). Work with the brightest minds...  ...develop ETL/ELT workflows and data pipelines for batch and real-time processing (GCP or any other cloud platforms are fine e.... 
    Pipeline
    Long term contract
    Contract work

    Motion Recruitment

    Charlotte, NC
    1 day ago
  •  ...The Senior Data Engineer will design, develop, and optimize scalable...  ...building high-performance ETL/ELT pipelines, implementing data...  ...practices. Support batch and near real-time data processing requirements...  ...equivalent. Experience with Apache Spark or PySpark for distributed... 
    Pipeline

    Compunnel

    Charlotte, NC
    2 days ago
  • A global consulting firm is seeking a Big Data Engineer to design, develop, and optimize large-scale data processing...  ...big data technologies such as Apache Hadoop, Spark, and Hive. Responsibilities include maintaining ETL pipelines, collaborating with teams to solve data... 
    Pipeline

    Synechron

    Charlotte, NC
    2 days ago
  • $106.9k - $176.5k

     ...highly skilled Senior Consultant Data Engineer with expertise in cloud data...  .... Lead end-to-end data pipeline development, including data...  ...Databricks and experience with Spark for big data processing....  ...intelligence. Development of a real‑time data processing pipeline... 
    Pipeline
    Summer holiday
    Flexible hours

    EY

    Charlotte, NC
    2 days ago
  • $120k - $135k

    Tata Consultancy Services is seeking a Senior Cloud Solutions Engineer in Charlotte, NC. The role involves designing, building, and maintaining...  ..., working extensively with event-driven architectures and data-driven solutions. The successful candidate will have a Bachelor... 

    Tata Consultancy Services

    Charlotte, NC
    3 days ago
  • $111k - $209k

     ...are seeking an experienced Data Engineer with a strong focus on Microsoft...  ...Data Engineering & Pipeline Development Design, develop,...  ...manage Fabric Notebooks (Python, Spark, SQL) for advanced transformations...  ...best practices. Exposure to real‑time analytics, event‑driven... 
    Pipeline
    Temporary work
    Work experience placement
    Local area
    Remote work
    Relocation
    Flexible hours

    K&L Gates LLP

    Charlotte, NC
    5 days ago
  • $128k - $249k

     ...Summary We are seeking a Senior Data Engineer to join the firm. The Senior...  ..., and Azure AI Foundry data pipelines. The role involves mentoring...  ...and automation, utilizing Spark and CI/CD pipelines through...  ...AI Foundry pipelines and real‑time, event‑driven architecture using... 
    Pipeline
    Temporary work
    Local area
    Remote work
    Relocation
    Flexible hours
    Afternoon shift

    K&L Gates LLP

    Charlotte, NC
    2 days ago
  •  ...In need of a Lead Data Engineer (MongoDB), please find the details below...  .../2 days remote) Charlotte, NC / Irving, TX Requirements...  ..., and enterprise deployment pipelines. Project Details:...  ...SQL and MongoDB solutions for real-time data reporting, Kafka-driven... 
    Pipeline
    Contract work
    Immediate start
    Remote work

    Saxon Global

    Charlotte, NC
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Real-Time Data Engineer - Spark Pipelines (Charlotte). Be the first to apply!