Big Data Engineer
ShiftCode Analytics
Job TitleVisa: USC and GC only Atlanta, GA (Remote 100%) 2+ years contractSkills and Experience:Required:9+ years of overall IT experience4+ years of experience with high-velocity high-volume stream processing: Apache Kafka and Spark StreamingExperience with real-time data processing and streaming techniques using Spark structured streaming and KafkaDeep knowledge of troubleshooting and tuning Spark applications4+ years of experience with data ingestion from Message Queues (Tibco, IBM, etc.) and different file formats across different platforms like JSON, XML, CSV4+ years of experience with Big Data tools/technologies like Hadoop, Spark, Spark SQL, Kafka, Sqoop, Hive, S3, HDFS, or Cloud platforms e.g. AWS, GCP, etc.4+ years of experience building, testing, and optimizing 'Big Data' data ingestion pipelines, architectures and data sets2+ years of experience with Kudu and Impala2+ years of experience with Scala (and/or Python) and PySpark/Scala-Spark2+ years of experience with NoSQL databases, including HBASE and/or CassandraKnowledge of Unix/Linux platform and shell scripting is a mustStrong analytical and problem solving skillsPreferred (Not Required):Experience with Cloudera/Hortonworks HDP and HDF platformsExperience with NIFI, Schema Registry, NIFI RegistryStrong SQL skills with ability to write intermediate complexity queriesStrong understanding of Relational & Dimensional modelingExperience with GIT code versioning softwareExperience with REST API and Web ServicesGood business analyst and requirements gathering/writing skillsEducation Bachelor's Degree required. Preferably in Information Systems, Computer Science, Computer Information Systems or related field
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Big Data Engineer in Atlanta, GA vacancy
- ...Arete Technologies, Inc. is seeking a Big Data Engineering Lead in Atlanta, GA, specializing in data ingestion from diverse systems. The role involves collaboration in developing the NCR Data Lake and inspiring the engineering team towards technical excellence. The ideal...Big data
Arete Technologies Inc
Atlanta, GA2 days ago - ...Publicis Sapient in Atlanta is seeking a Senior Associate in Data Engineering to translate client requirements into technical design and implement components for data pipelines across ingestion, storage and analytics. You will work with Hadoop stack and cloud services...Big data
Unavailable
Atlanta, GA4 days ago - Req ID:374073NTT DATA strives to hire exceptional, innovative and passionate individuals... ...We are currently seeking a Senior AWS Data Engineer to join our team in Atlanta, Georgia (US-GA... ...- Associate/Professional or AWS Certified Big Data Specialty.Experience with real-time...Big dataContract workWork at officeRemote workFlexible hours
- ...There's a seat at our table for you...Position Summary:The Senior Data Engineer guides the development of GFS’ Data Platform consisting of the... ...certified internal data products. Identifies and translates Big Data challenges into technical approaches yielding actionable recommendations...Big dataFull timeWork at officeMonday to Friday
- ...WithAt Slalom, we co-create custom software, data, and cloud products with clients who are... ...true change.Our Data Architecture & Engineering team sees data as the foundation for decision... ...solutions that inspire people to dream big and move faster.What You’ll DoAs a Data Engineer...Big dataTemporary workLocal area
- ...Qualifications (Data Engineer/Python Developer) 10+ years hands-on Python development experience for big data application. Extensive working experience in implementing scalable and efficient data processing pipelines using big data technologies, such as Hadoop/EMR, Spark...Big dataWork experience placement
TMV Global Inc
Atlanta, GA5 hours ago $153.58k
...Responsible for the technical leadership for data ingestion and processing engagements.... ...environment, using a vast range of data engineering and cloud technologies. Work on tasks independently... ...position.Experience must include: Big data Tools: Spark, hdfs, Sqoop, hive,...Big dataFull timePart timeWork experience placementLocal areaRemote workFlexible hours$123k - $170k
...consulting, and customer experience with agile engineering and problem-solving creativity. United by... ...combine experience across technology, data sciences, consulting, and customer... ...deep understanding of data integration and big data design principles in creating custom...Big dataWork experience placementWork at office- ...Senior Data EngineerLocation: Atlanta, GAClient- Southern CompanyContract- 1 YearExperience... ...We are seeking an experienced Senior Data Engineer with a strong background in designing and... ...and OLAP systems.Hands-on experience with big data frameworks: Apache Spark, Hive,...Big data
4P Consulting Inc
Atlanta, GA3 days ago - Publicis Sapient is hiring a Senior Associate Data Engineering for a hybrid role based in Atlanta and Chicago. You will translate client requirements into technical designs and implement data engineering components across ingestion, storage, and analytics. Responsibilities...Big data
- ...needs. Take a shot at us for your IT requirements and experience a radical change. Job Description Responsibilities Lead Big Data engineering team with specialization on data ingestion (from 100+ source systems in batch and near real-time), egestion and governance....Big data
- ...needs. RESPONSIBILITIES: Work with data scientist team to migrate the analytical... ...successfully build datasets and implement complex big data solutions (under senior lead's... ...SKILLS: ~3+ years of professional data engineering or data wrangling experience ~3+ years...Big dataFull timeContract workTemporary workFor contractorsWork experience placementLocal areaRemote workFlexible hours
Visionaire Partners
Atlanta, GA4 days ago - ...Senior Data EngineerAtlanta, GAAbout The Weather Company:The Weather Company is the world... ...com).Job Brief:We're seeking a Senior Data Engineer to design, build, and operate large-scale,... ...science use cases.You'll work with modern big data technologies and AWS services,...Big dataFlexible hours
The Weather Company
Atlanta, GA3 days ago - ...Data Engineer Employment Type: Full-Time, Mid-level Department: Business Intelligence CGS is seeking a passionate and driven Data Engineer... ...to manipulate data (Python, R, SQL, SAS). Strong knowledge of big data analysis and storage tools and technologies. Strong understanding...Big dataFull timeFlexible hours
Cgsfederal
Atlanta, GA4 days ago - STEMtech is seeking a Data Engineer in Atlanta to optimize operations through data manipulation and aggregation. This role combines data from... ...of data engineering experience with a strong background in Big Data technologies and programming languages. The position demands...Big data
- Publicis Sapient in a hybrid Atlanta-Chicago setting seeks a Senior Associate in Data Engineering to translate client requirements into technical design and build end-to-end data pipelines using Hadoop, Spark, and cloud services. You will lead design discussions, drive...Big data
- A leading software development firm in Atlanta is looking for a Big Data Engineering Lead to manage the ingestion and governance of data. The role requires a strong background in distributed computing and experience with the Hadoop Ecosystem. Candidates must have a passion...Big data
- Publicis Sapient in Atlanta is hiring a Senior Associate Data Engineer to design and implement data pipelines, transform data, and enable analytics... ...four in-office days in Atlanta requiring strong Hadoop and big data expertise. You will drive end-to-end data integration,...Big dataWork at office
- Publicis Sapient is seeking an Associate Data Engineer to design, develop, and maintain scalable Big Data solutions for clients. You will work with large datasets, real-time processing frameworks, and cloud-based data platforms to enable data-driven decision-making. You...Big data
$91k - $119k
...Associate Data EngineeringAs an Associate Data Engineer, you will be responsible for designing, developing, and maintaining scalable Big Data solutions. You will work with large datasets, real-time processing frameworks, and cloud-based data platforms to enable data-driven...Big dataStarcom Mediavest Group Germany Gmbh
Atlanta, GA2 days ago- The Home Depot is seeking an Online Data Engineer to translate business requirements into robust data infrastructure. You will design data models... ...across enterprise systems. Join a team that leverages big-data techniques to cleanse, organize, and transform data, enabling...Big data
- Tech Mirrors in Atlanta, GA seeks an AWS Data Engineer with 10+ years of Python and big data experience. You will build scalable pipelines using Hadoop/EMR, Spark, Hive, and AWS services like Lambda, EventBridge, Step Functions, Redshift, S3, and Glue to turn raw data...Big data
- Publicis Sapient in Atlanta, GA seeks an Associate Data Engineer to design, develop, and maintain scalable Big Data solutions. You will work with large datasets, real-time processing frameworks, and cloud-based data platforms to enable data-driven decision-making for clients...Big data
- ...CapTech Consulting is seeking a Data Engineer to design, build, and optimize data pipelines across cloud platforms. You will work with clients... ...across teams. Join a firm that values modern datawarehousing, big data, and open-source tooling to drive decision-making for...Big data
CapTech Consulting
Atlanta, GA10 hours ago - ...centers worldwide, gaining exposure to complex engineering challenges that redefine industrial... ...are currently searching for a Senior AWS Data Engineer: The Challenge (... ...lakes, data warehouses, data marts, and big data environments. ~ Strong understanding...Big dataRemote workWorldwide
Sequoia Connect
Atlanta, GA7 days ago - STEMtech has a client in the Atlanta area looking for a Data Engineer who will aid in the optimization of operations by manipulating and aggregating... ...data sources Uses Python, "R", Informatica, and other Big Data tools and technologies to code the data Engineering routines...Big data
- ...optimize ETL/ELT pipelines for ingesting, transforming, and processing data. Ensure data pipelines are scalable, reliable, and efficient.... ...solutions using relational (SQL) and NoSQL databases. Work with big data technologies like Spark, Hadoop, and Kafka for large-scale...Big dataWork at office
- Overview Job Purpose ICE Data Services (an Intercontinental Exchange company) is seeking a Senior Data Engineer to join its Data Impact & Innovation team. This team supports... ...monitoring, evaluating, and prototyping emerging big‑data, geospatial, and machine‑learning...Big data
- ...Role We are looking for a technically skilled and motivated Data Engineer with 7-12 years of experience to join our growing Data &... ...like Apache Airflow, dbt, or Palantir Foundry Familiarity with big data technologies such as Spark, Kafka, or Redshift Exposure...Big dataLocal areaRelocation
Thought Logic Consulting
Atlanta, GA24 days ago - ...love to meet you! ABOUT THE ROLE We are looking for a Senior Data Engineer to design and build scalable data lakes, warehouses, and lakehouse... ..., Polars , PySpark or DuckDB ; 2+ years of experience with Big Data technologies ( Spark , Snowflake ); Expert-level knowledge...Big dataWork at officeRemote workVisa sponsorshipWork visaFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!
Related searches
- junior data engineer remote Atlanta, GA
- director data engineering Atlanta, GA
- hadoop big data developer Atlanta, GA
- data engineer intern Atlanta, GA
- senior data quality engineer Atlanta, GA
- senior cloud data engineer Atlanta, GA
- data engineer contract Atlanta, GA
- data infrastructure engineer Atlanta, GA
- junior data developer Atlanta, GA
- entry level big data engineer Atlanta, GA