Big Data Engineer

Q1 Technologies

Job Description
Must Have Technical/Functional Skills

Primary Skill: Hadoop ecosystem (HDFS, Hive, Spark),PySpark,Python,Apache Kafka
Secondary: UI - Angular.
Experience: Minimum 9 years

Roles & Responsibilities

Architectural Leadership:
• Define end-to-end architecture for data platforms, streaming systems, and web applications.
• Ensure alignment with enterprise standards, security, and compliance requirements.
• Evaluate emerging technologies and recommend adoption strategies.

Data Engineering :
• Design and implement data ingestion, transformation, and processing pipelines using Hadoop, PySpark, and related tools.
• Optimize ETL workflows for large-scale datasets and real-time streaming.
• Integrate Apache Kafka for event-driven architectures and messaging.

Application Development :
• Build and maintain backend services using Python and microservices architecture.
• Develop responsive, dynamic front-end applications using Angular.
• Implement RESTful APIs and ensure seamless integration between components.

Collaboration & Leadership:
• Work closely with product owners, business analysts, and DevOps teams.
• Mentor junior developers and data engineers.
• Participate in agile ceremonies, code reviews, and design discussions.

Required Skills & Qualifications:

Technical Expertise:
• Strong experience with Hadoop ecosystem (HDFS, Hive, Spark).
• Proficiency in PySpark for distributed data processing.
• Advanced programming skills in Python.
• Hands-on experience with Apache Kafka for real-time streaming.
• Frontend development using Angular (TypeScript, HTML, CSS).

Architectural Skills:
• Expertise in designing scalable, secure, and high-performance systems.
• Familiarity with microservices, API design, and cloud-native architectures.

Additional Skills:
• Knowledge of CI/CD pipelines, containerization (Docker/Kubernetes).
• Exposure to cloud platforms (AWS, Azure, GCP).

Education:
• Bachelor's or Master's degree in Computer Science, Engineering, or related field.

Experience:
• 9+ years in software development, with at least 4 + years in architecture and Big Data technologies.

Preferred Qualifications:
• BFSI domain experience or large-scale enterprise systems.
• Understanding of data governance, security, and compliance standards.

Soft Skills:
• Strong analytical and problem-solving abilities.
• Excellent communication and leadership skills.
• Ability to thrive in a fast-paced, agile environment.

Apply

Vacancy posted 5 days ago

Similar jobs that could be interesting for youBased on the Big Data Engineer in Charlotte, NC vacancy

Big Data Engineer
...Job Title Required Skills: Applicant should be familiar with the following technologies/skillsets: Big Data knowhow is mandatory Big Data pipelines Optimization of big data Scalability of big data across different data stores (Aurora, RedShift, S3)...
Big data
Syntricate Technologies
Charlotte, NC
5 days ago
Data Architecture & Engineering - Senior - Financial Services - Consulting - NYC
$106.9k - $176.5k
...working world. Senior Consultant - Financial Services - Data & Analytics - Data Architecture and Engineering EY is the only professional services firm with a... ..., transformation, storage, mining/warehousing, big data analytics, and visualization. Understanding of...
Big data
Full time
Work experience placement
Summer holiday
Flexible hours
EY
Charlotte, NC
1 day ago
Senior Specialist - Data Engineering
We are seeking a skilled Data Engineer with strong handson expertise in Hadoop ecosystem technologies particularly in onpremise environments... ...using SqoopStrong knowledge of distributed data processing and big data conceptsAbility to design develop and maintain scalable data...
Big data
LTM
Charlotte, NC
22 hours ago
Big Data Hadoop Engineer
Job Title: Hadoop / HPE MapR Engineer (Data Platform) We are seeking a highly skilled Hadoop / HPE MapR Engineer to design, build, operate,... ...improvements. Required Qualifications 7+ years of experience in Big Data / Data Platform Engineering. Strong hands‑on experience...
Big data
Innoventrics
Charlotte, NC
4 days ago
Big Data Engineer
$100k - $110k
...Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy financial services and... ...key global markets. We are looking for an experienced Big Data Engineer with strong Software Engineering skills to design...
Big data
Temporary work
Flexible hours
Synechron
Charlotte, NC
1 day ago
Data Engineer - Manager
$99k - $232k
...Sector Not Applicable Specialism Data, Analytics & AI Management Level Manager... ...At PwC, our people in data and analytics engineering focus on leveraging advanced... ...Python, Java, or Scala - Familiarity with big data technologies like Hadoop, Spark, or...
Big data
Full time
H1b
PwC
Charlotte, NC
1 day ago
Data Engineer
...WithAt Slalom, we co-create custom software, data, and cloud products with clients who are... ...true change.Our Data Architecture & Engineering team sees data as the foundation for decision... ...solutions that inspire people to dream big and move faster.What You’ll DoAs a Data Engineer...
Big data
Temporary work
Local area
Slalom
Charlotte, NC
5 days ago
Ab Initio Data Engineer
...specialize in leveraging advanced technologies such as AI, cloud, and data-led innovation to help our clients accelerate growth and unlock... ...veteran; or disability.Technical/Domain Skill 1Technology|Big Data - Data Processing|PySpark Technical/Domain Skill 2Technology...
Big data
Full time
Temporary work
Relocation
Infosys Technologies
Charlotte, NC
1 day ago
Hadoop and PySpark Data Engineer
$82.42k - $126.6k
...specialize in leveraging advanced technologies such as AI, cloud, and data-led innovation to help our clients accelerate growth and unlock... ...with Hadoop ecosystem (Cloudera). Experience in working with a Big Data implementation in production environment. Experience with...
Big data
Full time
Temporary work
Relocation
Infosys Technologies
Charlotte, NC
1 day ago
Hadoop PySpark Data Engineer
$82.42k - $126.6k
OverviewThe Infosys Data and Analytics (DNA) unit is at the forefront of transforming data into actionable insights, driving business growth... ...with Hadoop ecosystem (Cloudera). •Experience in working with a Big Data implementation in production environment. •Experience with...
Big data
Full time
Temporary work
Relocation
Infosys Technologies
Charlotte, NC
5 days ago
Data Engineer - GCP
...Responsibilities Design and build scalable ETL/data pipelines using Spark and Python... ...Proficiency in Python for data engineering (ETL pipelines) Experience designing... ...data movement Experience working with big data technologies and handling large datasets...
Big data
Euclid Innovations
Charlotte, NC
2 days ago
Data Engineer
...and develop companywide best practices for engineering and technology solutions, ensuring... ...goals. Contribute to the development of Data Mesh capabilities to improve data accessibility... ..., and other GCP tools. Expertise in Big Data platforms (e.g., MapR, Cloudera)...
Big data
ASB Resourcing
Charlotte, NC
5 days ago
AWS Data Engineer
...Data Engineer The person on this role will play a crucial role in building scalable and cost-effective data pipelines, data lakes, and analytics... .... Must Have: Strong experience with AWS services on big data platforms. Strong Python experience in building Dash...
Big data
Samprasoft
Charlotte, NC
4 days ago
Data Engineer
...Data Engineer Hybrid 3 days onsite Charlotte (locals only) 10+ months Responsibilities: Build large-scale batch and real-time data... ...and problem-solving skills particularly those that apply to a big data environment. Strong expertise in applying narrow and wide...
Big data
Work experience placement
Local area
RIT Solutions
Charlotte, NC
3 days ago
Data Engineer
...Description We are currently seeking a talented and motivated Senior Data Engineer to join our Cloud data solutions team. This role will be... ...Enable streaming data use-cases and standard design patterns in a big data environment. Review and provide feedback on core code...
Big data
Work at office
Local area
Work from home
SMBC Group
Charlotte, NC
5 days ago
Data Engineer II
$93.69k - $114.51k
...Data Engineer II – Customer Data Providing for loved ones, planning rewarding retirements, saving enough for whatever lies ahead – our policyholders count on us to be there when it matters most. It’s a big ask, but one that we have the power to deliver when we work together...
Big data
Pacific Life
Charlotte, NC
3 days ago
Data Engineer
...Data Engineer Employment Type: Full-Time, Mid-level Department: Business Intelligence CGS is seeking a passionate and driven Data... ...manipulate data (Python, R, SQL, SAS). ~ Strong knowledge of big data analysis and storage tools and technologies. ~ Strong understanding...
Big data
Full time
Flexible hours
Contact Government Services LLC
Charlotte, NC
2 days ago
Data Engineer
...Data Engineer Location: Charlotte, North Carolina (Onsite) Role Overview We are seeking a Data Engineer to provide data-driven solutions... ...languages such as Java and Python. Experience with big data technologies like Hadoop or Spark. Proficiency in SQL and...
Big data
Apex Systems
Charlotte, NC
16 hours ago
Data Architecture & Engineering - Senior - Financial Services - Consulting - NYC
$106.9k - $176.5k
...Data Architecture & Engineering - Senior - Financial Services - Consulting - NYC Location: New York Other locations: Anywhere in Region Salary:... ...integration, transformation, storage, mining/warehousing, big data analytics, and visualization. Understanding of traditional...
Big data
Work experience placement
Summer holiday
Flexible hours
EY
Charlotte, NC
5 days ago
Data Engineer
Strong AWS engineering experience with cloud architecture, configuration, and application development... ...Security engineers to ensure optimal data delivery architecture throughout ongoing... .... Knowledge of building and optimizing ‘big data’ data pipelines, architectures and...
Big data
Brooksource
Charlotte, NC
5 days ago
Data Engineer
...months, offering an opportunity to work on critical enterprise data engineering projects aimed at enhancing Anti-Money Laundering (AML)... ...expertise in enterprise data management, ETL development, and big data technologies. You will be responsible for designing, developing...
Big data
Contract work
Worldwide
Socket.dev
Charlotte, NC
5 days ago
Data Engineer
$55.06 - $65.06 per hour
...SQL Skills - one or more of MS SQL, MySQL, HIVE, Impala, SPARK SQL Data ingestion experience from message queue, file share, REST API,... ...for having experience working with one or more tech like: Hadoop/Big Data and Distributed Systems, SPARK Structured steaming, elastic...
Big data
Hourly pay
Contract work
Temporary work
Work experience placement
Randstad USA
Charlotte, NC
5 days ago
AI Data Engineer
...driven solutions through our 5 core specialist Practices; Software Engineering, Data & Analytics, IT Operations, Change & Transformation, and Risk,... ...modern frameworks (e.g., React, Node.js) Familiarity with big data tools (e.g., Spark, Databricks) Understanding of APIs...
Big data
Work at office
Relocation
Visa sponsorship
3 days per week
FDM Group
Charlotte, NC
4 days ago
AWS Data Engineer
...TX/ Charlotte, NC Job Description We are seeking a skilled AWS Data Engineer to design, develop, and maintain scalable data pipelines and... ...engineering, ETL development, cloud-native architectures, and big data technologies, with the ability to support enterprise analytics...
Big data
Full time
Capgemini
Charlotte, NC
3 days ago
VP, Data Engineer
...Description Build large-scale batch and real-time data pipelines with data processing frameworks... ...solutions. Partner with architects, engineers, information analysts, business, and... ...analytical and problem‑solving skills in big data contexts. Experience with Python...
Big data
Work at office
Local area
Work from home
SMBC
Charlotte, NC
1 day ago
Senior Data Engineer (AWS, Azure, GCP)
..., a list that spans across the country.Job DescriptionCapTech Data Engineering consultants enable clients to build and maintain advanced data... ...tools (git, Jenkins, CI/CD, Jira) Fundamental understanding of big data, open source, and data streaming concepts Ability to...
Big data
Work at office
Remote work
Visa sponsorship
Work visa
Flexible hours
CapTech
Charlotte, NC
4 days ago
Data Integration Engineer
$62k - $93k
Data Integration Engineer What you will be doing: Understanding of databases, SQL, XML, Databricks, and data integration tools Familiarity... ...) Preferred QualificationsSkills: MSSQL XML Google Big Query Power BI / Tableau SSIS Python Basic...
Big data
Full time
Part time
For contractors
Work experience placement
Local area
Remote work
Premier, Inc.
Charlotte, NC
2 days ago
Lead Data engineer
$105k - $115k
We are seeking an experienced Lead Data Engineer to design, develop, and optimize scalable data platforms and pipelines that support enterprise... ...technologies, cloud platforms, data warehousing, and big data ecosystems, along with proven leadership experience in driving...
Big data
Temporary work
Flexible hours
Synechron
Charlotte, NC
5 days ago
Google Cloud Data Engineer
...Google Cloud Data Engineer Are you ready to step up to the New and take your technology expertise to the next level? Join and help transform... ...data pipelines from ingestion to consumption within a hybrid big data architecture, using Java, Python, Scala etc. Minimum 1...
Big data
Work experience placement
ClifyX
Charlotte, NC
3 days ago
GCP Data Engineer
Design, develop, and optimize scalable data pipelines for large-scale data processing and... ...transformation. Develop cloud-based data engineering solutions using GCP services , including... ...emerging technologies to build modern Big Data solutions. Required Skills 8+ years...
Big data
Fixity Technologies
Charlotte, NC
4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!