Data Engineer - Pyspark

Full-time

Highbrow Technology Inc

Raleigh, NC

Job Title: Data Engineer – PySpark

Location: RTP Area, NC

Experience: 8+ Years

Work Arrangement: Hybrid – Onsite

Employment Type: W2 Only

Job Description

We are looking for an experienced Data Engineer with strong PySpark expertise to design, develop, and maintain scalable data processing solutions. The ideal candidate will have extensive experience in big data technologies, distributed computing, ETL/ELT development, and cloud-based data platforms.

The candidate should be comfortable working with large datasets, building efficient data pipelines, optimizing Spark workloads, and collaborating with cross-functional teams to deliver reliable data solutions.

Required Skills & Qualifications

8+ years of experience in Data Engineering or related roles.
Strong hands-on experience with Python and PySpark .
Extensive experience with Apache Spark, Spark SQL, and distributed data processing.
Strong proficiency in SQL, including complex queries, joins, aggregations, and query optimization.
Experience developing and maintaining large-scale ETL/ELT data pipelines.
Strong understanding of data warehousing concepts, data modeling, and data lake architectures.
Experience working with cloud data platforms such as AWS, Azure, or GCP.
Experience with technologies such as Databricks, Delta Lake, Hive, or similar big data frameworks.
Knowledge of workflow orchestration tools such as Apache Airflow, Azure Data Factory, or AWS Glue.
Experience with Git, CI/CD pipelines, and Agile development methodologies.
Strong analytical, troubleshooting, and problem-solving skills.

Preferred Qualifications

Experience with streaming data processing using Kafka or Spark Streaming.
Exposure to Snowflake, Redshift, BigQuery, or Azure Synapse.
Experience implementing data governance and data quality frameworks.
Familiarity with Docker, Kubernetes, and cloud-native data engineering solutions.

Work Requirements

Location: Raleigh, NC.
Hybrid work arrangement with onsite presence required.
Must be willing to work on W2.
8+ years of relevant professional experience required.

How to Apply

Interested candidates can share their updated resumes at:

View email address on click.appcast.io

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Data Engineer - Pyspark in Raleigh, NC vacancy

Senior Data Engineer
...making significant investments in its enterprise data and AI capabilities. This is an opportunity for a senior-level engineer to help shape the future of a modern Azure-... ...leveraging Azure Data Lake, Delta Lake, Spark, and PySpark Creating robust data ingestion, integration...
Suggested
Full time
Robert Half
Raleigh, NC
1 day ago
AWS Python Data Engineer
...specialize in leveraging advanced technologies such as AI, cloud, and data-led innovation to help our clients accelerate growth and unlock... ...& Scalability.Preferred Skill and Experience• Experience with Pyspark• Experience with Fast API/Flask frameworks, Large-scale data...
Suggested
Full time
Temporary work
Relocation
Infosys Technologies
Raleigh, NC
1 day ago
Lead Data Engineer - Databricks (Remote)
...global enterprises to design and scale data platforms that power products used by millions... ...as a Tech Lead As a Tech Lead, Data Engineering , you will own the technical direction... ...-grade data pipelines using Databricks, PySpark, AWS, and SQL Lead by example with hands...
Suggested
Remote job
Full time
Monday to Friday
Flexible hours
Lumenalta
Raleigh, NC
3 days ago
Senior Data Engineer
...Job Description Job ID: 812170 \n Job Title: Senior Data Engineer \n Client: State of NC - FAST \n Location: 1915 Health Services... ..., and indexing strategies \n Hands-on experience with PySpark, Python, or Scala \n Experience with data modeling, performance...
Suggested
InfoStride
Raleigh, NC
3 days ago
Principal Data Engineer
$112.7k - $193.2k
...people with the care, pharmacy benefits, data and resources they need to feel their... ...Provide technical leadership for analytics engineering, owning solution design, implementation,... ...or similar tools, using Python and/or PySpark and SQL to build scalable applications and...
Suggested
Minimum wage
Full time
Work experience placement
Work at office
Local area
Remote work
UnitedHealth Group
Raleigh, NC
2 days ago
Lead Data Engineer
...Programming & Data Engineering Expert - level proficiency in Python, Scala, and PySpark, with a strong track record of designing and delivering production-ready, modular, and well-tested solutions; developing and troubleshooting Spark workloads; and optimizing large-...
Full time
Contract work
Altimetrik
Cary, NC
4 days ago
Lead Data Engineer, AI & Modernization
...expertise across Cybersecurity, Digital Core, Finance, Industry and Enterprise, Song, Supply Chain and Engineering, and Talent, with advanced capabilities in AI and Data, Industry and Process, and Technology. We serve approximately 9,000 clients and generated...
Work experience placement
Live in
Work at office
Local area
Accenture
Raleigh, NC
12 hours ago
Data Engineer III
$105.4k - $207.8k
Position Summary Our Deloitte AI & Engineering team works to transform technology platforms, drive innovation, and help make a significant... ..., and fuel growth through innovation.Work you'll do As a Data Engineer III on the AI & Data team, you will be responsible for...
Local area
Deloitte
Raleigh, NC
2 days ago
Data Engineer - East Region
...all.Who You’ll Work WithAt Slalom, we co-create custom software, data, AI, and cloud solutions with clients who are ready to... ...products, experiences, and organizations.Our Data Architecture & Engineering team sees data as the foundation for decision-making, artificial...
Temporary work
Work at office
Local area
Slalom
Raleigh, NC
3 days ago
Data engineer
...6a1kMvyS65NP3cxK69ePILRMCaJ-RyjemgeI=Job Code: ATS #57179Location: Raleigh, NC, United StatesJob Title: Data EngineerLocation: Raleigh, NCJob Summary - Data Engineer:We are seeking a skilled mid-level+ Data Engineer to join our team and focus on quality assurance, quality...
InstantServe
Raleigh, NC
4 days ago
Data Engineer
$100.79k - $160.26k
...DLA Piper is a place you can engage in meaningful work and grow your career. Let’s see what we can achieve. Together.SummaryThe Data Engineer, Solutions & Data role designs, builds, and operates data pipelines and data integration processes that translate raw data into...
Full time
Work at office
Remote work
Visa sponsorship
DLA Piper
Raleigh, NC
1 day ago
Senior Data Engineer
...Dive into innovation in Digital Transformation, Cybersecurity, IT, Data Analytics and Software Development. Your journey to impactful... ...is seeking a motivated, career and customer-oriented Senior Data Engineer to join our team in Raleigh, NC. This is a Hybrid position. In...
Work at office
Local area
ManTech International
Raleigh, NC
3 days ago
Data Engineer - Data Modernization
...Data Engineer - Data Modernization Snowflake · dbt · Data Engineering · Data Modernization · AI-Assisted Development Raleigh NC | Hybrid/ Remote | Full Time Opportunity ABOUT US Incedo is a global AI, data, and digital transformation specialist that helps...
Full time
Immediate start
Remote work
Incedo Inc.
Raleigh, NC
1 day ago
Lead AI and Data Science Engineer II- Marketing Measurement
$122k - $240.5k
Position Summary Are you an experienced, passionate pioneer in data and technology who wants to work in a collaborative environment? As an experienced LeadAI and Data Science Engineer II - Marketing Measurement, you will have the opportunity to share new ideas and...
Local area
Deloitte
Raleigh, NC
2 days ago
Lead Data Engineer
...Job Description Position : Lead Data Engineer (Hands-On) \n Location : Cary, NC (On-site / Hybrid) \n Duration : Full Time \n... ...without bespoke pipeline code for every feed. \n Write canonical PySpark and Scala Spark transformation jobs that serve as the team...
Full time
Contract work
Smart IT Frame LLC
Cary, NC
1 day ago
AWS Redshift Data Engineer
...driving digital transformation for financial institutions. We specialize in leveraging advanced technologies such as AI, cloud, and data-led innovation to help our clients accelerate growth and unlock business value. Our AI-driven solutions empower financial institutions...
Full time
Temporary work
Relocation
Infosys Technologies
Raleigh, NC
4 days ago
Security Engineer (Data Engineering/ Protection)
...America)Please review the following job description:The Security Engineer will serve as a technical expert for Protegrity within the... ...responsible for designing, implementing, and supporting enterprise level data protection solutions. This role will support the integration of...
Full time
Part time
Shift work
Day shift
Truist
Raleigh, NC
2 days ago
AI and Data Science Engineer III - Marketing Measurement
$113.3k - $188.9k
Position Summary Are you an experienced, passionate pioneer in data and technology who wants to work in a collaborative environment? As an experienced AI and Data Science Engineer III - Marketing Measurement, you will have the opportunity to share new ideas and...
Local area
Deloitte
Raleigh, NC
3 days ago
Data Engineering Intern
...mountains to the coast. Find your next opportunity today! Description of Work Are you seeking to learn the importance of Data Engineering within the public sector? The North Carolina Office of the State Auditor is searching for an intern to serve on our Data...
Hourly pay
Permanent employment
Temporary work
Internship
Work at office
Night shift
Office of the State Auditor
Raleigh, NC
2 days ago
Data Engineer
...some of the most reputable private equity firms in the world such as Advent, LGP, and Genstar. Job Overview We are seeking a Data Engineer to join our Professional Services Operations team. The ideal candidate will be a precision-focused problem solver who can execute...
Full time
Temporary work
Casual work
Immediate start
Prometheus Group
Raleigh, NC
2 days ago
Analytics Engineer
...utilities modernize their distribution grids by harnessing the power of data across all their devices and systems deployed throughout the... ...The Opportunity Tantalus is seeking an early-career Analytics Engineer to join our Customer Operations team, reporting to the Senior...
Internship
Flexible hours
Night shift
Tantalus
Raleigh, NC
3 days ago
Lead Data Engineer (Hands-On)
...Engagement Build a centralized, AI-first enterprise data hub for a global insurance and financial... ...layers, build reference implementations for the engineering team, and ship production-grade Python, Scala, and PySpark code every week. Candidates must have written or...
Full time
Shift work
Cephas Consultancy Services Private Limited
Cary, NC
a month ago
Data Engineering, Management, & Governance Specialist - Finance
We Are:The beginning of a new Data & AI decade that will reshape work and society has begun. Accenture is stepping boldly into this... ...Investment, Exceptional Talent, and Power Ecosystem.You Are:As a Data Engineering, Management & Governance professional, you will play a pivotal...
Work experience placement
Live in
Work at office
Local area
Accenture
Raleigh, NC
3 days ago
Data Center Engineer
...Skills: 1-3+ years’ experiencedeploying and maintaining complex data center environments (new facility, majorfacility renovations/... ...andprecisely -Editing Skills - Visual needs and Tools of the trade -Engineering Base - network knowledge and technicaljargons #J-18808-Ljbffr...
Shift work
Premier Inn Hotels LLC (UAE)
Raleigh, NC
2 days ago
Data Engineer (MDM Platform)
...Data Engineer (MDM Platform) Snowflake · dbt · Python · Apache Airflow · Master Data Management (MDM) Raleigh, NC | Hybrid | Full-Time Opportunity ABOUT US Incedo is a global AI, data, and digital transformation specialist that helps leading enterprises drive...
Full time
Flexible hours
Incedo Inc.
Raleigh, NC
1 day ago
Senior Data Analyst Databricks / PySpark / Power BI
...Job Description Job Description Job title : Senior Data Analyst – Databricks / PySpark / Power BI Location: Raleigh, NC Contract : 6 Months... ...and complex SQL queries Collaborate closely with data engineers to design and deliver high-quality data products Develop...
Contract work
Crescens Inc.
Raleigh, NC
13 days ago
Senior Azure Data Analyst / Data Engineer - NC FAST | Hybrid - Raleigh, NC
$70 - $85 per hour
...Human Services (NC DHHS) \n Official Classification: NC FAST Data Scientist – Mid Level \n Location: Raleigh, NC 27607 \n Work... ...LLC is seeking an experienced Senior Azure Data Analyst / Data Engineer to support NC DHHS in designing, developing, and optimizing...
Hourly pay
Full time
Contract work
Relocation
ConnectDotSsolution
Raleigh, NC
3 days ago
Director, Data Engineering
$170k
...A Day in the Life: The Director, Data Engineer owns the strategy, reliability, and evolution of the enterprise business intelligence and analytics platform ecosystem. This role ensures BI platforms are scalable, secure, cost-efficient, and operationally excellent—enabling...
Worldwide
The Hertz Corporation
Raleigh, NC
2 days ago
Data Solutions Engineer
...Job Description Data Solutions Engineer \n Remote (US) \n Klickly is a fast-growing, award-winning startup that is revolutionizing eCommerce. Based in the heart of Silicon Beach in Santa Monica CA, Klickly’s proprietary data and AI transform how innovative brands...
Remote work
Klickly
Raleigh, NC
6 days ago
AI & DATA Senior Manager - Platform Engineering Lead
...while leading the development and deployment of AI models and data architectures to improve business processes and system performance... ...’s degree in Computer Science, Data Science, AI, Software Engineering, a related field, or equivalent education, training, and work-related...
Permanent employment
Full time
Part time
H1b
Work at office
Work visa
Shift work
Day shift
Truist
Raleigh, NC
3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Engineer - Pyspark. Be the first to apply!