Data Engineer - Pyspark
Full-time
Highbrow Technology Inc
Job Title: Data Engineer – PySpark
Location: RTP Area, NC
Experience: 8+ Years
Work Arrangement: Hybrid – Onsite
Employment Type: W2 Only
Job Description
We are looking for an experienced Data Engineer with strong PySpark expertise to design, develop, and maintain scalable data processing solutions. The ideal candidate will have extensive experience in big data technologies, distributed computing, ETL/ELT development, and cloud-based data platforms.
The candidate should be comfortable working with large datasets, building efficient data pipelines, optimizing Spark workloads, and collaborating with cross-functional teams to deliver reliable data solutions.
Required Skills & Qualifications
- 8+ years of experience in Data Engineering or related roles.
- Strong hands-on experience with Python and PySpark .
- Extensive experience with Apache Spark, Spark SQL, and distributed data processing.
- Strong proficiency in SQL, including complex queries, joins, aggregations, and query optimization.
- Experience developing and maintaining large-scale ETL/ELT data pipelines.
- Strong understanding of data warehousing concepts, data modeling, and data lake architectures.
- Experience working with cloud data platforms such as AWS, Azure, or GCP.
- Experience with technologies such as Databricks, Delta Lake, Hive, or similar big data frameworks.
- Knowledge of workflow orchestration tools such as Apache Airflow, Azure Data Factory, or AWS Glue.
- Experience with Git, CI/CD pipelines, and Agile development methodologies.
- Strong analytical, troubleshooting, and problem-solving skills.
Preferred Qualifications
- Experience with streaming data processing using Kafka or Spark Streaming.
- Exposure to Snowflake, Redshift, BigQuery, or Azure Synapse.
- Experience implementing data governance and data quality frameworks.
- Familiarity with Docker, Kubernetes, and cloud-native data engineering solutions.
Work Requirements
- Location: Raleigh, NC.
- Hybrid work arrangement with onsite presence required.
- Must be willing to work on W2.
- 8+ years of relevant professional experience required.
How to Apply
Interested candidates can share their updated resumes at:
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - Pyspark in Durham, NC vacancy
$95k - $120k
...StatesSchedule: Full timePosted: 2026-11-04The OpportunityThe Data Engineer III will design, develop, and maintain scalable data pipelines... ...Develop data transformation and analytics solutions using Python, PySpark, and SQL.Ensure data quality, consistency, and governance...SuggestedRemote workFlexible hoursAbt Associates
Durham, NC11 hours ago- Job Summary:The Data Solutions Engineer designs, builds, and operates reliable, scalable data pipelines on PowerSecure’s Azure data platform to... ...highquality data using Azure Data Factory, Databricks (PySpark/Spark SQL), and Delta Lake (including Delta Live Tables). Responsibilities...SuggestedFull timeWork at officeVisa sponsorshipWork visaFlexible hours
PowerSecure
Durham, NC11 hours ago - Oracle Migration. Introduce a new product. Majority will be hands-on coding (PL/SQL) but will have some DBA knowledge/experience. Knowing the existing code (package), use this as the base to create new lines of code. Informatica – extract the logic and incorporate...Suggested
Mastech
Durham, NC3 days ago - ...Data EngineerCandidate must sit in Smithfield, RI. As a 2nd option, Westlake, TX can also be a location.Must Have Skills:10+ years of Data Engineer/Analytics ExpStrong database design, relational and dimensional modeling, and SQL skillsPowerBI and TableauExperience working...Suggested
Software Technology Inc
Durham, NC11 hours ago - ...Data Engineer/Reports Developer(ETL/OBIEE)Hybrid Mode(5 Days in Month)Westlake, TX/Durham, NCJob DescriptionSeeking a Systems Analyst/QA to play a key role in enhancing a Reporting application with new data and reports. The reporting application uses OBIEE for reporting...Suggested
Samprasoft
Durham, NC11 hours ago - ...Hybrid Every other week onsite/5 days in Durham, NC Our client seeks a senior Data Engineer to design, build, and maintain operational and analytical capabilities across modern data platforms. The role focuses on Snowflake, AWS, and Python to enable scalable data lakes...
Eliassen Group
Durham, NC1 day ago - ...Enterprise AI (EAI) builds the data foundations that GSK's AI systems depend on. In our operating model, digital workers do the hands-on data engineering. They generate pipelines, test and deploy them, monitor them in production and remediate failures. Our engineers direct...Permanent employmentLocal areaRemote workFlexible hours
NCBioImpact
Durham, NC2 days ago - ...Durham North CarolinaDuration: Long TermRequired:• Degree in a technology related field (e.g. Engineering, Computer Science, etc.).• 8+ years of experience in implementing data engineering solutions in data analytics space• 3+ years of experience in developing data...
Samprasoft
Durham, NC11 hours ago - ...Job OpportunityJoin a skilled Fidelity's Asset Management Technology team to develop advanced data engineering solutions that power critical investment decisions. This role involves developing scalable data processing solutions, designing robust data models and ETL pipelines...Full time
Fidelity Corp
Durham, NC19 hours ago - ...Data EngineerHybrid Mode(5 Days in Month)Durham,NC/Westlake,TXJob DescriptionNeeds to be comfortable in extracting data from Oracle databases, PLSQL Python AWS Glue – EC2, S3 What are the NICE to have skills (what is your team willing to teach the person)? Knowledge of...
Samprasoft
Durham, NC11 hours ago - ...NOTES:Project Information: Will be on the Content Recommendation Engineer Squad and building out processes, recommending content to... ...tier and back end 30/70) and python programming & experience with data movement technologies (ETL/ELT)Preferred skills: API Development...Remote work
Samprasoft
Durham, NC11 hours ago - Hybrid two weeks onsite a month in either Merrimack, NH or Durham, NC We are seeking a Senior Data Engineer to build scalable data solutions within a highly collaborative technology team. The role focuses on Oracle PL/SQL, SQL scripting, ETL with Informatica, and data...
- ...Data EngineerHybrid Mode(5 Days in Month)Westlake,TX/Durham,NCJob Description5+ years in Enterprise application development (design, development, testing, deployment, and support)5+ years of development experience in Database DevelopmentWriting SQL queries and debugging...
Software Technology Inc
Durham, NC11 hours ago - ...Fidelity Investments Inc. is seeking a senior data engineer to join Asset Management Technology. You will develop scalable data processing solutions, design data models, and build robust ETL pipelines while applying modern DevOps and Agile practices. The ideal candidate...
Fidelity Investments
Durham, NC1 day ago - UX ResearcherSmithfield, RIOpen to Boston, MA and Jersey City, NJ as well. (Remote Until Covid)12 MonthsJob DescriptionLooking for hands on user researchers, not someone who just read a book and wants to get into this space. Looking for associates who know how to design...Remote work
Samprasoft
Durham, NC11 hours ago - ...Headwater Science is seeking a Sr Data Engineer to design, build, and maintain data pipelines and software infrastructure powering real‑world evidence research. You will work primarily in R, Python, and SQL, delivering scalable data resources and collaborative development...
Headwater Science
Durham, NC2 days ago - ...Data EngineerDynamic work schedule - This is 5 days on site a month in the same week then the remainder of the month is working from... ...organization that focuses on designing and implementing data engineering solutions such as Batch and Streaming Data Processing/Ingestion...Work experience placementWork from home
Samprasoft
Durham, NC4 days ago - ...Senior Data EngineerFidelity Investments Workplace Solutions (WS) organization is looking for senior Data Engineers. This person will be playing a key role in designing and crafting a modern data and information delivery and analytics platform in the cloud to support...Work experience placement
Software Technology Inc
Durham, NC11 hours ago - ...Hiring: Senior Data Engineer – Oracle PL/SQL Location: Durham, NC Work Mode: Onsite Candidate Requirement: Local candidates only Employment Type: W2 or Full-Time We are looking for a Senior Data Engineer with strong hands-on experience in Oracle PL/SQL development, database...Full timeLocal area
HCL Global Systems
Durham, NC2 days ago - ...Senior Technical Recruiter @ Seven Step | MBA, Global Talent Acquisition. Job Summary We are seeking an experienced Senior Big Data & Cloud Engineer to design, build, and deliver advanced API and data solutions that support financial goal planning, investment insights, and...Contract work
- ## Principal Data EngineerApply: Durham, NC: Full time: Posted Today: End Date: November 17, 2026 (30+ days left to apply): 2136514... ...implementation discussions.* Develops and implements advanced data engineering frameworks.* Advises senior leadership on data infrastructure...Full time
Soteria Reinsurance Ltd.
Durham, NC4 days ago $95k - $154k
...self-taught and certificate-heavy jobseekers who want AI, ML, and data scientist jobs. JOPP replaces isolated courses with live... ...Python, statistics, ML, deep learning, NLP, LLMs/GenAI, and data engineering context so you can answer "how would you productionize this?” You...Full timeSynergisticIT
Durham, NC5 days ago$104.8k - $157.2k
...farmers' success. To support this mission, Syngenta’s Analytics & Data Sciences Team is seeking an R&D Analytics - AI Developer in... ...at scale. The successful candidate will combine strong software engineering, applied AI, and scientific data understanding to deliver tools...H1bFlexible hoursSyngenta Seeds , Inc.
Durham, NC2 days ago- ...Senior Cloud Data EngineerLocation: Durham North Carolina, Remote till CovidDuration: 12+ monthsThe Expertise and Skills You BringBachelor... ...required8+ years of total experience with 4+ years on data engineering in cloud native tools and technologiesStrong SQL knowledge and...Work experience placementRemote work
Samprasoft
Durham, NC5 days ago - ...Senior Data Cloud EngineerProven track record of working in robust teams to deliver high quality data solutions in a multi-developer... ...equivalent experienceWhat we are looking for is a strong AWS Cloud Engineer with experience working in Public Cloud, not private cloud. We...
Samprasoft
Durham, NC11 hours ago - ...Fabric Data EngineerAre you ready to work for a more active world? At Bioventus, our business depends on developing our people. We invest... ...to help patients resume and enjoy active lives.The Fabric Data Engineer administers, governs, and continuously improves the Microsoft...
Bioventus
Durham, NC3 days ago - ...Vice President, Analytics Engineering About the Company Well-established full-service banking institution Industry Banking... ...Analytics Engineering to lead the design and delivery of enterprise data products on its Data Foundation Platform. This hands-on, player...
- ...application-level execution. Defines technical approaches, coordinates engineering activities, and promotes engineering excellence and continuous... ...equivalent) and six (6) years of experience as a Director, Data Engineering (or closely related occupation) designing and...Full time
Fidelity Investments
Durham, NC4 days ago - ...performance, availability and reliability Collaborating with team engineers to design and implement servers, storage, containers, network... ...across all areas of technologyThis person will work within the Data Cetner Operations team, its Partner engineers to provide key...Full timeWork experience placementLocal areaRemote work
WolfSpeed
Durham, NC4 days ago - ...Data Etl Reports DeveloperLocation: Durham NCPosition: HybridDuration: Long TermJob Description:This position requires the services of a report developer to provide business and systems analysis, modeling, design, and programming for FNS and SNAP reports development and...
Software Technology Inc
Durham, NC5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - Pyspark. Be the first to apply!
Related searches
- senior cloud data engineer Durham, NC
- big data cloud engineer Durham, NC
- data engineering intern summer Durham, NC
- data engineer analytics Durham, NC
- sr data engineer Durham, NC
- data developer Durham, NC
- sr information security engineer Durham, NC
- etl data engineer Durham, NC
- aws data engineer Durham, NC
- data science developer Durham, NC