Data Engineer
Techvilla Solutions
We are seeking a Senior Data Engineer to design, develop, and maintain scalable data pipelines and data ingestion processes using modern Big Data technologies. The ideal candidate will have strong experience in data aggregation, transformation, validation, quality management, ETL, and troubleshooting across large-scale data environments.
Roles and Responsibilities
-
Design, develop, and maintain complex data pipelines and data ingestion processes for data lake environments.
-
Develop data transformation processes to aggregate, standardize, link, validate, and load data from multiple sources.
-
Build and maintain data engineering solutions using Spark, Scala, Hadoop, Hive, SQL/T-SQL, and Shell scripting.
-
Perform data validation, quality checks, analysis, and troubleshooting to ensure data accuracy and integrity.
-
Analyze data issues, perform root-cause analysis, and implement solutions to improve data quality and pipeline performance.
-
Develop tools and processes to improve data processing efficiency, reliability, and performance.
-
Work with technical and operations teams to troubleshoot complex issues involving databases, Linux environments, storage, and servers.
-
Support production environments and resolve critical data and pipeline issues as needed.
-
Generate statistical and operational reports and perform program/data validation.
-
Mentor junior data engineers on data engineering, troubleshooting, and data quality best practices.
-
Collaborate with cross-functional teams in an Agile/SAFe environment to deliver projects within defined timelines.
Required Qualifications
-
8–10 years of overall industry experience with a Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field.
-
5+ years of experience in data aggregation, standardization, data linking, quality checks, and reporting.
-
5+ years of experience with Big Data technologies such as Hadoop and Spark.
-
5+ years of experience with relational databases such as Oracle and Microsoft SQL Server.
-
Strong experience with SQL, advanced SQL queries, ETL processes, and data integration tools.
-
Strong understanding of Linux environments, shell scripting, and file systems.
-
Hands-on experience with Spark and Scala.
-
Strong data analysis, debugging, troubleshooting, and root-cause analysis skills.
-
Experience working in an Agile development environment.
Preferred Qualifications
-
Experience working with healthcare data or in a healthcare data operations environment.
-
Experience with Hive and additional Big Data technologies.
-
Experience with data lake and large-scale data ingestion platforms.
-
Experience working in Scaled Agile Framework (SAFe) environments.
-
Availability to support production issues outside standard business hours when required.
-
EST time-zone availability is preferred.
- ...Senior Data Engineer As a Senior Data Engineer, you will work with the team to build complex data pipelines using a new cutting-edge Data Ingestion Platform as well as several new and exciting products in the Big Data/Advanced Analytics space. You will operate in a...Suggested
Samprasoft
Blue Bell, PA2 days ago - ...Data Solutions DesignerDesign of data solutions on Databricks including delta lake, data warehouse, data marts and other data solutions... ...and manage the pipelining (collection, storage, access), data engineering (data quality, ETL, Data Modelling) and understanding (...Suggested
Omni Inclusive
Lansdale, PA3 days ago - Job Title: Data Engineer We are seeking a Data Engineer with strong experience in Databricks. The ideal candidate will possess the following qualifications: Experience: Good experience in Databricks. Data Analytics: Strong Data Analytics experience, knowledge of data modeling...Suggested
$87.3k - $137.4k
Job DescriptionSpecialist — BPP to NGB Pilot Plant Master Data (SAP)Our Research Scientists are our Inventors. We identify and target... ...candidate will work closely with Process Staff, Formulators, Engineers, IT, Quality, and other team members to understand and execute...SuggestedFull timeFor contractorsWork experience placementLocal areaRelocationFlexible hours- ...performance-based cash incentive awards.Salary Range$139,400.00 - $230,000.00Target Openings1What Is the Opportunity?Travelers Data Engineering team constructs pipelines that contextualize and provide easy access to data by the entire enterprise. As a Data Engineering...SuggestedFull timeWork experience placementLocal area
- ...least 4 years of experience in software design, development and release.Well versed with agile development methodologies.Experience in Data Ingestion, Streaming and Big Data technologies.Excellent communication and stakeholder managementAbility to work with distributed...Permanent employmentFull timeH1b
$84.38k - $135k
Alternate Locations: Greensboro, NC (North Carolina); Charlotte, NC (North Carolina); Fort Wayne, IN (Indiana); Radnor, PA (Pennsylvania) Work Arrangement:Hybrid : Employee will work 3 days a week in a Lincoln officeRelocation assistance: is not available for this opportunity...Work experience placement3 days per week- Data Engineer - AIDay to Day job Duties: (what this person will do on a daily/weekly basis)Design and build scalable ingestion, transformation, and processing pipelines for AI/ML.Develop and maintain data storage, such as vector databases, that support Retrieval-Augmented...
- ...TitleThis role will be responsible for transforming extensive and complex data into consumable business capabilities. Create system architecture, design, and specification using in-depth engineering skills and knowledge to solve complex development problems and achieve...Live in
Software Technology Inc
Horsham, PA3 days ago - ...Data Warehousing And Infrastructure ArchitectThis position provides strategic direction and thought leadership for enterprise-wide... ...Information Systems or related field.7 years experience in data engineering and infrastructure with a demonstrated ability to influence organizational...
Jefferson
Fort Washington, PA9 hours ago - ...Data Engineer - Contract to Hire (Hybrid, No Sponsorship) Location: Wayne, PA | Plano, TX | Charlotte, NC | Scottsdale, AZ Job Type: Contract to Hire (Hybrid, 3 days on-site/week) Eligibility: US Citizens/Green Card Holders only (no sponsorship provided) About...Permanent employmentContract workLocal area3 days per week
Indotronix International Corporation
Wayne, PA2 days ago - ...Data Engineer Apex Systems is looking for a Data Engineer to join a Test Data Management Team at our client in Horsham, PA. We are looking for someone with strong SQL experience as the core skill. They need to have some Python and ideally some experience with Java as...
Software Technology Inc
Horsham, PA20 hours ago - ...Data EngineerLocation: West Conshohocken PADuration: 6 + monthsJob Description: Candidate should have previous experience building a data lake using Oracle database technologies. Candidate should be expert in improving extraction time and DB performance tuning. Candidate...
ClifyX
Conshohocken, PA3 days ago $120.38k - $192.6k
...officeRelocation assistance: is not available for this opportunity.Requisition #: 76603The Role at a GlanceAs a Senior Lead Risk Data Engineer and Architect, you will architect and lead the design and implementation of new technology solutions for the Risk Investments...Work experience placementWork at office3 days per week- ...We are seeking a passionate and skilled Senior Data Engineer to join our dynamic team in King of Prussia, PA. In this role, you will lead the design and implementation of advanced data pipelines for Business Intelligence (BI) and reporting. Your expertise will transform...
Eigen X
King of Prussia, PA2 hours ago - Data Engineer (TS/SCI with Full Scope Polygraph) {S} Job Category : ENG Requisition Number : DATAE003244 Apply now Posted : February 23, 2026 Full-Time Locations Showing 1 location King Of Prussia, PA 19406, USA Description ARKA Group L.P. (“ARKA”) is an advanced...Full timeTemporary workWork experience placementWork at officeVisa sponsorshipRelocation packageFlexible hours
$124.4k - $207.4k
ROLE SUMMARYBuild and operate the production data pipelines that power the Innovation, Data & Analytics team's AI-based workflows, implementing... ...conformancePartner with the Senior Data Scientist and AI/ML Engineer to ensure pipelines meet the data needs of analytical models and...Permanent employmentFull timeH1bLocal areaVisa sponsorshipWork visaRelocation packageFlexible hours2 days per week3 days per week- ...software company in King of Prussia, PA is seeking a candidate with over 4 years of experience in software design, specializing in Data Ingestion, Streaming, and Big Data technologies. The role requires a Bachelor’s degree or equivalent experience in IT and strong skills...Permanent employmentFull time
- ...software design, development, and release. This full-time, permanent position requires strong analytical skills and familiarity with Big Data technologies and agile methodologies. Ideal candidates will possess a Bachelor’s degree or equivalent experience and must have...Permanent employmentFull timeH1b
- Coforge is seeking a Senior Data Engineer (Agentic Retrieval & Memory) in Oaks, PA for a full-time role. You will design agent memory interfaces across session state, short-term history, and long-term memory, and select optimal stores for each tier. You will integrate with...Full timeTemporary work
- Role: Senior Data Engineer (Agentic Retrieval & Memory) Mode of Hire: Full time RESPONSIBILITIES Design and build the agent memory interface across the tiers an agent actually needs: working or session state within a run, short-term conversation history, and long-term...Full timeTemporary work
- Position Title:Senior Data Scientist, Software EngineeringPosition Duties/Requirements: Research, design, and implement advanced machine... ...data pipelines for large-scale data processing, feature engineering, and model development, ensuring robustness and scalability across...Work from home
- A leading technology company in King of Prussia is looking for a skilled Data Engineer/Scientist to support AI/ML algorithm development. You will work on complex data sets, improve data processes, and collaborate with a team of experts. Ideal candidates have a bachelor'...Relocation packageFlexible hours
- ...provide budgeting/financial flexibility by offering contingent labor as a variable cost.Job DescriptionLooking for a Sr. Network / Data Engineers with the Largest Pharma company in USA.Immediate Interview and Hire, Multiple positions open.Position will design and implement...Contract workWork at officeImmediate startFlexible hours
$162.9k - $271.5k
...orchestration layer and agent architecture behind the Innovation, Data & Analytics team's AI-based workflows, building and hardening... ...agentsProduction Reliability & CollaborationPartner with the Data Engineer to ensure agents consume clean, well-structured dataPartner with...Permanent employmentFull timeH1bLocal areaVisa sponsorshipWork visaRelocation packageFlexible hours2 days per week3 days per week$117k - $201.25k
...dignity of our employees and recognize their merit.Job Function: Data Analytics & Computational SciencesJob Sub Function: Data... ...partner closely with scientists, safety experts, data scientists, AI engineers, and platform teams to create knowledge assets that support...Full timeLocal areaImmediate start$50k - $85k
...partners to scale safely across lending, card services, and embedded finance. The Role We are seeking a solution-oriented Data Engineer with deep expertise in Azure Synapse, SQL Server, and Power BI . In this role, you will design and deliver end-to-end data...Bing Recruitment
Conshohocken, PA20 days ago$102k - $177.1k
...dignity of our employees and recognize their merit.Job Function: Data Analytics & Computational SciencesJob Sub Function: Data... ...case by case basis.Role OverviewWe are seeking a Principal Data Engineer to provide technical leadership within Global Medical Safety (GMS...Full timeContract workLocal areaImmediate startRemote work- Join to apply for the Data Operations & Database Engineer role at The Hartford . We’re determined to make a difference and are proud to be an insurance company that goes well beyond coverages and policies. Working here means having every opportunity to achieve your goals...Temporary work
- A leading insurance company is seeking a Data Operations & Database Engineer to design and monitor ETL/ELT pipelines, manage databases, and partner with other data professionals. Ideal candidates will have over 7 years of experience in data engineering, with strong SQL...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
