Senior Data Engineer
Euclid
Data Engineer
We are seeking an experienced Data Engineer to design, build, and enhance data ingestion pipelines and metadata frameworks that support data lineage, modernization, and analytics enablement. The ideal candidate has hands-on experience with AWS Glue, Python/PySpark, ETL pipelines, and metadata management tools, with the ability to work in a dynamic, cloud-centric data environment.
Key Responsibilities
- Design, develop, and maintain data ingestion and ETL pipelines for metadata integration across systems like SQL Server, Informatica, AWS Glue, and Azure Data Factory.
- Implement end-to-end data lineage tracing, including source-to-target mapping, column-level lineage, and job dependency tracking.
- Automate metadata ingestion and transformation for lineage reporting and quality validation.
- Identify static, orphaned, or unused tables/jobs and contribute to data-quality improvement.
- Collaborate with data architects, analysts, and AI strategist to align metadata ingestion with modernization goals.
- Participate in the deployment and hardening of solutions in AWS (and Azure as applicable).
- Contribute to modernization and AI-readiness initiatives through metadata standardization and automation.
Required Skills & Experience
8–10 years of experience in Data Engineering / ETL Development.
Strong experience with:
AWS Glue, S3, Redshift, Lambda
SQL (SQL Server, Oracle, or Snowflake) – complex queries, stored procedures, performance tuning
Python / PySpark for metadata parsing and ETL automation
ETL Tools: Informatica, AWS Glue, Azure Data Factory
Data Lineage / Metadata Management: Collibra, Alation, or custom lineage repositories
CI/CD & Orchestration: GitHub, Jenkins, or Azure DevOps
Nice to Have
Experience with BladeBridge or similar lineage analysis tools
Familiarity with Client Batch Processing
Exposure to Airflow, dbt, or Databricks for orchestration and transformation
Basic understanding of AI/ML data pipeline readiness
Azure data platform exposure (ADF, Synapse, Data Lake)
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Engineer in Fort Mill, York County, SC vacancy
- ...Job Title: Senior AWS Data Engineer Experience: 8+ Years Skills: AWS Glue, Lambda, Pyspark, Python Location: Fort Mill, SC Job Description : Develop migration workflows based on the approved metadata extraction design. Implement support for full...SeniorFull time
Coforge
Fort Mill, York County, SC3 days ago - ...Senior Data Engineer Fort Mill, SC Responsibilities: Design, develop, and maintain ETL processes to extract, transform, and load data from various sources into Snowflake. Utilize AWS services such as Airflow and Glue for orchestration and data processing tasks...SeniorFull time
Coforge
Fort Mill, York County, SC4 days ago - ...Position Summary We are seeking an experienced Data Engineer to design, build, and enhance data ingestion pipelines and metadata frameworks that support data lineage, modernization, and analytics enablement. The ideal candidate has hands-on experience with AWS...Senior
Euclid Innovations
Fort Mill, York County, SC4 days ago $121.98k - $130.53k
...0 regional best places to work lists. First American will always strive to be a great place to work, for all. What We DoThe Senior Data Engineer is a data expert and plays a key role in the development and deployment of innovative big data platforms for advanced analytics...SeniorWork experience placementFirst American
Fort Mill, York County, SC4 days ago- ...Job Description Job Title: Senior AWS Data Engineer \n Experience: 8+ Years \n Skills: AWS Glue, Lambda, Pyspark, Python \n Location: Fort Mill, SC \n \n Job Description : \n \n \n Develop migration workflows based on the approved metadata extraction...Senior
Coforge
Fort Mill, York County, SC20 days ago - LPL Financial seeks a VP, Principal Data Engineer to modernize investor data platforms across Reporting, Statements and Letters for 2.4M+ investors. You will architect high-performance pipelines, real-time processing, and cloud-native services with strict standards and...Senior
$159.96k - $266.56k
...success while helping clients pursue their financial goals.Job Overview:LPL Financial is seeking a highly accomplished VP, Principal Data Engineer to drive the modernization of the investor experience across the Reporting, Statements and Letters platforms, serving more than...Work from homeLPL Financial LLC
Fort Mill, York County, SC2 days ago- ...Lead Data Engineer Location : Fort Mill SC Skills : Snowflake, AWS glue, Hive, Python and Pyspark. Experience: +10 years We at Coforge are hiring Lead Data Engineer with the following skill sets. Design, develop, and implement data pipelines using AWS...Full time
Coforge
Fort Mill, York County, SC2 days ago - ...AWS Data Engineer Location: Austin Tx, Fort Mill or NY – hybrid Contract Visa: USC & GC Job Description Architect and implement data pipelines using AWS Glue for ETL processes. Design event-driven workflows using AWS Lambda and Step Functions. Ensure...Contract work
AceStack LLC
Fort Mill, York County, SC3 days ago - ...We are looking for an experienced Data Automation Engineer with strong expertise in Python, PyTest, database testing, SQL, and ETL validation to support data quality and automation initiatives. Key Responsibilities: Develop and maintain Python-based automation frameworks...Full time
Axiom Global Technologies
Fort Mill, York County, SC13 hours ago - ...Hiring: Python Data Automation Engineer – Fort Mill, SC We are seeking a skilled Data Automation Engineer with expertise in Python automation PyTest automation framework development database testing and ETL validation The ideal candidate will have a strong background...Long term contract
New York Technology Partners
Fort Mill, York County, SC5 hours ago - Data Engineer - Fort Mill, SC - Contract - W2 - $44-54 Per HR Description Engage in platform administration and data engineering within a cloud environment. Collaborate with teams to enhance the data ecosystem and ensure operational excellence. Design scalable data solutions...Contract work
- Must Have Skills ETL process validation Database Testing and SQL Complex Queries Handson experience with cloud platforms such as AWS Azure or GCP Strong Python programming skills with handson experience in PyTest Experience designing developing and maintaining...
GAC Solutions
Fort Mill, York County, SC5 hours ago $60 - $87 per hour
Description Principal AWS Data Engineer: This is a top-level technical leader who designs, builds, and manages large-scale, cloud-native data platforms and pipelines using Amazon Web Services. Project: Retirement Partners - This is hands-on, but they will oversee and lead...Contract workTemporary work$159.75k - $225.5k
...moments that reflect who they uniquely are.We are seeking a Senior Principal Software Engineer to lead the design and development of next-generation... ...efficiency, and business KPIsPartner with Product, UX, and Data teams to translate AI capabilities into: Increased...SeniorRemote workShutterfly
Fort Mill, York County, SC6 days ago- ...Implement and maintain security controls including SSO, MFA, dynamic data masking, row-level security, and secure data sharing.... ...and provide proactive recommendations for improvement. Data Engineering Design, develop, and maintain scalable ETL/ELT pipelines using...Ongoing contractFull timeLocal area
NLB Services
Fort Mill, York County, SC2 days ago - The Aegis Companies is seeking a Senior Project Controls Engineer to provide scheduling support for our data center practice. You will join a team of project managers and engineers to develop, manage, and monitor CPM schedules throughout the project life cycle on site....Senior
- LPL Financial is seeking a strategic VP, Software Development Manager to lead multiple engineering teams focused on Digital Experience, Cash & Liquidity, and Money Movement platforms. You will drive cloud modernization, AI-enabled engineering practices, and governance to...Senior
- ...Senior Product Owner with Scrum Master experience - Wealth Management Location: Fort Mill, SC Mode of Hire: Full Time Job Summary: We are seeking a highly experienced Senior Product owner with a strong background in the Wealth Management domain ....SeniorFull time
Coforge
Fort Mill, York County, SC3 days ago - ...strong emphasis on utilizing innovation and technology to cultivate client relationships. What You Will Be Doing: Hands on Senior SQL Database Administrator responsible for the secure design, administration, and operational integrity of enterprise Microsoft SQL...SeniorFull time
Associate Staffing
Fort Mill, York County, SC2 days ago - ...level of autonomy with visibility across business functions and senior leadership Opportunity to identify efficiencies, introduce new... ...Evaluate product performance, market trends, and operational data to identify opportunities for improvement and growth Oversee...SeniorFull time
AccruePartners
Fort Mill, York County, SC3 days ago - ...Job Title: Senior Automation QE Tead Skills: HVR and AWS services, specifically Airflow and Glue, automation, data integration, and ETL processes, Snowflake and SQL for data management... ...in automation technologies and data engineering. The ideal candidate will have...SeniorFull time
Coforge
Fort Mill, York County, SC3 days ago - Pike Corporation in Fort Mill, SC offers an Engineering Intern role supporting a wide range of engineering initiatives and projects. This position emphasizes hands-on learning, data management, and documentation under supervision. You will assist in creating project schedules...Internship
- A public retail company in Fort Mill, SC seeks a General Counsel to oversee all legal matters and ensure compliance with regulations. The ideal candidate will have a J.D., over a decade of legal experience, particularly with public companies, and strong M&A expertise. Responsibilities...Senior
- ...agency is seeking a compassionate Caregiver in Fort Mill, South Carolina. This full-time position involves providing in-home care for seniors with diverse needs like dementia and Alzheimer's. You will assist with daily activities as well as household tasks, ensuring a safe...SeniorFull time
- ...Collaborate with other technology teams and business users to understand complex user journeys, workflows, and data interactions.Work with the company’s Engineering - 3rd Line Support Manager and collaborate with internal and external business collaborators.Manage,...Senior3 days per week
Axelon
Fort Mill, York County, SC4 days ago - ..., a leading foodservice distributor, is seeking an experienced Senior Diesel Mechanic to maintain and repair our fleet of delivery trucks... ...in troubleshooting, repairing, and maintaining diesel engines, along with possessing a strong understanding of hydraulic and...Senior
- ...Full-Time Senior Veterinary Assistant UrgentVet bridges the gap between daytime general practice and emergency/overnight care, offering after-hours care for canine and feline patients without the additional expense of emergency clinic fees. Our hospitals offer care...SeniorFull timeNight shift
Clinical Support Staff
Fort Mill, York County, SC3 days ago - Sunbelt Rentals, Inc. is seeking a senior leader for a high-impact recruiting organization. You will lead and develop a team of Sr. Recruiters... ...talent plans. You will drive standardized processes, leverage data for hiring decisions, and strengthen employer branding while...Senior
- LPL Financial in the United States seeks an AVP, Scrum Master II to coach agile delivery teams, ensure transparency, and drive predictable delivery within SAFe. You will partner with Product Owner, Tech Lead, and RTE to manage PI planning, dependency tracking, and reporting...Senior
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Engineer. Be the first to apply!
Related searches
- finance data engineer Fort Mill, York County, SC
- data engineer machine learning Fort Mill, York County, SC
- data engineering intern summer Fort Mill, York County, SC
- senior cloud data engineer Fort Mill, York County, SC
- aws data engineer Fort Mill, York County, SC
- data engineer analytics Fort Mill, York County, SC
- data engineer Fort Mill, York County, SC
- senior data center engineer Fort Mill, York County, SC
- data science developer Fort Mill, York County, SC
- data center engineer Fort Mill, York County, SC