Data Engineering Lead
AceStack LLC
Job Title: Data Engineering Lead Location: Malvern, PA Contract Job Description: Skills: Java| AWS | Python | PySpark | Event-Driven Pipelines | Data Architecture Summary We are seeking an experienced Tech lead- Data Engineer (15+ years) with a strong background in Java, AWS, Python, PySpark, and event-driven architectures. You will design and build scalable batch and streaming data pipelines, optimize cloud data platforms, and deliver high-quality, reliable datasets that support analytics, reporting, and machine learning workloads. Key Responsibilities Architect, build, and maintain event-driven data pipelines using AWS services such as Kinesis, MSK/Kafka, Lambda, Step Functions, SQS/SNS, and Glue/EMR. Develop ETL/ELT workflows using Python and PySpark, ensuring performance, scalability, and cost efficiency. Implement and optimize Spark-based data transformations, partitioning strategies, and data processing frameworks. Design and manage data lake and warehouse structures using S3, Glue Catalog, Athena, and/or Redshift. Build streaming solutions with checkpointing, stateful transformations, idempotency, and schema evolution. Ensure high standards of data quality, observability, monitoring, and alerting (CloudWatch, Datadog, etc.). Implement data security best practices including IAM, encryption (KMS), networking, and governance. Create reusable frameworks, internal libraries, and CI/CD pipelines for automated deployments. Collaborate with data scientists, analysts, and business teams to deliver well-modeled, reliable datasets. Lead design reviews, mentor junior engineers, and contribute to engineering best practices. Required Qualifications 15+ years of professional experience in Data Engineering. Strong expertise in Python and PySpark for large-scale data processing. Advanced hands-on experience with AWS (S3, Glue, EMR, Lambda, Step Functions, Kinesis/MSK, DynamoDB, Athena, Redshift). Deep experience building event-driven and streaming data pipelines. Strong SQL experience for analytical and ETL workloads. Hands-on experience with workflow orchestration tools such as Airflow or Step Functions. Experience with CI/CD, Git, and Infrastructure-as-Code (Terraform or CloudFormation). Strong understanding of distributed systems, Spark performance tuning, data modeling, and cloud cost optimization. Knowledge of data security, encryption, networking, and compliance best practices in cloud environments. Soft Skills Strong design and architectural understanding. Excellent communication and stakeholder interaction skills. Ability to work in a globally distributed team. AceStack LLC
- Role Summary We are seeking a highly skilled Senior Data Engineer (or Lead Data Engineer ) to design, build, and optimize scalable data pipelines and ETL solutions across enterprise environments. The ideal candidate will bring deep expertise in Big Data technologies ,...Suggested
- Role Purpose Serve as the onsite Data Engineering and Architecture lead, driving technology innovation, evaluation, and adoption across the enterprise data landscape. Key Responsibilities Act as the onsite SME for Data Engineering and Data Architecture. Lead Proof of...Suggested
$111.44k - $183.08k
...our name, it's our mission and our purpose. The Enterprise Data Management Lead drives the strategy, standards, and execution of enterprise... ...and architecture principles, this is not a hands-on engineering or ETL/ELT role; the focus is on embedding governance, quality...SuggestedWork experience placementWork at officeFlexible hours- ...designs database systems, and develops tools for real-time and offline analytic processing. Troubleshoots software and processes for data consistency and integrity. Integrates data from a variety of sources for business partners to generate insight and make decisions....Suggested
- ...Job Description Job Description Lead Data Engineer / Data Architect (Innovation & POC Lead) Location : Exton, PA Rate : DOE Job Description Serve as the onsite Data Engineering and Architecture lead, driving technology innovation, evaluation, and adoption...Suggested
- Machine Learning Engineer We are seeking an experienced Machine Learning Engineer to join our AI/ML Engineering team. You will be responsible for developing and optimizing complex data pipelines, integrating model pipelines, and building scalable AI/ML solutions, including...Work experience placement
- Data Lake 2.0 Developer Data Lake 2.0 is a multi-year program aimed at modernizing legacy... ...that will be in use by Analysts and Data Engineers. The deliverables will help data... ...Work on features and POCs with industry-leading technologies and tools Work on greenfield...
- ROLE SENIOR DATA ENGINEER INDUSTRY Pharma, Development & Manufacturing DURATION CONTRACT, probable extension LOCATION REMOTE (off-shore or near-shore), Malvern, PA 19355 INTERVIEW MS TEAMS conversation the candidate will be required to work in EST starting around 1...Contract workImmediate startRemote work
- Responsibilities Pulling data from GitHub and Jira transforming it for input into our data warehouse. Qualifications Familiarity with AWS including lambda, event bridge, s3. Proficient in typescript Datalake / Data warehouse experience is a big plus. Required Skills :...
- Senior Data Engineer - 7+ Years of Experience We are seeking a highly experienced Senior Data Engineer with 7+ years of expertise in designing, building, and optimizing robust data solutions. The ideal candidate must possess top-tier skills in Python, AWS services, API...
- Data Engineer Duties And Responsibilities Data engineers create and maintain an optimal data infrastructure. They combine raw information to create machine-readable formats. These are the major duties and responsibilities of a data engineer: Putting together large, intricate...
- Experience in the below technologies. ETL using EMR & Glue require working experience. SQL/PYSPARK/PYTHON/Jupyter Notebooks/GitHub functions/ CI/CD Knowledge on cloud Services: EMR/GLUE/S3/EC2/Service Catalog/PostgreSQL/Lambda/cloudFormation/SNS/EventBridge Knowledge ...Work experience placement
- Data Engineer Location: Malvern, PA - Hybrid Important Submission Requirement Candidates must take and pass a technical assessment before submission . Only submit candidates who are willing and available to complete the assessment. Position Overview Vanguard is seeking...
- Data Engineer As a Data Engineer, you’ll build sophisticated solutions that support a wide range of advanced data driven capabilities. Provides... ...in specific business areas and company-wide systems. Leads all phases of solution development. Explains technical considerations...
- Job Title Responsibilities Ability to run and deploy jobs, view creation and deployment, testing, analyzing, and cleansing large data files Qualifications Expert in SQL AWS S3 and Glue experience Advanced Data Modeling principles Required Skills : Data Warehouse Basic...
- Job Summary The Databricks AWS Data Engineer will develop and support a new data environment from the ground up, focusing primarily on data pipelines and cloud-based data engineering solutions. The role requires strong development experience with Python, PySpark, and SQL...
- ...Troubleshoots software and processes for data consistency and integrity. Integrates complex... ...business to implement data solutions. Leads all phases of solution development.... ...client. Educates and develops junior data engineers on the team while applying quality control...
- Job Title: Senior Data Engineer Location: Malvern, PA Onsite Duration:6 Months Experience Required: 10 & Above EXPERIENCE_RANGE_IN_REQUIRED_SKILLS_ Python, AWS- S3, AWS Lambdas, AWS Glue, SQL, DynamoDB , Kafka/ Kinesis , pySpark ROLE_DESCRIPTION - Overview: We are seeking...Local area
- Details given below:- Role:-ETL Data Engineer Malvern, PA Charlotte, NC Phoenix, AZ JD:- • Writes ETL (Extract / Transform / Load) processes, designs database systems and, develops tools for real-time and offline analytic processing. • Deep technical knowledge - including...
- Job Summary The AWS Data Engineer will design, develop, and implement scalable, high-performance data ingestion and transformation pipelines using AWS services and Apache Spark. The role involves integrating complex datasets from multiple sources, developing advanced SQL...
- Purple Drive AWS Data Engineer - Python, AWS Glue, PySpark & Attunity We are seeking an experienced AWS Data Engineer with strong expertise in Python, AWS Glue, PySpark, Attunity (Qlik Replicate), and AWS Cloud to design, develop, and maintain scalable enterprise data integration...
- Role : Data Engineer with Python Location: Malvern, PA (Onsite) Contract Job Summary We are seeking a Data Engineer with strong Python expertise to design, build, and maintain scalable data pipelines and data infrastructure. The ideal candidate will work with large datasets...Contract work
$130k - $160k
...Job Description Job Description Job Title: Data Governance Lead _ Data Governance, Data Quality, Lineage & Observability Location:... ...with the ability to engage business, risk, compliance, data, engineering and executive stakeholders. Preferred certifications in...Full time- We are looking for a Snowflake Data Engineer to build modern data solutions that support analytics, AI, and enterprise decision-making. This role will focus on creating reliable pipelines, scalable data models, and secure data products within a Snowflake environment. The...
- ...Role at a GlanceLincoln Financial's AI, Data and Analytics (AIDA) organization is building... ...with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams, with... ....As a Senior Data Engineer, you wil lead the end-to-end technical direction and delivery...For contractorsWork experience placement3 days per week
- ...least 4 years of experience in software design, development and release.Well versed with agile development methodologies.Experience in Data Ingestion, Streaming and Big Data technologies.Excellent communication and stakeholder managementAbility to work with distributed...Permanent employmentFull timeH1b
- ...Overview This is a hybrid role - 2 days remote and 3 days in the Malvern, PA office. CubeSmart is seeking a Manager, Data Engineering to lead the team responsible for building and evolving the enterprise data platform that powers analytics, reporting, AI, and...Work at officeRemote work
- A leading technology firm is seeking a Software Developer with at least 4 years of experience in software design, development, and release... ...requires strong analytical skills and familiarity with Big Data technologies and agile methodologies. Ideal candidates will possess...Permanent employmentFull timeH1b
- ...opportunity.Requisition #: 76677The Role at a GlanceAbout AIDA Data Strategy and GovernanceLincoln Financial's AI, Data and Analytics... ...the result. We deliver with AIDA's infrastructure, platform, engineering, MLOps, and AI-native platform teams, with governance built into...For contractorsWork experience placement3 days per week
- Role Overview: This role is for a Data Engineer focused on designing, implementing, and optimizing robust ETL/ELT pipelines and complex data models primarily using PySpark and Snowflake. The engineer will be responsible for ensuring high data quality, integrity, and efficient...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineering Lead. Be the first to apply!



