Sr. Data Engineer
Long Finch Technologies
Job Description: Sr. Data Engineer – Data Ingestion Framework
Python, DBT, and orchestration technologies such as Dagster or Control-M. The solution will utilize AWS S3 as
the data Lakehouse storage layer and Snowflake as the target data platform.
Experience: 10–12 Years
Location: Remote – USA (Onshore)
Work Arrangement: Remote or Hybrid (preferred local to Boston)
Engagement: Data Platform Modernization / Data Ingestion Framework
Position Overview
We are looking for an experienced Lead Data Engineer with strong expertise in Python and DBT to design and develop a scalable, reusable, and technology-agnostic data ingestion framework.
The role involves replacing existing ETL capabilities with an open-source technology stack leveraging Python, DBT, and orchestration technologies such as Dagster or Control-M. The solution will utilize AWS S3 as the data Lakehouse storage layer and Snowflake as the target data platform.
The ideal candidate should have hands-on experience building enterprise-grade data ingestion frameworks, developing reusable connectors, implementing metadata-driven pipelines, and integrating heterogeneous source systems with modern cloud data platforms.
Key Responsibilities
1. Data Ingestion Framework Development
- Develop a reusable, scalable, and technology-agnostic data ingestion framework using Python.
- Design modular and extensible components to support multiple source systems.
- Develop metadata-driven and configuration-based ingestion capabilities.
- Build reusable connectors and adapters for different source technologies.
- Implement full and incremental data ingestion patterns, including CDC where applicable.
- Implement robust error handling, logging, auditing, retry, and reprocessing mechanisms.
2. Data Integration & Lakehouse Architecture
- Develop ingestion pipelines to integrate data from heterogeneous source systems into AWS S3 and Snowflake.
- Implement data ingestion from SQL Server, Oracle, PostgreSQL, REST APIs, SFTP, AWS S3, and other source systems.
- Design and implement data lakehouse ingestion patterns using AWS S3.
- Work with file formats such as CSV, JSON, Parquet, and other structured/semi-structured formats.
- Implement schema handling, data validation, partitioning, and data movement strategies.
3. Python & dbt Engineering
- Develop production-grade Python applications and reusable ingestion libraries.
- Build and maintain dbt models for data transformation and integration.
- Implement dbt testing, documentation, incremental models, and reusable macros.
- Optimize Python-based ingestion processes and dbt transformations for performance and scalability.
- Follow engineering best practices for modularity, maintainability, and code quality.
4. Workflow Orchestration
- Implement and manage workflow orchestration using Control-M and Dagester.
- Develop reusable workflows for scheduling, dependency management, retries, and pipeline monitoring.
- Integrate orchestration workflows with Python ingestion components and Snowflake.
5. Testing, Deployment & Production Support
- Develop unit tests and integration tests for ingestion framework components.
- Support migration and deployment across DEV, SIT, UAT, and PROD environments.
- Collaborate with QA teams to perform functional, integration, regression, and performance testing.
- Troubleshoot ingestion failures, data discrepancies, and performance bottlenecks.
- Support production deployment, stabilization, documentation, and knowledge transfer.
6. Technical Leadership & Collaboration
- Work closely with the onshore solution architect, customer stakeholders, and offshore engineering teams.
- Provide technical guidance, code reviews, and engineering best practices.
- Participate in architecture discussions, technical design reviews, and sprint planning.
- Mentor developers and ensure adherence to established engineering standards.
Required Technical Skills
- Python: Expert-level hands-on experience in Python development, including object-oriented programming, reusable libraries, APIs, error handling, and performance optimization.
- DBT: Strong hands-on expertise in DBT Core, including models, macros, tests, incremental processing, and deployment practices.
- Data Engineering: Extensive experience building enterprise-scale ETL/ELT pipelines and reusable data ingestion frameworks.
- Snowflake: Strong experience with Snowflake architecture, data loading, stages, Snowpipe, SQL, and performance optimization.
- AWS: Hands-on experience with AWS S3, IAM, and cloud-based data integration patterns.
- Data Integration: Experience integrating relational databases, REST APIs, SFTP, and cloud-based storage systems.
- SQL: Advanced SQL development and query optimization skills.
- CI/CD: Experience with Git-based version control, automated builds, testing, and deployment pipelines.
Good-to-Have Skills
- Hands-on experience with Dagster for data orchestration and asset-based pipeline development.
- Experience with utilizing Control-M JOB scheduler.
- Experience with Apache Iceberg and open table formats.
- Experience implementing metadata-driven ingestion frameworks.
- Experience migrating ETL workloads from SnapLogic or similar integration platforms.
- Experience with Docker, Terraform, and infrastructure automation.
- Experience implementing data quality, observability, and monitoring solutions.
- Experience working in an Agile environment utilizing SCRUM
Qualifications
- 10–12 years of overall experience in data engineering and data integration.
- Strong architectural and hands-on engineering capabilities.
- Experience working on enterprise data modernization initiatives.
- Excellent communication and stakeholder management skills.
- Ability to work independently in a distributed onshore/offshore delivery model.
Mandatory Skills
- Expert-level Python development.
- Strong hands-on dbt experience.
- Enterprise data ingestion framework development.
- Snowflake and AWS S3 experience.
- Strong SQL and data integration experience.
Preferred: Dagster, Control-M, Apache Iceberg, and SnapLogic migration experience.
- ...exceptional team that embodies a true startup mindset. The Analytics Engineering Team at ABCorp partners closely with Product Management, Engineering, Sales, and Customer Success to build the data foundations that drive product strategy and business decisions....SeniorWorldwide
- ...NessContact: Meghana GorusuCompany: SRI Tech SolutionsJob Title: Sr Data EngineerLocation: Boston, MA (3-4 days a week onsite)Skill Set:... ...SQL and PythonJob Description:We are looking for a Senior Data Engineer with added emphasis on data analysis and Salesforce knowledge,...SeniorWork at office3 days per week
$116.2k - $229.1k
Position Summary Join Deloitte’s AI & Engineering practice and help organizations transform enterprise technology platforms, modernize data environments, and unlock value through innovation. As a Databricks Engineer in our AI & Data practice, you will design, build...SeniorLocal areaVisa sponsorship$150k - $165k
...your opportunity to grow professionally in a dynamic, global environment, surrounded by forward-thinking peers.The role of Senior Data Engineer is essential for the successful modernization of our data platforms and practices. The Senior Data Engineer will collaborate...SeniorFull time$63 - $67 per hour
...Location: Boston, MA Salary: $63.00 USD Hourly - $67.00 USD Hourly Description: Our client is currently seeking a Senior Data Engineer Position Summary We're seeking a Senior Data Engineer with a strong backend foundation to build and operate data pipelines...SeniorHourly pay- ...Job Description Job Description Senior Analytics Engineer – Data Architecture, dbt, SQL & AI Job Title: Senior Analytics Engineer Job ID: 1006 Location: Boston, MA Duration: 12 Months Contract Work Arrangement: Remote/Hybrid – Preference for candidates...SeniorContract workLocal areaRemote work
$120k - $187k
...: Information TechnologyLocation: Boston, MA, United StatesWorkplace: HybridSchedule: Full timePosted: 2026-10-27As a Senior BI Data Engineer, you will design, build, and optimize enterprise-scale data solutions that power analytics and business decision-making across Boston...SeniorFull timeWork at office- ...Vice President, Senior Data Engineer About the Company Mission-driven provider of tax credit syndication services Industry Real Estate Type Privately Held Founded 2001 Employees 51-200 Specialties lihtc tax credit syndication low income...Senior
$140k - $160k
...company that enables organizations to share and analyze sensitive data at speed without compromising security or control. VIA’s... ...impressive mission requires an equally impressive Senior Data Analytics Engineer. As a Senior Data Analytics Engineer at VIA, you'll play a...SeniorSummer workRemote workWork from homeFlexible hours$98.5k - $141.5k
...institutions who rely on us to help them build more secure and prosperous futures. THE ROLEThe Investment Data Management Office is seeking a Senior Investment Data Engineer with strong SQL and data quality assurance experience to support the implementation, testing, and...SeniorFull timeWork at officeLocal areaRemote workFlexible hours$150k - $180k
...humanitarian missions worldwide. With 1,700+ team members, 1,500+ AI/data experts, and 100+ prime contracts, we deliver at scale and with... ...Summary Credence has an immediate need for a Senior AI Data Engineer to join our growing AI and Automation practice. You will be a...SeniorTemporary workImmediate startWorldwide- ...passionate about getting physicians and healthcare providers away from the keyboard and back to clinical care. Overview The Senior Data Engineer is a member of the Data Platform team, reporting to the Director of Machine Learning Engineering and Data. The Data Platform...SeniorFull timeTemporary workWork at office
- Job Title: Sr.Data Engineer and Data Engineer Location: CST or EST PS Offfice (Boston, Atlanta, Houston, Chicago, New York) must be 40miles to an office and able to pick up laptop/id verification day 1-3 days a week onsite Length: 6-10 months Build and maintain...SeniorContract workWork at office3 days per week1 day per week
$132.9k - $182.7k
...our best and know that we’re making a difference. As a Senior Structural Engineer professional based within our Advanced Manufacturing team, you’ll work on a variety of projects related to Data Centers, including new builds, retrofits, colocations, etc. You will be performing...SeniorFull timeRemote work$171.2k - $228.2k
...in their home. We're investing in a multi-quarter Enterprise Data Platform program to move our data and analytics stack onto a... ...modern lakehouse. We're looking for a Senior Manager, Data Platform Engineering to lead the team that builds, operates, and evolves that...SeniorWork at office$100k - $150k
...Job Description As a Senior Data Scientist / Product Engineer at Kearney Activate, you will play a key role in delivering secure, cloud-hosted data and analytics solutions for clients within Kearney’s Mobility, Defense and Advanced Industrials practice. This role is designed...SeniorFull timeH1bImmediate startHome officeVisa sponsorshipFlexible hours$179k - $236k
...intervention points for new therapies. Vesalius was founded in 2019 by Flagship Pioneering. Role Overview We are seeking a Principal Data Engineer to lead the architecture and evolution of the data platform underpinning our human genetics, multi-omics, and high-content...H1bRelocationWork visa$152k - $228.5k
...Principal Data EngineerWith 1,000+ intelligence professionals serving over 1,900 clients worldwide, Recorded Future is the world's most... ...Protection data streams. We are looking for a Principal Data Engineer to drive improvements in the Digital Risk Protection department....Full timeWorldwide- Senior to Principal Data Engineer (SQLserver, Financial Services), Boston, MA. Hybrid, min 3 days week in office, 135k to 175k base on experienceCompensation Commensurate with experience, EOE, diverse highly educated culture Candidates must be a U.S. citizen or national...SeniorPermanent employmentContract workWork at officeLocal area3 days per week
- ...Control Tower team at State Street. The role focuses on delivering reporting and analytics by designing Power BI dashboards, building data models, and performing ETL and data transformations to turn data into actionable insights. Key Responsibilities Design, develop, and...SeniorTemporary work
$119k - $299.93k
...Senior Manager Job Description & Summary The Opportunity As part of the AI Engineering team within the Digital Assurance & Technology team, you will play a pivotal role in transforming raw data into actionable insights, enabling informed decision-making and driving business...SeniorFull timeH1b$115k - $150k
About the jobSenior Associate, Data Products | Finance | Boston, MAOverviewJoin a high-performing investment-focused organization as... ...platforms.Required Experience6+ years of experience in Data Engineering, Data Platform Engineering, or Software Engineering.Deep expertise...SeniorWork at office3 days per week$300k
Senior Data EngineerOur client is one of the world's largest privately held investment management firms, overseeing hundreds of billions... ...and best practices across the organizationPartner closely with engineering, data, business, and investment stakeholders to translate...Senior- Description At Liberty Mutual Investments Technology, we are seeking a skilled Senior Data Engineer who is ready to take on exciting challenges, contribute to our cloud-native data platform, and make a real impact across our investment insights. You will build modern...SeniorWork at officeLocal area
$140k - $200k
Boston, MAHybridFull Time$140k - $200k Job Description A rapidly growing scientific data and AI technology company is seeking a Senior Scientific Data Engineer to join its fully remote team. This role offers the opportunity to work at the intersection of data engineering...SeniorFull timeTemporary workImmediate startRemote work$175.5k - $180k
...technical owner, designing and leading the implementation of scalable data architectures for cutting-edge AI and agentic systems. You will... .... You will also spearhead the design of complex feature engineering workstreams, ensuring our data assets are not only robust but also...SeniorApprenticeshipEasy work$195.3k - $199.2k
.... In this role you will design, develop, and maintain scalable data pipelines and systems using Databricks on Azure (among other environments... ...sharing capabilities. You will be leading and executing data engineering projects from inception to completion, ensuring timely delivery...SeniorApprenticeshipEasy work- Senior Data Engineer - Boston, United StatesWe are partnering with our client, a leading institutional asset manager based in Boston's financial district, to appoint a Senior Data Engineer to join our client's team. In this permanent role, you will be instrumental in building...SeniorPermanent employmentWork at officeRemote workMonday to Friday
- Senior Investment Data Engineer, Boston, MA, Hybrid, 2 days in, 150k to 190kCompensation Commensurate with experience, bonus, benefits, EOE, diverse highly educated cultureCandidates must be a U.S. citizen or national, refugee, asylum, or lawful permanent resident. Principals...SeniorPermanent employmentWork experience placementCasual workWork at office
- As a Senior Data Engineer, CASM Platform, you will: Data Integration, API Development: Integrate diverse cybersecurity data sources using variety of API mechanisms and to standardize and streamline data across the data and user planes.Build and maintain data APIs for seamless...SeniorContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Sr. Data Engineer. Be the first to apply!
- senior cloud data engineer Boston, MA
- big data cloud engineer Boston, MA
- junior big data engineer Boston, MA
- data engineering intern summer Boston, MA
- data engineer analytics Boston, MA
- sr data engineer Boston, MA
- big data developer Boston, MA
- principal data engineer Boston, MA
- data developer Boston, MA
- sr information security engineer Boston, MA




