Senior Data Engineer

Overview Incedo Inc. is a high-growth Digital, Data and AI Transformation Specialist firm headquartered in New Jersey. We are a long-term strategy execution partner for Fortune 500 enterprises, operating at the intersection of business and technology across Banking & Payments, Wealth Management, Telecom, Hi-Tech, and Life Sciences. We are building Incedo 4.0 - an AI-native, execution-focused, founder-led organization designed for scale, speed, and long-term impact. Incedo delivers ROI from AI @ Scale through the “Power of 3”: Engineering & Operations excellence Incedo is a US-based consulting, data science and technology services firm with over 3,500 people helping clients from our six offices across US and India . We help our clients achieve competitive advantage through end-to-end digital transformation. Our uniqueness lies in bringing together strong engineering, data science, and design capabilities coupled with deep domain understanding. We combine services and products to maximize business impact for our clients in telecom, financial services, product engineering and life science & healthcare industries . Working at Incedo will provide you an opportunity to work with industry leading client organizations, deep technology and domain experts, and global teams . Incedo University , our learning platform, provides ample learning opportunities starting with a structured onboarding program and carrying throughout various stages of your career. A variety of fun activities are also an integral part of our friendly work environment . Our flexible career paths allow you to grow into a program manager, a technical architect or a domain expert based on your skills and interests. Role Description We are seeking an experienced Senior Data Engineer with deep expertise in Databricks, PySpark, and cloud data platform delivery . The role involves owning the design and build of production data pipelines on Databricks, leading technical decisions on lakehouse architecture, and working directly with client stakeholders to translate business requirements into working data products. This is a hands-on, client-facing position based in the US (SSF). You will serve as the technical lead for a life sciences data platform build , guiding architecture, mentoring offshore engineers, and owning delivery quality across the engagement. Role and Responsibilities Design and build production data pipelines on Databricks using PySpark, Python, and SQL across Bronze, Silver, and Gold medallion layers. Define lakehouse architecture patterns including Delta Lake table design, partitioning strategy, and compute optimization using Spark/Photon. Configure and manage Unity Catalog for data governance, access control, lineage tracking, and audit logging. Build and maintain ingestion frameworks using Databricks Auto Loader, Lakeflow Connect, and batch/API connectors. Implement data quality checks, validation rules, and monitoring as part of every pipeline deployment. Set up CI/CD pipelines for Databricks notebooks and jobs using GitHub, Databricks Repos, and Lakeflow Jobs. Own technical communication with client stakeholders: architecture walkthroughs, design reviews, sprint demos. Mentor offshore data engineers on Databricks best practices, code quality, and pipeline design standards. Requirements 7+ years of hands-on data engineering experience. 2+ years working on Databricks (notebooks, workflows, Delta Lake, Unity Catalog). 2+ years working with PySpark or Python in a pipeline development context. Experience designing data models for analytical and BI workloads (dimensional models, SCD patterns, medallion architecture). Solid understanding of CI/CD practices for data pipeline deployments (GitHub, Databricks Repos). Experience operating in client-facing, consulting, or services delivery environments. Strong communication skills. Comfortable presenting architecture decisions to both technical and business stakeholders. Hands-on experience with Jira and Confluence . Strong understanding of Agile / Scrum methodologies . Good to Have Life sciences, pharma, or biotech domain experience. Experience with Veeva systems, clinical data, or regulatory data environments. Exposure to data observability tools (Monte Carlo, Databricks Lakehouse Monitoring). Experience with Vector Search, ML Runtime, or MLFlow on Databricks. AWS or Azure cloud platform experience alongside Databricks. Qualifications A bachelor's degree in Computer Science, Information Systems, Engineering, or a related field. A master’s degree may be preferred but is not required. We are an Equal Opportunity Employer We value diversity at Incedo. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. #J-18808-Ljbffr Incedo Inc.