Senior Data Engineer

ABOUT US:As a world leading provider of integrated solutions for the alternative investment industry, Alter Domus (meaning “The Other House” in Latin) is proud to be home to 90% of the top 30 asset managers in the private markets, and more than 6,000 professionals across 24 jurisdictions.With a deep understanding of what it takes to succeed in alternatives, we believe in being different in what we do, how we work, and most importantly in how we enable and develop our people. Invest yourself in the alternative, and join an organization where you progress on merit, where you can speak openly with whoever you are speaking to, and where you will be supported along whichever path you choose to take. Find out more about life at Alter Domus at careers.alterdomus.com Your role:Join our team as a Senior Data Engineer building and operating the Alter Domus data platform, a Databricks lakehouse on AWS that hydrates data from our fund administration, accounting, and market data sources and delivers it to internal teams, client-facing products, and our AI engineering group. You will own pipelines end to end, from source ingestion through governed publication, and provide technical leadership and mentorship to engineers across our onshore and offshore teams in a regulated financial services environment.Your responsibilities:Lakehouse Pipeline EngineeringDesign, build, and operate end-to-end data pipelines on Databricks using Lakeflow jobs, Delta Lake, and Databricks Asset Bundles, deployed through CI/CD across development, simulation, and production environments.Implement pipelines against our medallion architecture, moving data from raw staging through cleansed and conformed layers to business-ready published tables, including quarantine handling for exception records.Build ingestion from JDBC databases, file drops, SaaS APIs, and change data capture feeds, using Auto Loader and incremental load patterns.Operate and improve the existing Airflow (Amazon MWAA) and help migrate workloads onto Databricks-native orchestration.Tune Spark and SQL workloads for performance and cost, including partitioning, file layout, table maintenance, and appropriate use of serverless versus classic compute.Data Modeling and QualityDesign data models across the lakehouse and build modular, testable transformations in dbt, PySpark, and SQL that serve BI reporting, outbound client delivery, downstream APIs, and AI-assisted data access.Build reconciliation and validation directly into pipelines: control count checks across layers, deduplication, idempotent reprocessing, and self-healing recovery paths.Own production support for your pipelines, including monitoring, alerting, and root cause resolution against delivery SLAs, some of which are measured in minutes.Governance and Client DeliveryImplement governance in Unity Catalog: catalog and schema design, tagging, lineage, and group-based grants synchronized from our identity provider.Work with the platform and security teams on fine-grained entitlements so that each client and internal team sees only the data it is permitted to see in a multi-tenant environment.Deliver data outbound to clients and downstream systems through Snowflake and Databricks secure views and shares, file exports, and APIs, and support the onboarding of new clients onto the platform.Enabling AI EngineeringModel and publish curated, well-documented datasets that our AI engineering team can consume reliably.Build and maintain the Genie spaces that allow AI engineers and their agents to query platform data in natural language, including the semantic metadata and metric definitions that make results accurate.Build and operate MCP (Model Context Protocol) services that expose governed platform data to AI agents and applications, with entitlements and audit logging enforced at the access layer.Technical LeadershipProvide technical leadership and mentorship to engineers, including distributed and offshore delivery teams, through design review, code review, and pairing.Drive and document architectural decisions, set standards for pipeline development and testing, and contribute to the platform roadmap.Communicate technical solutions clearly to non-technical stakeholders and senior leadership and translate business requirements into scalable technical designs.Your skills:Education & Experience6+ years of experience in data engineering, including substantial time running a cloud data platform in production.Demonstrated technical leadership: mentoring engineers, leading design, and owning delivery of complex projects.Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience.Technical SkillsDatabricks: strong hands-on experience with PySpark, Delta Lake, job orchestration, and Spark performance tuning at scale, plus Unity Catalog governance, Asset Bundles or equivalent declarative deployment, and Auto Loader. Exposure to Genie spaces, Databricks Apps, or Lakebase is a plus.Advanced proficiency with dbt or a comparable SQL transformation framework, including testing and modular design, with strong dimensional and semi-structured data modeling skills.Expert-level Python and SQL, with solid software engineering fundamentals in testing, documentation, and code quality.AWS: S3, IAM, Secrets Manager, Lambda, Glue, and CloudWatch, with infrastructure as code in Terraform. Working knowledge of Athena and Apache Iceberg is valuable, as part of our estate still runs on them.Airflow, ideally Amazon MWAA, including DAG design, dependency management, retries, and alerting.Experience delivering data to consumers through warehouse shares, file exports, or APIs. Snowflake (secure views, shares, schemachange) is a plus, as is an understanding of how AI agents consume structured data through MCP or similar tool-based access patterns.CI/CD for data pipelines using GitHub Actions or equivalent, containerization with Docker, and environment-promoted, version-controlled deployments.Soft SkillsStrong problem-solving and analytical ability, with a track record of resolving complex production issues.Clear written and verbal communication, with the ability to influence stakeholders and explain trade-offs.Comfortable in a fast-moving environment with shifting priorities and multiple concurrent client commitments.Preferred Certifications & Experience:Databricks certifications (Data Engineer Associate or Professional) are a strong plus.AWS certifications (Solutions Architect, Data Engineer) are a plus.Experience supporting AI or analytics engineering teams as a data provider.Experience in financial services, particularly private equity, private credit, real estate, or fund administration, is not required but will be highly impactful in this role.#LI-SD1WHAT WE OFFER:We are committed to supporting your development, advancing your career, and providing benefits that matter to you.Our industry-leading Alter Domus Academy offers six learning zones for every stage of your career, with resources tailored to your ambitions and resources from LinkedIn Learning. Our global benefits also include:Support for professional accreditations Flexible arrangements, generous holidays, plus an additional day off for your birthday!Continuous mentoring along your career progression Active sports, events and social committees across our offices 24/7 support available from our Employee Assistance Program The opportunity to invest in our growth and success through our Employee Share Plan Plus additional local benefits depending on your location Alter Domus is an Equal Opportunity Employer: Equity StatementAll qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability, or protected veteran status. (Alter Domus Privacy notice can be reviewed via Alter Domus webpage: )#LI-HYBRID