Senior Data Engineer

Senior Data EngineerAs a Senior Data Engineer at Regard, you will own the design, development, and production deployment of the data services that power the Regard platform. From ingesting and standardizing clinical data across health systems to making it reliably available for downstream product, analytics, and machine learning workflows, you'll build and evolve the infrastructure that enables the platform. This includes analyzing and tuning Spark workloads and partitioning strategies to control costs, adapting to upstream breaking changes, and enforcing rigorous data quality standards so our analytics are as dependable as our application code. We prioritize transparent, code-driven systems over black-box services, and you'll help architect the data platform that supports that philosophy.Our mission is to bring world-class healthcare to everyone. Regard is an AI-powered Proactive Documentation platform that advances how care is delivered by reviewing all patient data in the EHR to recommend diagnoses and surface clinical evidence. Regard drafts a note even before the physician sees the patient, enabling an approach that gets documentation right at the point of care - we call it Proactive Documentation. This improves quality of care, reduces physician burden, and improves hospital finances. We are excited by challenges, mission-oriented work, and meaningful relationships. We work closely with some of the top health systems in the country and are leading the change that healthcare - one of the largest and most inefficient industries in the world - needs. We want you to join us.Our tech stack:Data: S3, Apache Iceberg, EMR, PySpark, Dagster, Kubernetes, Clickhouse, PostgreSQL, FastAPI, MetabaseResponsibilities:Collect, model, and consolidate data into the data platform to support analytics, ML development, and research initiativesDesign, build, and evolve data models and pipelines that reliably transform and deliver data to downstream consumersOwn data quality in collaboration with engineering teams, ensuring datasets are trustworthy and production-readyPartner closely with product to deliver analytics and actionable insights to internal and external stakeholdersOwn the reliability and day-to-day operation of the data platform and its pipelines through proactive monitoring, alerting, and operational managementMinimum qualifications:Bachelors degree in Computer Science, Mathematics, Statistics, or a related field, or equivalent practical experience5+ years of experience in data engineering roles3+ years of experience using PySpark to build data pipelines3+ years of experience in public cloud provider technologies (AWS tooling such as S3, EMR, or Athena)Strong proficiency in Python and SQLHands-on experience across the full data stack, with particular depth in data modeling and pipeline designPractical experience with LLM-assisted development, with an understanding of its capabilities and limitationsWillingness to participate in on-call operational support for owned systemsPreferred qualifications:Experience with one or more of the following technologies: Apache Iceberg, Dagster, Clickhouse, PostgreSQL, FastAPI, MetabaseExperience working with healthcare data, including HIPAA compliance, data de-identification, and familiarity with open data standards such as OMOP CDMExperience building and supporting data pipelines for ML workflows, including model training, validation, deployment, and ongoing performance evaluationHybrid work | Location | Work authorization:For this role, Regard is currently only considering candidates who are authorized to work in the US without visa sponsorship, and are within the New York City, Los Angeles, or San Francisco metro areasWe expect our Engineers to be in the office on Tuesdays and Thursdays. We also require more frequent in-office work during the onboarding period and team onsite weeks up to once per monthWe will provide relocation assistance to anyone who does not already reside in the NYC metro areaWe prefer hiring people within commuting distance of our offices because we value getting together in person regularlyFor those who enjoy working from our LA or Manhattan offices on a more regular basis, we offer catered lunches and other fun perksAdditionally, hybrid employees have the flexibility to work from locations outside of their home office from up to 6 weeks per yearComp | Perks | Benefits:Eligible for equity99% employer paid health benefits (Medical, Dental, and Vision) + One Medical subscription18 PTO days/yr + 1 week holiday breakMonthly health & wellness budgetCompany-sponsored team retreat + social eventsA sabbatical programOur goal at Regard is to provide and maintain a work environment that fosters mutual respect, professionalism and cooperation. Regard is proud to be an equal opportunity employer that does not discriminate on the basis of actual or perceived race, creed, color, religion, national origin, ancestry, alienage or citizenship status, age, disability or handicap, sex, gender identity, marital status, familial status, veteran status, sexual orientation or any other characteristic protected by applicable federal, state or local laws. We celebrate diversity and are proud of our supportive, inclusive workplace.All candidates must successfully complete a background check as part of the hiring process.