Senior Data Architect

Senior Data Lakehouse Architect (Databricks), Vice PresidentCorporate Functions TechnologyWho We Are Looking ForWe are seeking a Senior Data Lakehouse Architect to design and lead the build-out of a Legal Data Lakehouse platform on AWS and Databricks. This role will drive the architecture, engineering, and governance of scalable, secure, and compliant data capabilities supporting legal operations, contract intelligence, eDiscovery, and AI/ML use cases.The ideal candidate brings deep expertise in Databricks, AWS data platforms, and enterprise data architecture, with experience delivering solutions in regulated environments aligned to security, compliance, and audit requirements.Why This Role Is Important to UsState Street's Legal function operates across a broad set of contracts, matters, regulatory obligations, documents, and workflows that are distributed across multiple systems and formats. Building a modern Legal Data Lakehouse is critical to creating a trusted, governed foundation that brings these data sources together—making legal information easier to access, analyze, and use at scale.This role is critical to establishing a secure and scalable data foundation that enables legal analytics and AI use cases while strengthening governance, auditability, and global consistency across Legal.What You Will Be Responsible For1. Architecture & Platform DesignDefine and implement the end-to-end Legal Data Lakehouse architecture using Databricks (Delta Lake, Unity Catalog, Workflows) on AWSDesign multi-layered data architecture (Bronze, Silver, Gold) to support: Contract metadata and document ingestionLegal matter management dataeDiscovery datasetsExternal regulatory and compliance feedsEstablish scalable ingestion frameworks (batch and streaming) for structured and unstructured legal data (PDFs, contracts, emails)2. Data Engineering & IntegrationLead development of ETL/ELT pipelines using Databricks, Spark, and Python/SQLIntegrate with enterprise platforms, including: Contract lifecycle management systemsAI platforms and LLM pipelinesDocument repositories and enterprise content systemsDesign patterns for extracting structured data from unstructured legal documents and persisting into Delta LakeEnable downstream integration with enterprise data platforms, analytics tools, and AI/ML pipelines3. Governance, Security & ComplianceImplement data governance frameworks using Databricks Unity Catalog and AWS-native controls (IAM, KMS)Establish: Fine-grained access controls (row/column-level security)Data lineage and auditabilityEnsure compliance with: Data privacy regulations (e.g., GDPR)Internal security and audit requirementsPartner with IAM teams to integrate with enterprise identity providers (e.g., Entra ID / Azure AD)4. AI/ML & Advanced Analytics EnablementArchitect data models supporting: Contract analytics, clause extraction, and obligation trackingLegal AI use cases (contract review, litigation insights, compliance monitoring, legal spend analytics)Design search and retrieval architectures (RAG) for enterprise legal knowledge basesEnable entity extraction and knowledge graph frameworksIntegrate with LLM/GenAI platforms to support capabilities such as document summarization, Q&A, and workflow automation5. DevOps & Platform OperationsEstablish CI/CD pipelines and infrastructure-as-code (Terraform, Git-based workflows)Define standards for: Code quality and versioningEnvironment promotion (Dev / QA / Prod)Implement observability and alerting for platform health and reliability6. Leadership & Stakeholder EngagementPartner with Legal and Technology leadership to define platform roadmap and prioritiesProvide architectural governance and design oversightMentor data engineers and platform teamsTranslate business and legal requirements into scalable, enterprise-grade solutionsOperate within a federated data and platform model, collaborating across engineering, security, and domain teamsWhat We ValueThe skills that will help you succeed in this role include:10+ years of experience in data architecture, engineering, or analytics platforms5+ years of hands-on experience with Databricks and Apache SparkStrong experience with AWS-based data platformsExpertise in data governance, security, and compliance in regulated environmentsExperience working with unstructured data and NLP/document processing pipelinesEducation & Preferred QualificationsEducationBachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related technical disciplineRelevant certifications strongly preferred: Databricks Certified Data Engineer / ArchitectAWS Certified Solutions Architect (Associate or Professional)Preferred Qualifications (Core – Databricks, AWS, Data Lakehouse)Strong hands-on experience with the Databricks Lakehouse platform, including Delta Lake, Unity Catalog, Workflows, and MLflowDeep expertise in AWS data platform services, including S3, Glue, EMR, Lambda, Redshift, and IAMProven experience architecting and delivering enterprise-scale data lakehouse solutions on AWS using DatabricksAdvanced proficiency in Apache Spark (PySpark/Scala), SQL, and PythonStrong understanding of data governance and security, including access controls, metadata management, and encryption (KMS, CMK/BYOK)Experience building end-to-end data pipelines (batch and streaming) and supporting AI/ML workloads within a lakehouse architectureNice to Have (Domain & Industry Experience)Experience in Legal, Compliance, Financial Services, or other regulated industriesUnderstanding of legal data constructs, including contracts, clauses, obligations, and mattersExperience supporting legal AI use cases (contract analytics, document summarization, compliance monitoring)Experience handling sensitive data in highly regulated, audit-driven environmentsSalary Range:$120,000 - $202,500 AnnualThe range quoted above applies to the role in the primary location specified. If the candidate would ultimately work outside of the primary location above, the applicable range could differ.Employees are eligible to participate in State Street's comprehensive benefits program, which includes: our retirement savings plan (401K) with company match; insurance coverage including basic life, medical, dental, vision, long-term disability, and other optional additional coverages; paid-time off including vacation, sick leave, short term disability, and family care responsibilities; access to our Employee Assistance Program; incentive compensation including eligibility for annual performance-based awards (excluding certain sales roles subject to sales incentive plans); and, eligibility for certain tax advantaged savings plans.