Senior Data Engineer

Senior Data Engineer Information Technology All Practice Areas Austin, TX, Boston, MA, Charleston, SC, Charlotte, NC, Chicago, IL, Dallas, TX, Durham, NC, Harrisburg, PA, Houston, TX, Irvine, CA, Kansas City, MO, Los Angeles, CA, Miami, FL, Nashville, TN, New York, NY, Newark, NJ, Palo Alto, CA, Pittsburgh, PA, Portland, OR, Raleigh, NC, San Francisco, CA, Seattle, WA, Washington, DC, Wilmington, DE At K&L Gates, we are looking for smart, imaginative and hard-working people with diverse backgrounds, experiences and ideas to join us. Perhaps our search for talented visionaries and your search for important and impactful work lead to the same place. We are seeking a Senior Data Engineer to join the firm. The Senior Data Engineer contributes to the Enterprise Applications, Data, and AI Platforms group, designing scalable Microsoft Fabric data platforms. Responsibilities include establishing engineering standards, leading design, implementation, and optimization of Lakehouse, Warehouse, and Azure AI Foundry data pipelines. Role involves mentoring staff, collaborating with business and technology leaders, and delivering reliable governed products aligned with the firm’s AI strategy. Reporting to the Senior Manager, Data Engineering, this position focuses on technical leadership and does not include direct supervisory responsibilities. KEY RELATIONSHIPS Director, Enterprise Applications, Data, and AI Platforms Senior Manager, Data Engineering Senior Manager, Automation and Development Senior Manager, Enterprise Applications Senior Applications Architect Senior Data Architect Team Lead, Workflow and Automation Technical Program Manager Project Manager Cloud Platform Engineer Security Engineer Business Data Experts and Business Data Sponsors ESSENTIAL DUTIES Designing scalable and resilient data platforms with Microsoft Fabric, including Lakehouse and Warehouse solutions on OneLake, while making independent technology decisions. Establishing and maintaining data engineering standards, patterns, and best practices across the data engineering group and the wider EA organization. Designing, developing, and overseeing complex data pipelines using Fabric Data Factory, Dataflow Gen2, and Fabric Notebooks for ingestion, transformation, and advanced automation. Leading schema design, dimensional modeling, Medallion Architecture implementation, and semantic model development to enable analytics, reporting, and AI-related workloads effectively. Optimizing data performance, reliability, and cost by proactively tuning, monitoring, and validating data quality through established processes. Ensuring data governance, quality, and security standards are integrated into solutions, collaborating with governance leads to apply classification, lineage, and access controls. Designing and maintaining data pipelines supporting Azure AI Foundry workloads, including grounding datasets, vector indexing preparation, and feature data for retrieval-augmented generation. Providing mentorship and guidance to junior data engineers through code reviews, design sessions, and technical coaching, enhancing the team's overall engineering quality. Partnering with business stakeholders to convert complex requirements into scalable data architectures while collaborating within an Agile DevOps model. Supporting CI/CD pipelines, automated deployments, and maintaining architecture documentation. Serving as a model for the Enterprise Applications team philosophy, advocating enterprise processes, procedures, security, standards, and knowledge-first support. Defining and overseeing testing strategies for assigned data solutions, including quality standards, user acceptance frameworks, and validation of data accuracy. Leveraging AI tools like Microsoft Copilot and Claude to aid development and contribute to AI solutions on the Microsoft platform stack. Adhering to governance standards, ensuring responsible use, data classification, and implementing model deployment controls effectively. EXPERIENCE & SKILLS REQUIRED Hold Bachelor’s degree in a technical field; equivalent experience accepted, with ability to obtain Microsoft certifications including DP-600, DP-203, and AI-900. Understand Data Mesh and Microsoft Purview, including domain ownership, federated governance, metadata management, cataloging, and sensitivity labeling. Maintain a secure workspace and willingness to travel up to 10% for offices, vendor sites, conferences, or seminars. Participate in scheduled release windows and production support events, including evenings or weekends, with advance notice and flexible scheduling. Demonstrate the ability to implement data governance, ensure quality, optimize performance, and integrate security controls into enterprise data solutions. Translate complex business requirements into scalable architectures and communicate designs visually using diagramming, whiteboarding, and collaborative tools. Collaborate effectively with technical and executive stakeholders while mentoring engineers and leading technical decisions to elevate engineering quality. Adapt to new technologies and utilize AI-powered tools in enterprise settings to enhance productivity and achieve improved results. 5+ years designing enterprise data platforms using Microsoft Fabric, including pipelines, schema design, dimensional modeling, Medallion Architecture, and semantic layer development. Demonstrate expertise in SQL and Python for data transformation and automation, utilizing Spark and CI/CD pipelines through Azure DevOps or GitHub Actions. Possess experience with Azure AI Foundry pipelines and real-time, event-driven architecture using Microsoft Fabric Real-Time Intelligence. Possess exceptional experience applying enterprise data governance practices or contributing to data governance or architecture councils. Compensation Salary $128,000 - $ 249,000 / year In accordance with the Los Angeles Fair Chance Ordinance, San Francisco Fair Chance Ordinance, and the California Fair Chance Act, qualified applicants for our Los Angeles and San Francisco offices with arrest and conviction records will be considered. BENEFITS Medical/Prescription Drug Coverage (including a Health Savings Account feature) Back-up Child/Elder Care and access to a caregiving concierge Wellness Program Vision Insurance 401(k) Retirement Plan and Profit Sharing Business Travel Accident Insurance Short- and Long-term Disability Protection Pet Insurance Life Insurance (including Basic, Supplemental, Spouse, Child, and Accidental Death and Dismemberment) Health Advocacy Services Paid Time Off (25-30 days per year) Identity Protection/Restoration and Fraud Insurance Parental Leave (18 weeks of which 6 are paid; short-term disability may provide additional paid time off) Student loan refinancing options and access to a student loan concierge service Paid Holidays (12) Addiction Resources Family Building Benefits Breast Milk Delivery and Lactation Support Services Flexible Spending Accounts Employees also may be eligible to receive bonuses and certain expense reimbursements Employee Assistance Program Professional Development and CLE Credit Opportunities 529 Deductions Relocation Employee Referral Program Critical Illness Insurance Hospital Indemnity Insurance Perks including: Technology, Entertainment, and Travel Discount Programs All other benefits (such as leaves of absence) required by law EQUAL EMPLOYMENT OPPORTUNITY The Firm is an equal opportunity employer. It is the policy of the Firm that employment decisions shall be based on merit, qualifications and competence. Employment practices will not be influenced or affected by virtue of a person’s race (including traits associated with race, including hair texture and protective hairstyles such as afros, braids, locks, and twists), religion, creed, color, national origin, ancestry, citizenship, physical disability, mental disability, medical condition, genetic information, marital status, sex, pregnancy, childbirth, breastfeeding, gender, parental status, gender identity, gender expression, sexual orientation, age, familial status, domestic violence victim status, military and veteran status, or any other characteristic protected by applicable federal, state, or local law. This policy governs all aspects of employment including, without limitation, recruiting, hiring, compensation, benefits, promotion, assignment, and dismissal. In addition, it is the Firm’s policy to provide an environment that is free of prohibited harassment of any kind including, without limitation, that which is based on sex, race, age, disability, ethnic background, or any other category protected by law (including the categories listed above). The Firm complies with federal and state disability laws and makes reasonable accommodations for applicants and employees with disabilities. If you require reasonable accommodation in completing this application, interviewing, or otherwise participating in the employee selection process, please contact View email address on click.appcast.io . Notice: We participate in E-Verify in certain Firm locations for purposes of verifying employment eligibility. It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability. #J-18808-Ljbffr