Lead Data Platform Engineer

Lead Data Platform Engineer

We are seeking an exceptionally skilled and motivated Lead Data Platform Engineer, to guide our team in building robust, scalable, and high-performance data solutions primarily within the Microsoft SQL Server ecosystem. The ideal candidate will combine deep technical expertise in SQL Server development and database administration, with significant proven experience in managing Very Large Databases (VLDBs), along with leadership abilities to drive innovation, mentor junior talent, and ensure the delivery of high-quality data products that drive critical business insights.

Responsibilities:

• Very Large Database (VLDB) Management: Managing, developing, and optimizing solutions for very large databases, specifically within SQL Server version 19 and above, ensuring high performance and reliability.
• Performance Optimization: Leading performance tuning efforts for SQL Server databases, including comprehensive query optimization, advanced indexing strategies, and overall database performance.
• Collaboration with DBA Team: Working closely with the dedicated DBA team to ensure adherence to best practices and operational excellence for VLDB environments.
• Data Loading & ETL Leadership: Overseeing and guiding robust data loading processes and ETL development, primarily utilizing Microsoft BI tools, to establish efficient data pipelines for large datasets. Mentoring other ETL developers in best practices.
• Data Architecture & Modeling: Applying advanced data modeling principles and driving data architecture decisions for data warehouses, ensuring data integrity, scalability, security, and optimal performance for VLDBs.
• Code Quality & Best Practices: Conducting rigorous code reviews for SQL (including stored procedures) to ensure quality, performance, and adherence to DBA best practices, particularly for large-scale operations.
• Database Security & Access Management: Maintaining and enhancing existing entitlement and permission models to ensure robust database security, including the implementation of database encryption methods such as Transparent Data Encryption (TDE).
• AI-Enhanced Development & Optimization: Leveraging AI tools to assist in SQL code refactoring, identifying performance bottlenecks, suggesting optimization strategies, and enhancing overall developer productivity.
• Mentorship: Serving as an advisor and coach to mid-level developers and analysts, including work allocation.
• Risk Assessment: Appropriately assessing risk in business decisions, ensuring compliance with laws, rules, and regulations, adhering to policy, applying ethical judgment, and escalating/managing control issues with transparency.

Recommended Qualifications:

• 6+ years of relevant experience in Apps Development or systems analysis with extensive SQL Server expertise.
• Very Large Database (VLDB) Expertise: Proven and extensive experience in developing, maintaining, and optimizing solutions for very large databases (VLDBs) in SQL Server version 2019 and above.
• Performance Tuning Expertise: Demonstrated ability in performance tuning for SQL Server, including expert-level skills in query optimization, indexing strategies, and overall database performance enhancements.
• Data Modeling & Architecture: Deep understanding and practical experience in advanced data modeling principles, data architecture, and data warehouse concepts and design, ensuring data integrity, scalability, security, and optimal performance for VLDBs.
• Data Loading & Microsoft BI & ETL Architecture: Extensive experience with data loading processes and ETL development, primarily utilizing Microsoft BI tools (SSIS, SSAS). Proven ability to architect, design, and implement highly scalable and robust ETL pipelines for large data volumes.
• SQL Development Best Practices: Strong experience in SQL code reviews, fine-tuning stored procedures, and inculcating DBA best practices within development teams.
• Experience with NoSQL databases such as MongoDB or CouchDB.
• Database Security: Proven experience in managing database security, including maintaining and enhancing existing entitlement and permission models. Familiarity with database encryption technologies, specifically Transparent Data Encryption (TDE).
• AI-Powered Development: Ability to leverage AI tools for efficient SQL code refactoring, optimization, and identifying potential code improvements, thereby enhancing code quality and developer productivity within a SQL Server context.
• Project Management & Leadership: Experience in managing and implementing successful projects, with demonstrated leadership and project management skills.
• Communication: Consistently demonstrates clear and concise written and verbal communication.
• Nice to have: Python/Pyspark skills for data processing and analytics.

Education:

• Bachelor's degree/University degree or equivalent experience
• Master's degree preferred