Senior Specialist - Architecture

Job Title: Databricks ArchitectWork LocationDallas TXJob Description: Seeking a Data Architect Specialist with 5 to 7 years of experience in Databricks and Data Architecture to design and implement scalable cloud native data solutionsDesign develop and optimize scalable data pipelines using Databricks within the Azure Data Factory ecosystem Collaborate with data engineers and architects to build cloud native data solutions leveraging Databricks and Python scripting Implement data transformation cleansing and integration workflows to support business intelligence and analytics needs Ensure data quality reliability and performance of data processing jobs on Databricks clusters Utilize Python for automation scripting and enhancing data workflows on Databricks Work with Azure Data Factory to orchestrate and monitor end-to-end data workflows Participate in the evaluation and adoption of new tools and technologies to improve data platform capabilities Maintain comprehensive documentation of data pipelines processes and architecture for knowledge sharing and compliance Apply Data Architecture principles to design scalable maintainable and efficient data solutions Possess knowledge of Data Lakehouse Architecture and Cloud Solution Architecture to enhance data platform design and integration Roles and Responsibilities Lead the design and development of scalable ETLELT pipelines using Databricks and Azure Data Factory Optimize Spark jobs and Databricks clusters for maximum performance and cost efficiency Collaborate with cross functional teams including data scientists analysts and infrastructure teams to deliver robust data solutions Troubleshoot and resolve data pipeline issues to ensure uninterrupted data flow and system reliability Develop and enforce best practices for data security governance and compliance within the data platform Mentor junior team members on Databricks technologies and Data Architecture best practices Drive continuous improvement initiatives for data engineering processes automation and platform scalability Stay updated with the latest trends and advancements in Databricks Data Architecture and related cloud technologies