Cloud Data Engineer

OverviewAmTrust is seeking a highly motivated and accomplished Cloud Data Engineer with expertise in Azure Databricks, Azure Synapse Analytics, SQL, and cloud-based data platforms. The ideal candidate will have experience designing, developing, and optimizing scalable data lakehouse and data warehousing solutions that support enterprise reporting, analytics, and business intelligence. Strong technical skills in Python, PySpark, SQL, T-SQL, and Scala are required, along with experience building high-volume ETL/ELT pipelines, developing highly scalable and reusable Databricks notebooks, and leveraging Azure DevOps, Git, CI/CD, and cloud automation best practices to deliver secure and efficient data solutions. The successful candidate will also support AI and machine learning initiatives, including data preparation, feature engineering, Generative AI integrations, and MLOps processes. ResponsibilitiesDesign, develop, and maintain scalable cloud-based data engineering solutions, including ETL/ELT pipelines, data integration processes, reports, and analytics workflows using technologies such as Azure Databricks, Azure Synapse Analytics, Azure Data Factory, and SQL. Demonstrate expert knowledge of modern cloud architecture, data lakehouse design, and data warehousing best practices, evaluating technical design alternatives and their business and operational impacts. Develop, optimize, and troubleshoot complex SQL, T-SQL, Spark SQL, and PySpark workloads, with a strong focus on performance tuning, scalability, reliability, and cost efficiency. Design, code, test, tune, deploy, and document new and existing data solutions, including Databricks notebooks developed in Python, PySpark, Spark SQL, and Scala, as well as cloud-native applications, APIs, and data pipelines utilizing Git, CI/CD, and Azure DevOps best practices. Manage the complete development lifecycle for complex, high-impact data initiatives, including requirements gathering, solution design, development, testing, deployment, production support, and continuous improvement. Support and enable AI, machine learning, and Generative AI initiatives by developing curated datasets, feature engineering pipelines, and scalable data platforms that power advanced analytics and intelligent business solutions. Stay current with emerging trends and technologies in cloud computing, data engineering, artificial intelligence, Databricks, Azure services, automation, and analytics, recommending innovative solutions that drive business value and operational excellence. Collaborate effectively with business stakeholders, architects, developers, and leadership teams, leveraging exceptional verbal and written communication skills to translate business requirements into scalable technical solutions and communicate complex concepts to both technical and non-technical audiences. Perform other functionally related duties and responsibilities as assigned. Qualifications7+ years of experience in cloud data engineering, ETL/ELT development, data warehousing, and enterprise analytics solutions. Strong hands-on experience with Azure Databricks, Azure Synapse Analytics, Azure Data Factory, and modern cloud-based data lakehouse architectures. Proven experience implementing the Medallion Architecture (Bronze, Silver, and Gold layers) to support scalable, governed, and high-quality data pipelines for enterprise analytics and reporting. Advanced proficiency in Python, PySpark, Spark SQL, SQL, T-SQL, and Scala, with the ability to develop, optimize, and support highly scalable Databricks notebooks and distributed data processing workloads. Proven experience designing, developing, and tuning complex queries and large-scale data processing solutions across high-volume datasets. Experience implementing CI/CD pipelines, Git-based source control, Azure DevOps, and automated deployment frameworks. Strong understanding of data warehousing concepts, including dimensional modeling, Star and Snowflake schemas, data governance, metadata management, and data quality best practices. Experience supporting AI, Machine Learning, and Generative AI initiatives, including data preparation, feature engineering, model integration, vectorized data architectures, and MLOps processes. Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, Statistics, or a related technical discipline, or equivalent work experience. Strong analytical and problem-solving skills with a demonstrated ability to design innovative solutions for complex business and technical challenges. Experience working with large datasets, data cleansing, data transformation, reporting, statistical analysis, and advanced analytics. Exceptional verbal and written communication skills, with the ability to communicate effectively with technical and non-technical stakeholders, lead requirements gathering sessions, produce technical documentation, and present solutions to leadership audiences. Proven ability to work independently while collaborating effectively in a fast-paced, team-oriented environment. Strong commitment to continuous learning, innovation, and staying current with emerging cloud, data engineering, AI, and analytics technologies. Preferred: Experience in the Property & Casualty Insurance, Financial Services, Accounting, or Actuarial domains. Microsoft certifications such as Azure Data Engineer Associate (DP-203), Azure AI Engineer Associate, Databricks Data Engineer Associate/Professional, or related cloud certifications. This job description is designed to provide a general overview of the requirements of the job and does not entail a comprehensive listing of all activities, duties, or responsibilities that will be required in this position. #LI-BL1What We OfferAmTrust Financial Services offers a competitive compensation package and excellent career advancement opportunities. Our benefits include: Medical & Dental Plans, Life Insurance, including eligible spouses & children, Health Care Flexible Spending, Dependent Care, 401k Savings Plans, Paid Time Off.AmTrust strives to create a diverse and inclusive culture where thoughts and ideas of all employees are appreciated and respected. This concept encompasses but is not limited to human differences with regard to race, ethnicity, gender, sexual orientation, culture, religion or disabilities.AmTrust values excellence and recognizes that by embracing the diverse backgrounds, skills, and perspectives of its workforce, it will sustain a competitive advantage and remain an employer of choice. Diversity is a business imperative, enabling us to attract, retain and develop the best talent available. We see diversity as more than just policies and practices. It is an integral part of who we are as a company, how we operate and how we see our future.Job SummaryRequisition ID: JR1006685Category: Information TechnologyPosition Type: Regular Full-Time