Senior Databricks Architect

Job Title: Senior Databricks Architect
Location: 100% Remote
Duration of project: 6+ Months

Required Visa: Any visa or authorization (Except H1 and CPT)

Job Overview:

We are seeking a highly skilled Senior Databricks Architect/Engineer to lead the design, development, and implementation of a next-generation data platform. This role will be instrumental in building a scalable Databricks-based solution to support treasury functions such as cash management, liquidity, forecasting, and payments, while enabling future BI and AI capabilities.

Key Responsibilities:

Design and architect a scalable Databricks-based data platform from the ground up
Perform hands-on development using Databricks, PySpark, and SQL
Build and optimize data pipelines to ingest data from Azure/SQL Server environments
Implement data architecture frameworks (e.g., medallion architecture)
Configure and optimize Databricks clusters for performance and cost efficiency
Collaborate with stakeholders to gather requirements and translate them into technical solutions
Provide technical leadership and guidance to development teams
Ensure platform stability through monitoring, optimization, and best practices
Support integration with downstream tools for BI and AI use cases

Required Skills & Experience:

Strong hands-on experience with Databricks (Azure preferred)
Proficiency in PySpark, SQL, and data engineering concepts
Experience designing and implementing data platforms and architectures
Solid understanding of Azure services (ADF, ADLS, SQL Server, etc.)
Experience with data pipeline frameworks and ETL/ELT processes
Knowledge of cluster management, performance tuning, and optimization
Strong communication and stakeholder management skills
Ability to work independently in a remote environment
Experience in financial or treasury data domains (cash, liquidity, payments)
Familiarity with Delta Lake, Unity Catalog, and data governance practices
Exposure to BI tools and AI/ML integration