AWS Databricks Architect (AWS, Databricks, Apache Spark Specialist)

AWS Databricks Architect (AWS, Databricks, Apache Spark Specialist)

Data Architect II

Responsibilities

• Engage with business and technology stakeholders to define project scope, goals, and architectural direction.
• Architect and implement end-to-end Databricks solutions on AWS, leveraging services such as S3, Glue, Lambda, Kinesis, EMR, Redshift, IAM, and Step Functions.
• Design and manage lakehouse architectures optimized for scalability, performance, governance, and cost‑efficiency.
• Oversee the design and development of scalable ETL/ELT pipelines using Databricks and Apache Spark.
• Lead solution design sessions, conduct code reviews, and ensure adherence to data engineering best practices.
• Create and deliver technical presentations, architecture documents, and case studies for internal and external stakeholders.
• Stay updated on Databricks platform enhancements, AWS data services, and emerging trends in data engineering, AI, and ML.
• Provide technical leadership and mentorship to engineering teams across multiple projects and client engagements.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or a related field.
• 10+ years of experience in data engineering with significant hands‑on expertise in Databricks and Apache Spark.
• Strong end‑to‑end experience with the AWS data ecosystem, including S3, Glue, Kinesis, Lambda, Redshift, DynamoDB, IAM, and Step Functions.
• Extensive experience building and managing ETL/ELT pipelines at scale.
• Proficiency in PySpark, SQL, and Python.
• Solid understanding of data warehousing, data lakes, and lakehouse architectures.
• Strong communication skills, with the ability to simplify and present complex technical topics.
• Proven ability to create detailed architecture documents and technical presentations.
• Willingness to travel based on project or client needs.
• Databricks certifications (e.g., Associate Data Engineer, Professional Data Engineer).
• AWS certifications highly preferred (e.g., Solutions Architect, Data Analytics Specialty).
• Exposure to Azure and GCP cloud platforms is advantageous.
• Experience with CI/CD (e.g., AWS CodePipeline, GitHub Actions), infrastructure‑as‑code (Terraform or CloudFormation), and DevOps practices.
• Previous technical leadership or data architecture experience in consulting or multi‑client environments.

Compensation and Location

Remote. Compensation range: $161,000 – $242,000 per year.

Benefits

Full‑time employees accrue a minimum of 10 days of paid vacation per year, 6 days of paid sick leave, 10 paid holidays, and eligibility for a 401(k) Retirement Plan with employer matching. Medical, dental, and vision insurance, as well as basic life, accidental death and disability, and short‑ and long‑term disability benefits are available. Additional voluntary benefits such as an HSA and FSA may be purchased.

Equal Employment Opportunity Statement

UST is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other applicable characteristics protected by law. We will consider qualified applicants with arrest or conviction records in accordance with state and local laws and "fair chance" ordinances.