Full Stack Tech Lead

Location: Bloomfield, CA (Remote)

Fulltime Permanent

Job Description

Must Have Technical/Functional Skills

Databricks

- Python Spark

- AWS

Production experience, query optimization

- SQL Complex queries, indexing, schema design

Cloud & DevOps

- AWS Experience with EC2, S3, Lambda, Glue, EKS

- CI/CD Jenkins / GitLab / GitHub Actions

GOOD-TO-HAVE SKILLS

AI / ML Integration

- LangChain or LLM orchestration frameworks

- OpenAI API integration experience

- Prompt engineering and embeddings

- Semantic search implementation

Roles & Responsibilities

KEY RESPONSIBILITIES

Design, develop, and optimize large-scale data processing solutions using Databricks.

Build and maintain robust ETL/ELT pipelines using Python and Databricks.

Develop scalable data ingestion, transformation, and integration frameworks from multiple data sources.

Leverage Databricks features including Delta Lake, Spark SQL, Unity Catalog, Workflows, and ML capabilities.

Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.

Work with AWS services to support cloud-based data platforms and analytics workloads.

Implement data quality checks, monitoring, performance tuning, and operational support processes.

Participate in architecture discussions and contribute to best practices for data engineering and cloud adoption.

Ensure compliance with security, governance, and data management standards.

Actively participate in Agile ceremonies including sprint planning, backlog grooming, daily standups, sprint reviews, and retrospectives.

Generic Managerial Skills, If any

Collaboration

- Translate requirements into technical designs

- Work with product, data, and AI teams

- Communicate technical decisions to stakeholders