Databricks Engineer

Role Description

We are seeking a highly skilled Databricks Engineer to join our team. This role will be hands-on, working closely with architects, data scientists, and customers to build, optimize, and deploy high-performance data and AI solutions.

• Build and optimize data pipelines, implementing data processing frameworks, and enabling AI/ML solutions within Databricks.
• Work across data ingestion, transformation, and orchestration while ensuring scalability, performance, and security.
• Collaborate with architects and business stakeholders to ensure solutions align with customer needs and best practices.

Above all, the ideal candidate embodies RevStar’s core values:

• Self-Mastery: We hold a high bar for how we think, communicate, and improve.
• Ownership: We own outcomes, not just effort.
• Shared Destiny: We rise or fall together.

Qualifications

• 3+ years of hands-on experience in data engineering, with a focus on big data processing and cloud-native architectures.
• 2+ years of hands-on experience with Databricks, including Apache Spark, Delta Lake, and MLflow.
• Databricks Certifications (Mandatory): Databricks Certified Data Engineer Associate (or higher).
• Proficiency in Python, SQL, and Spark-based frameworks.
• Experience in developing and optimizing large-scale ETL/ELT pipelines.
• Strong understanding of Lakehouse architecture and cloud-agnostic data solutions.
• Familiarity with CI/CD pipelines and Infrastructure-as-Code (IaC) for Databricks (e.g., Terraform, Databricks CLI).
• Knowledge of data governance, security, and compliance best practices.
• Experience working in Agile development environments, following DevOps/MLOps best practices.

Requirements

• Develop and optimize data pipelines using Apache Spark and Delta Lake within Databricks.
• Implement ETL/ELT workflows, ensuring efficient data ingestion, transformation, and storage.
• Design Lakehouse architecture-based solutions that scale across structured and unstructured data sources.
• Integrate Databricks with cloud storage solutions (Azure Data Lake, AWS S3, Google Cloud Storage) for seamless data management.
• Optimize Spark jobs for scalability, cost efficiency, and low latency.
• Implement monitoring and alerting solutions to track job performance and detect failures.
• Develop automated data validation, testing, and quality assurance processes.
• Support ML model training and deployment within Databricks, integrating with MLflow for experiment tracking and model versioning.
• Collaborate with data scientists and ML engineers to enable scalable AI solutions.
• Implement feature engineering pipelines and integrate models into production environments.
• Ensure data security, access control, and compliance with industry standards (GDPR, HIPAA, SOC 2, etc.).
• Follow Databricks best practices for data lineage, governance, and metadata management.
• Document processes, configurations, and best practices for internal and client use.

Benefits

• Paid Time Off – Take the time you need to recharge and stay productive.
• Remote-First Working Environment – Collaborate from anywhere while staying connected with our global team.
• Comprehensive Health Coverage – Medical, Dental, Vision.
• 401(k) Retirement Plan – Plan for your future with access to a company-sponsored 401(k) program.
• Annual Learning & Development Stipend – Invest in your skills with conferences, certifications, or courses.
• Peer Mentorship & Coaching – Learn from experienced engineers, product managers, and architects to accelerate your growth.
• Professional Growth Opportunities – Exposure to cutting-edge AWS GenAI, data, and cloud technologies across diverse industries.
• Company Outings & Volunteer Opportunities – Build relationships and give back to the community.
• Collaborative, Innovative Culture – Work alongside top talent in a fast-paced, supportive environment that values curiosity and initiative.