Databricks Engineer
iLink Multitech Solutions
Databricks Engineer
Role: Databricks Engineer Experience 4–8 years Location As per business requirement Employment Type Full-time
Job Summary: We are looking for a skilled Databricks Engineer with strong expertise in designing, developing, and optimizing modern data engineering solutions on the Databricks Lakehouse Platform. The ideal candidate should have experience building scalable ETL/ELT pipelines, working with large-scale data, and leveraging Apache Spark to deliver high-performance data solutions.
Key Responsibilities:
- Design, develop, and maintain scalable data pipelines using Databricks.
- Build ETL/ELT workflows for batch and streaming data processing.
- Develop solutions using PySpark, Spark SQL, and Delta Lake.
- Implement Medallion Architecture (Bronze, Silver, Gold) for data transformation.
- Integrate data from various sources including relational databases, APIs, cloud storage, and streaming platforms.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Collaborate with Data Architects, Data Scientists, BI developers, and business stakeholders.
- Implement CI/CD pipelines and deployment automation for Databricks workloads.
- Ensure data quality, security, governance, and compliance.
- Monitor, troubleshoot, and optimize production data pipelines.
- Document technical solutions and follow engineering best practices.
Required Skills:
- Core Technologies: Databricks Lakehouse Platform, Apache Spark, PySpark, Spark SQL, Delta Lake, Python, SQL, Cloud Platforms (one or more) Microsoft Azure (preferred) AWS Google Cloud Platform Azure Technologies (Preferred) Azure Data Factory (ADF) Azure Data Lake Storage (ADLS Gen2) Azure Synapse Analytics Azure Key Vault Azure DevOps
- Data Engineering Data Warehousing Data Modeling ETL/ELT Development Batch Processing Streaming (Kafka/Event Hubs) Data Lake Architecture DevOps & Version Control Git Azure DevOps / GitHub CI/CD Pipelines
- Preferred Qualifications: Experience with Unity Catalog. Knowledge of Databricks Workflows and Jobs. Hands-on experience with Delta Live Tables (DLT). Exposure to MLflow is an added advantage. Experience with data governance and security best practices. Familiarity with Infrastructure as Code (Terraform) is a plus.
- Educational Qualification: Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
- Preferred Certifications: Databricks Certified Data Engineer Associate Databricks Certified Data Engineer Professional Microsoft Certified: Azure Data Engineer Associate (DP-203) Azure Fundamentals (AZ-900)
Good to Have:
- Experience with real-time analytics. Knowledge of Lakehouse architecture. Experience with Agile/Scrum methodologies. Strong analytical and problem-solving skills. Excellent communication and stakeholder management abilities.
Mandatory Skills:
- Databricks PySpark Spark SQL Delta Lake Python SQL Azure/AWS/GCP (at least one cloud platform) ETL/ELT Development Data Lake Architecture
Nice to Have:
- Unity Catalog Delta Live Tables (DLT) MLflow Kafka/Event Hubs Azure Data Factory Terraform Azure DevOps/GitHub Actions
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Databricks Engineer in Milpitas, CA vacancy
- ...experience in SQL is a MUST. 8+ years of experience in modern data engineering/data warehousing/data lakes technologies on Azure, Data... ...Google Big Query etc. Strong experience with big data tools(Databricks , Spark etc..) and programming skills in PySpark and Spark SQL...Databricks
Blue Rose Technologies
Fremont, CA1 day ago $168k - $258.75k
...Business Insights team is looking for a Customer Success Insights Engineer to scale platform adoption through data and AI-native... ...enterprise systems, vendor APIs, and public developer platforms into Databricks, with data quality gates, freshness monitoring, and fail-safe...DatabricksFull time$135.91k - $223.29k
Role SummaryWe are looking for a Senior AI Engineer to lead the transformation of our marketing ecosystem. You won’t just be maintaining... ...: Mastery of SQL and cloud data warehouses. Experience with Databricks for feature engineering is a huge plus.Generative AI: Proven...DatabricksTemporary workRelocation packageFlexible hours- NVIDIA is seeking a Customer Success Insights Engineer to scale platform adoption using data and AI-native solutions in Santa Clara.... ...Ideal candidates have 8+ years in data/analytics roles, deep Databricks and LLM experience, and a strong record of executive analytics...Databricks
- NVIDIA is seeking a Customer Success Insights Engineer to scale platform adoption through data- and AI-native analytics. You will design data pipelines into Databricks, implement agentic AI workflows, and build dashboards that measure adoption and ecosystem health. The...Databricks
$165k - $190k
...momentum, a growing partner ecosystem including SentinelOne, Databricks, and Google Cloud, and a major fundraise ahead, Obsidian is scaling... .../ SRE TeamThe DevOps/SRE team at Obsidian ensures that engineering excellence translates into stable, scalable, and high-performing...DatabricksWork from home- ...characterization at IC and PCBA levels Provide early feedback to design engineers regarding design decisions and testability Develop and... ...with low-level firmware and C programming Experience in Databricks or similar environment Experience with automated test equipment...DatabricksWork experience placement
$217k - $288.4k
...At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection to cancer... ...Data and AI platform in the world. Security Foundation Engineering is a cornerstone of this mission in protecting our platform and...DatabricksWorldwideMenlo Ventures
Mountain View, CA1 day ago- ...data solutions using Azure cloud data technologies. As a Data Engineering Consultant, you will collect, aggregate, store, and reconcile... ...and batch integrations using tools/frameworks like Azure Databricks, Azure Data Factory, Spark, Spark Streaming, etc. Understanding...Databricks
- ...Job Description: Senior Data Engineer responsible for designing, developing, and supporting scalable data pipelines in Databricks and AWS for supply chain, capacity planning, and global sourcing use cases. Expertise includes onboarding new feeds, schema...Databricks
Trinus
Santa Clara, CA2 days ago - ...Site Reliability Engineer III There's nothing more exciting than being at the center of a rapidly growing field in technology and... ...practices within your team, including modern technologies such as Databricks, Snowflake, AWS, and Kubernetes Collaborates with other...DatabricksWork at office
Chase
Palo Alto, CA1 day ago - ...ETL pipelines, and integrating AI features into reporting and analytics processes. You will develop data models, implement security, and manage Databricks-based dashboards while optimizing DAX and Python scripts for automation and reporting tasks. #J-18808-Ljbffr AdobeDatabricks
$85.3k - $186.2k
Adobe is seeking an Advanced Analytics & Reporting Analyst 3 to build and maintain full BI solutions utilizing Power BI and Databricks. This role involves developing data models and ETL pipelines, optimizing DAX for performance, and integrating AI features into analytics...Databricks$168k - $270.25k
...the Chief Security Officer is looking for an AI and Automation Engineer to help us create something new — an AI-native, agent-enabled... ...MCP patterns.Prior experience with AWS, Terraform, Airflow, and Databricks or equivalent large-scale data platforms.Strong ownership,...DatabricksFull timeWork experience placementWork at officeRemote work$192k - $260k
...company in Mountain View, California, is seeking a Staff Software Engineer to work on distributed data systems. The ideal candidate will... ...value. It offers a competitive salary range of $192,000—$260,000 USD along with comprehensive benefits. #J-18808-Ljbffr DatabricksDatabricks$66.52 - $88.14 per hour
...We are seeking a highly experienced and strategic Senior Data Engineer to lead the advancement of our enterprise data governance capabilities... ...governance infrastructure across key platforms, including Databricks Unity Catalog, to ensure seamless metadata management, data...DatabricksHourly payFull timeStanford Health Care
Palo Alto, CA4 days ago$217k - $288.4k
...P-1107 At Databricks, we are obsessed with enabling data teams to solve the world's toughest problems, from security threat detection... ...Data and AI platform in the world. Security Foundation Engineering is a cornerstone of this mission in protecting our platform and...DatabricksLocal areaWorldwideDatabricks
Mountain View, CA1 day ago- ...Job Description: Senior AI Search & Agentic Systems Engineer Onsite - Santa Clara, CA W2 Position Name Required... ...deployments. Exposure to enterprise data pipelines (Databricks, Hadoop, or similar). Prior experience integrating LLM-based...Databricks
Saransh Inc
Santa Clara, CA2 days ago $180k - $210k
...centrally store it, and make available to any downstream query engine and use case (from traditional analytics to real-time AI / ML).... ...EMR, Amazon Redshift, Amazon Kinesis, AWS Glue, Google BigQuery, Databricks, Snowflake, Azure HDInsight, etc. ~ Comprehensive...DatabricksWork at officeRemote workOnehouse
Sunnyvale, CA3 days ago$217k - $288.4k
Senior Engineering Manager - Trust and Safety Databricks is seeking a Senior Engineering Manager to lead Security Foundation Engineering within Trust & Safety. This role focuses on building and scaling secure frameworks, primitives, and tooling across the Databricks platform...DatabricksLocal areaWorldwide- Job-ID18347302Reference21-13197 Job Description: Design and implementation of high-performance compute clusters Solid knowledge on the HPC cluster systems, including scalable/robust storage, high-bandwidth inter-connects, CPU / GPU architecture, and a knowledge of cloud...
- Join to apply for the Junior Field Data Engineer role at Jobright.ai 1 day ago Be among the first 25 applicants Join to apply for the... ...Lakehouse implementations (ex. S3/GCS/ADLS, Snowflake and/or Databricks) • Deep experience with at least one cloud platform (AWS, GCP,...DatabricksFull timeH1bTrial period
- ...extraction from unstructured sources Experience with workflow engines, event-driven architectures, or complex business process automation... ...-on experience with cloud data platforms (e.g., Snowflake, Databricks, Redshift, Fabric/Power BI or similar) Familiarity with...DatabricksWork at office
$185k - $215k
...Description In this role, you will operate as a Staff AI Systems Engineer, setting technical direction across multiple teams,... ...orchestration tools (e.g., AWS, Azure, GCP, Airflow, Argo Workflows, Databricks). Familiarity with hardware/software testing environments, including...DatabricksOdd jobWork experience placementInternshipLocal areaWorldwide$146.9k - $183.6k
...production vehicles Analyze fleet telemetry and logs (SQL, PySpark, Databricks) to detect emerging issues, quantify scope and impact, and... ...Qualifications B.S. or M.S. in Computer Science, Computer Engineering, Electrical Engineering, or a related field - or equivalent...DatabricksFull timeLocal areaRemote work- ...the fleet flags the next one on its own. We're looking for an engineer with strong software and systems fundamentals who either brings... ...vehicles Analyze fleet telemetry and logs (SQL, PySpark, Databricks) to detect emerging issues, quantify scope and impact, and validate...DatabricksFull timeContract workLocal areaRemote work
$192k - $260k
Staff Software Engineer - Distributed Data Systems P-186 Get AI-powered advice on this job and more exclusive features. At Databricks, we are obsessed with enabling data teams to solve the world’s toughest problems, from security threat detection to cancer drug development...DatabricksWork at officeLocal area- ...enterprise-scale modernization initiatives involving Azure, Databricks, and modern Lakehouse architectures, delivering secure, governed... .... Provide technical leadership and architectural guidance to engineering teams throughout the project lifecycle. Collaborate with...Databricks
Omniskope Inc
Milpitas, CA2 days ago - ...Big Data Engineer – RemoteBright Vision Technologies is a technology consulting and software development company delivering cloud, AI... ...Hadoop on cloud platforms such as AWS EMR, Azure HDInsight, or Databricks.Familiarity with modern lakehouse formats (Delta, Iceberg,...DatabricksH1bLocal areaImmediate startRemote workVisa sponsorship
Bright Vision Technologies
Milpitas, CA2 days ago - ...high-impact, high-paying career in Data Science, Analytics, or Engineering , it's time to stop guessing and start choosing results. At... ...Data Engineering Track: Python/Java, Spark, Airflow, Databricks ETL architecture, Snowflake, S3, BigQuery, Redshift...Databricks
SynergisticIT
Alviso, CA1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Databricks Engineer. Be the first to apply!
