Tech Lead - Software Engineer

Black Rock Groups

We are seeking an experienced Senior Databricks Engineer with strong expertise in Databricks, Python, PySpark, and SQL to design, develop, and optimize large scale data processing solutions. The ideal candidate will have hands on experience building data pipelines, implementing ETL/ELT processes, and working with cloud based data platforms to support analytics and business intelligence initiatives.

Key Responsibilities

Design, develop, and maintain scalable data pipelines using Databricks, PySpark, Python, and SQL .
Build and optimize ETL/ELT frameworks for processing large volumes of structured and unstructured data.
Develop data ingestion solutions from multiple source systems into cloud data lakes and warehouses.
Optimize Spark jobs for performance, scalability, and cost efficiency.
Collaborate with data architects, analysts, and business stakeholders to understand data requirements.
Implement data quality, validation, and monitoring frameworks.
Perform data modeling and transformation for reporting and analytics use cases.
Troubleshoot production issues and provide timely resolutions.
Follow best practices for coding, testing, version control, and deployment.

Required Skills

8+ years of experience in Data Engineering.
Strong hands on experience with Databricks .
Expertise in PySpark, Python, and SQL .
Experience with Spark optimization techniques and performance tuning.
Strong understanding of data warehousing concepts and dimensional modeling.
Experience building ETL/ELT pipelines.
Knowledge of Delta Lake, Spark SQL, and Databricks workflows.
Experience with Git, CI/CD, and Agile methodologies.
Strong analytical and problem solving skills.

Preferred Skills

Experience with cloud platforms such as Azure, AWS, or GCP .
Knowledge of Azure Data Factory (ADF) , Airflow, or other orchestration tools.
Experience with streaming technologies such as Kafka or Structured Streaming.
Exposure to Power BI, Tableau, or other analytics tools.
Databricks or cloud certifications are a plus.

Qualifications

Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Excellent communication and stakeholder management skills

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Tech Lead - Software Engineer in Santa Clara, CA vacancy

Tech Lead Software Engineer, Fleet Connectivity
$193.93k - $291.15k
...Fidelity, T. Rowe Price, and other leading investors.About the RoleAs... ...of fault-tolerant software across both cellular and Wi-Fi... ...cohesive system designs, mentor engineers, and shape the long-term architectural... ...track record of acting as a Tech Lead on complex software...
Suggested
Immediate start
Remote work
Flexible hours
Nuro
Mountain View, CA
2 days ago
Technical Lead, AI-Native Software Engineering (hybrid) - 2025652
$171.6k - $245k
The application window is expected to close on: 10/29/2026Technical Lead, AI-Native Software Engineering (hybrid) - 2025652*hybrid role, some onsite activity required Meet the TeamThe Security Workflow Engineering team, as part of the Security & Trust Organization, is...
Suggested
Full time
Temporary work
Local area
Flexible hours
CISCO Systems
San Jose, CA
3 days ago
Tech Lead of Software Engineering
As part of the Platform Engineering team, you will help build DataVisor... ...before they impact delivery Lead design and architecture... ...and Shell scripting8+ years of software development experienceHands-on... ...technical leadership experience as a tech lead, staff engineer,...
Suggested
DataVisor
Mountain View, CA
1 day ago
Senior/Lead Software Engineer, Layer1
$140k - $200k
...advancements in cloud computing, artificial intelligence, and software-defined networking to provide our clients with a competitive edge... ...has earned us several prestigious awards, such as Best Engineering Team, Best Company for Diversity, Compensation, and Work-Life...
Suggested
Arista Networks
Santa Clara, CA
1 day ago
Lead Software Engineer- Englewood, CO or Sunnyvale, CA- ONSITE 4X Weely- FreeWheel
$160.11k - $240.16k
...company, we have offices in nine countries and can insert advertisements around the world.Job SummaryFreeWheel is seeking a Lead Software Engineer (ENG 5) to serve as a principal-level individual contributor driving the technical direction of our Buyer Cloud Ad Serving...
Suggested
Full time
Night shift
Weekend work
Comcast
Sunnyvale, CA
3 days ago
Lead Software Engineer
Role OverviewSupport the Search Engineering team by managing vulnerability lifecycles, ensuring secure architectural practices, and overseeing automated fix validations.Key Responsibilities :Threat Modeling & Asset Profiling: Document trust boundaries, data flows, and architectural...
Virtusa
San Jose, CA
8 hours ago
Lead Software Engineer
$135k - $165k
...platform services and shared infrastructure that power the technology ecosystem across the organization.About the RoleThe Lead Software Engineer will be responsible for leading the design, development, and delivery of enterprise-grade, scalable platform services and APIs...
H1b
Work at office
Local area
Home office
Relocation package
Monday to Thursday
Williams-Sonoma
San Jose, CA
1 day ago
Linux Kernel Senior/Lead Software Engineer
$140k - $210k
...advancements in cloud computing, artificial intelligence, and software-defined networking to provide our clients with a competitive edge... ...has earned us several prestigious awards, such as Best Engineering Team, Best Company for Diversity, Compensation, and Work-Life...
Arista Networks
Santa Clara, CA
3 days ago
Lead Software Engineer
$126.5k - $223.9k
We are the leading provider of professional services to the middle market globally, our purpose is to instill confidence in a world... ...like you and that’s why there’s nowhere like RSM.The Lead Software Engineer is a hands-on technical leader responsible for designing and...
Full time
Work experience placement
Internship
Local area
RSM International
San Jose, CA
4 days ago
PCIe Senior/Lead Software Engineer
$140k - $210k
...advancements in cloud computing, artificial intelligence, and software-defined networking to provide our clients with a competitive edge... ...has earned us several prestigious awards, such as Best Engineering Team, Best Company for Diversity, Compensation, and Work-Life...
Arista Networks
Santa Clara, CA
15 hours ago
Senior Software Engineer - Tech Lead (Memory Test, San Jose, CA)
$177.9k - $284.7k
...business results.Opportunity OverviewMemorty Test is looking for software engineers to join an exciting, dynamic, hardworking, engaging, and... ...history of consistent on time software delivery.Experience as a Tech LeadMastery implementing software that directly interacts...
Local area
Relocation
Flexible hours
Universal Robots
San Jose, CA
2 days ago
Software Engineer & Technical Lead
...Full Time)Employment Type: Full TimeIndustry: Computer SoftwareClient: WiproContact: Meghana GorusuCompany: SRI Tech SolutionsSoftware Engineer & Technical Lead ( two requirements with same JDSunnyvale, CAEx A*** will be preferredSubmit with skill matrix pleaseSoftware...
Full time
SRI Tech
Sunnyvale, CA
1 day ago
BIOS Senior/Lead Software Engineer
$140k - $210k
...advancements in cloud computing, artificial intelligence, and software-defined networking to provide our clients with a competitive edge... ...has earned us several prestigious awards, such as Best Engineering Team, Best Company for Diversity, Compensation, and Work-Life...
Arista Networks
Santa Clara, CA
3 days ago
Staff Software Engineer, Tech Lead, Infrastructure
$207k - $300k
Lead the technical strategy, architectural design, and implementation of next-generation... ...the organization, setting rigorous engineering standards, driving best practices for system... ...of experience testing, and launching software products.5 years of experience building...
Google
Sunnyvale, CA
4 days ago
Lead Software Engineer, ML Network Stack - Annapurna Labs
$193.3k - $261.5k
We are seeking an experienced engineer and technical leader to join... ...AWS that develops hardware and software components that are critical... ...highly regarded. You'll be leading senior, mid-level, and junior... ...experience- Experience as a mentor, tech lead or leading an...
Local area
Flexible hours
Amazon
Cupertino, CA
4 days ago
Lead Principal Systems Software Engineer
$146.3k - $306.4k
Defines architecture for large-scale systems software, firmware integration, and fleet... ...performance, and cost at hyperscale. Champions engineering excellence: coding standards, threat... ...management, concurrency, and fault isolation. Leads multi-quarter initiatives, from...
Temporary work
Flexible hours
Shift work
Oracle Corporation
Santa Clara, CA
2 days ago
Director, Technical Lead Forward Deployed Engineer
$204.73k - $370.53k
...opportunities, a world-class training facility, and leading market tools, we help our people... ..., Technical Lead Forward Deployed Engineer to join our Advisory Services practice.Responsibilities... ...shipping end-to-end production secure software systems, with a focus on data, machine...
H1b
Local area
KPMG
Santa Clara, CA
4 days ago
Dynamics and Technical Lead - Acoustics Mechanical Engineering
$50 per hour
...models generated from both in-house and commercially available software codes, that simulate mechanical loading conditions, mechanism... ...to support peer and customer reviews, present analysis data to engineering teams, customer technical representatives, and Program Office...
Full time
Temporary work
Work experience placement
Casual work
Work at office
Flexible hours
Lockheed Martin
Sunnyvale, CA
4 days ago
Lead Software Engineer AI for Hardware Verification
$119k - $221k
...Cadence has been nominated as a Great Place to Work globally and is also a Fortune 100 Best Companies to Work For . As a Lead Software Engineer you will be part of the HSV Software team in the SVG group in San Jose California. The HSV Software team builds AI-...
Full time
Cadence Systems
San Jose, CA
6 days ago
Lead Systems Software Test Engineer - CSP Engagements
$184k - $287.5k
NVIDIA is seeking a Senior Systems Software test (lead) Engineer to join our Cloud Service Provider (CSP) Engagements team, focusing on ML software stack validation for Datacenter products such as GB200 and Vera Rubin. This role combines deep technical expertise from cluster...
Full time
Local area
Nvidia
Santa Clara, CA
15 hours ago
Software Security Lead - Principal Software Engineer - S3E
$107.5k - $204.5k
...complex problems. With our three market leading businesses, world-class operations and investments... ...100 years of experience and renowned engineering expertise to meet the needs of today's... ...and defense. Job Summary : The Software Security Sensors and Effectors...
Temporary work
Work experience placement
Interim role
Work at office
Remote work
Flexible hours
Raytheon
San Jose, CA
2 days ago
Senior/Lead Software Engineer - (Network Systems)
$140k - $210k
...computing, artificial intelligence, and software-defined networking to provide our clients... ...prestigious awards, such as Best Engineering Team, Best Company for Diversity, Compensation... ...Networks is looking for world-class Senior/Lead Network Systems software engineers....
Arista Networks
Santa Clara, CA
15 hours ago
Lead Principal Software Development Engineer
$96.8k - $251.6k
Job Title Sr. Principal Software Development Engineer — OCI Core Platform (OCI Key Management Service) Overview As a senior engineer in the Oracle... ...that meet rigorous enterprise and compliance requirements. Lead the design and development of new features and...
Temporary work
Flexible hours
Oracle
Santa Clara, CA
5 days ago
Staff Software Engineer, Tech Lead, CCME
...decision-making team, serving as its primary tech lead and guiding architecture, system... ...and planning partners, mentor and develop engineers, and improve engineering practices, development... ..., five years testing and launching software products, and three years in software...
Google
Sunnyvale, CA
3 days ago
Strategic Technical Recruiting Lead - Engineering & GTM
Orkes, based in Santa Clara, CA, is seeking a Lead Technical Recruiter for Engineering and GTM to partner with HR and executive leadership. This strategic, hands-on role owns full-cycle recruiting for technical and GTM functions in a fast-growing startup. You will build...
Orkes
Santa Clara, CA
5 days ago
Lead Principal Platform Software Engineer
$96.8k - $306.4k
...all. Discover your potential at a company leading the way in AI and cloud solutions that... ...Level - IC5Key ResponsibilitiesPlatform Software Development:Set group-wide guardrails for... ...software development lifecycle; coaches engineers across teams or units to drive...
Temporary work
Flexible hours
Shift work
Oracle Corporation
Santa Clara, CA
3 days ago
Lead Principal Platform Software Engineer
$135.2k - $306.4k
...networking platforms. We're looking for engineers with strong computer science fundamentals... .... Discover your potential at a company leading the way in AI and cloud solutions that impact... ...a Lead Principal Platform Software Engineer, you will:Lead the architecture...
Temporary work
Flexible hours
Oracle Corporation
Santa Clara, CA
2 days ago
Technical Lead Engineer — Power Validation & Characterization
...build and maintain extensive lab automation to do it at volume and velocity. Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience 12+ years of relevant industry experience Demonstrated hands-on validation...
Jobleads-US
Sunnyvale, CA
4 days ago
Technical Lead, Hardware Engineering, Google Cloud
$188k - $274k
...technical leadership within the Hardware Engineering team to identify, scope, and deliver... ...technical team. As the Technical Engineering Lead in Hardware Engineering, you will... ...-tier electrical, mechanical, firmware, software, and thermal engineers. You will drive an...
Google
Sunnyvale, CA
4 days ago
Technical Lead Engineer — Network Validation & Characterization
...build and maintain extensive lab automation to do it at volume and velocity. Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience 12+ years of relevant industry experience Demonstrated hands-on validation...
Jobleads-US
Sunnyvale, CA
4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Tech Lead - Software Engineer. Be the first to apply!