Lead Data Engineer

Job ID: SRJ_8546Posted: Location: San Diego, CASalary: USD 125-130 / Yearly (Full Time) | USD 65-70 / Hourly (C2C)Employment Type: C2C, Full TimeIndustry: Computer SoftwareClient: WiproContact: Meghana GorusuCompany: SRI Tech SolutionsRole:Lead Data Engineer(Priority 1)Location:San Diego( Onsite only )Opening -1Lead Data EngineerJD:Role SummaryWe are seeking a highly skilledLead Data Engineerwith deep, hands-on experience in building large-scale data ingestion pipelines, real-time streaming solutions, and high-performance data processing systems. The ideal candidate excels at writing clean, efficient code, refactoring complex systems, improving scalability and performance, addressing production issues, and delivering reliable data solutions across cloud platforms such asAWS, Azure, or GCP.Key ResponsibilitiesDesign, build, and maintainhigh-volume data ingestion and processing pipelinesfor batch and real-time workloads.Implement and optimizereal-time streaming pipelinesusing platforms such asKafka.Develop scalable data solutions usingDatabricks, PySpark, Python, and SQL.Performcore refactoringto modernize and optimize existing pipelines and data services.Build robust, fault-tolerant pipelines capable oflarge-scale, high-throughputdata processing.Writeunit tests, automate validation, and ensure high code quality and reliability.Integrate pipelines intoCI/CD workflowsto streamline and automate deployment processes.Identify, troubleshoot, and fixproduction issues, ensuring system reliability and stability.Addressperformance bottlenecksand implement improvements for scalability, throughput, and efficiency.Work extensively acrossAWS, Azure, or GCPcloud environments and cloud-native data services.Design and orchestrate end-to-end pipelines using workflow and orchestration tools.Collaborate withData Scientists and BI Engineersto deliver clean, analytics-ready datasets.Communicate complex technical topics clearly to non-technical stakeholders.Required Qualifications8+ yearsof hands-on data engineering experience building and maintaining large-scale data systems.Proven experience withhigh-volume data ingestion, ETL/ELT, and real-time data processing.Strong expertise withKafkaor similar streaming technologies.Advanced proficiency inDatabricks, PySpark, Python, and SQL.Experience withcore refactoring, improving code maintainability and system performance.Demonstrated ability to design and buildscalable, low-latency data pipelines.Strong skills indebugging, performance optimization, and pipeline tuning.Hands-on experience with at least one major cloud platform:AWS, Azure, or GCP.Experience writingunit testsand integrating solutions withCI/CD pipelines.Strong problem-solving skills and excellent communication abilities.Qualification:BE/BTech/MS in Computer Science or related fieldSRI Tech Solutions is an equal opportunity employer and does not discriminate on the basis of race, color, gender, religion, age, sexual orientation, national origin or citizenship status or ethnic origin, disability, marital status, veteran status, or any other occupationally irrelevant criteria.