Data Platform SRE & Spark Engineer

Key Responsibilities and Qualifications In-depth knowledge of the Hadoop ecosystem, including HDFS, MapReduce, Spark, Scala, Hive Expertise in Hive for data warehousing and querying large datasets Cloud composer, Big Query, Airflow, GCP hands‑on experience Strong experience with Apache Spark for distributed data processing and real‑time analytics Advanced proficiency in Scala programming for big data application development Familiarity with data modeling, ETL processes, and data integration techniques Experience with cloud platforms and big data tools (e.g., Google Big Query) Strong analytical and problem‑solving skills to address complex data challenges Basic knowledge of Java, Spring Boot Strong SQL skill Architect and design scalable big data solutions leveraging technologies like Hadoop, Hive, Spark, and Scala Develop and optimize data pipelines and workflows for processing and analyzing large‑scale datasets Manage and fine‑tune distributed computing frameworks to ensure high performance and reliability Collaborate with cross‑functional teams, including data engineers, analysts, and business stakeholders, to gather and define data requirements Implement data security, governance, and compliance measures in big data environments Diagnose and resolve technical issues in big data systems to maintain operational efficiency Base Salary Range: $100,000 to $125,000 per annum. Benefits Discretionary Annual Incentive Comprehensive medical coverage: medical & health, dental & vision, disability planning & insurance, pet insurance plans Family support: maternal & parental leaves Insurance options: auto & home insurance, identity theft protection Convenience & professional growth: commuter benefits, certification & training reimbursement Time off: vacation, time off, sick leave & holidays Legal & financial assistance: legal assistance, 401(K) plan, performance bonus, college fund, student loan refinancing #J-18808-Ljbffr Tata Consultancy Services Limited