Spark Engineer

Must Have Technical/Functional Skill

We are looking for an experienced Apache Spark Developer with strong expertise in large-scale data processing and performance optimization. The role involves designing, building, and optimizing scalable ETL pipelines using Spark in cloud environments.

Key Skills
• Strong hands-on expertise in Apache Spark 3.5.x or later
• Solid experience with Spark DataFrames, Datasets, and RDDs
• Strong proficiency in Spark SQL
• Experience working in cloud environments (AWS and/or Azure)
• Hands-on experience with cloud storage solutions such as Amazon S3 and Azure Data Lake Storage (ADLS)
• Strong understanding of ETL pipeline design and data modeling
• Proven experience in Spark job performance tuning and query optimization

Preferred / Good to Have
• Experience with Snowflake for data warehousing and analytics
• Exposure to real-time/streaming data processing
• Familiarity with distributed systems and large-scale data architectures
• Scala Programming Language

Roles & Responsibilities
• Design, develop, and maintain large-scale batch and real-time data processing solutions using Apache Spark (v3.5.x or later)
• Work extensively with Spark DataFrames, Datasets, and RDDs for high-volume data processing
• Develop and optimize queries using Spark SQL
• Build and manage ETL pipelines to support analytics and downstream data consumption
• Perform Spark job tuning and performance optimization to improve efficiency and scalability
• Collaborate with data engineers, architects, and analytics teams to deliver end-to-end data solutions
• Ensure data quality, reliability, and adherence to best practices across pipelines

Required Qualifications
• Strong hands on experience with Spark
• In depth knowledge of Apache Spark architecture, internals, and execution model
• Experience working with Spark DataFrames, datasets, RDDs (large scale data processing - batch & real-time)
• Strong SQL skills for data analysis and transformation
• Familiarity with ETL pipelines and data modeling(Snowflake-preferrred)
• Experience with Linux/Unix environments
• Good understanding of distributed computing pri nciples

Good to Have
• Experience with Snowflake for data warehousing and analytics
• Exposure to real-time/streaming data processing
• Familiarity with distributed systems and large-scale data architectures

Salary Range $ 80,000-$100,000 Per year

TCS Employee Benefits Summary:


Discretionary Annual Incentive.


Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.


Family Support: Maternal & Parental Leaves.


Insurance Options: Auto & Home Insurance, Identity Theft Protection.


Convenience & Professional Growth: Commuter Benefits & Certification & amp; Training Reimbursement.


Time Off: Vacation, Time Off, Sick Leave & Holidays.


Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

#LI-SP1