Big Data Developer

Job Description

Title: Big Data Engineer

\n

Location: Basking Ridge, NJ (5-day onsite/week)

\n

\n

Job Responsibilities:

\n

\n

\n
• Design and lead end-to-end data integration
\n
• Perform performance tuning, diagnose, and resolve complex big data and streaming technical issues.
\n
• Design and implement scalable solutions across the technology stack, including data processing, database, web services, network, and security layers.
\n
• Perform integration testing and provide support to QA teams on large-scale projects.
\n
• Contribute to requirements analysis, project estimation, system design, coding, and testing.
\n
• Evaluate and implement appropriate third-party libraries and software to enhance product capabilities.
\n
• Stay current with emerging techniques and trends in big data and AI/ML.
\n

\n

\n

Job Requirement:

\n

\n

\n
• 5+ years of hands-on experience designing and implementing data applications in production using Java, Python, or Scala on big data platforms, with deep expertise in big data technologies and data optimization techniques.
\n
• 5+ years of hands-on experience with streaming technologies, specifically with a strong focus on Kafka, and various open-source software platforms (e.g., Apache ecosystem, Linux).
\n
• Proven ability to optimize data flow and design efficient, scalable data pipelines, including data lakes and warehouses on containerized clusters.
\n
• Strong understanding of distributed data architecture concepts (e.g., caching, MapReduce, stream processing).
\n
• Solid knowledge of data security, governance, and compliance.
\n
• Proficiency in programming languages: Java, Python, Scala, and SQL.
\n
• Strong analytical capabilities, creativity, and critical thinking skills.
\n
• Effective written and verbal communication, with the ability to facilitate team collaboration.
\n

\n

\n

\n