Data/Spark Engineer-US

Java/Spark Engineer

We are currently sourcing for a Java/Spark Engineer to work in Durham, NC!

The Role

As part of UAP (Unified Acquisition Platform) team you will be working on enhancing the next generation in-house acquisition platform using cloud technologies. UAP will be built using Java Spark and Python programming language and use AWS EMR as the infrastructure for execution. You would be developing lambda, Spark, Python utilities for managing end to end feed flow. We will focus on building tools and capabilities to accelerate acquisition, compare data across vendors, externalize rules for data conversion, build inventory/registry of feeds and track lineage.

You would be focusing on:

• Develop new application for managing feed inventory and tracking their usage
• Developing application interface using Java Spark for converting vendor feeds into business-friendly format
• Develop accelerators for new feed acquisition
• Develop accelerators for data comparison reports
• Developing Lambda triggers for event driven and batch driven use case
• Supporting production during warranty period
• Participate in code review and define standard methodology
• Manage branching and deployments without conflicts
• Provide L3 production support
• Enhancing framework for future needs and make it a more generic product.

The Expertise and Skills You Bring

• Bachelor in information technology
• 6+ years of experience with application development and hands on experience with Spark/Scala
• Expertise in providing leadership support by providing technical solutions
• Expertise in handling huge dataset
• Expertise in code reviews for standards and efficient coding practices
• Expertise in building monitoring and alerting tools to report failure or potential performance degradation