Sr. Big Data Engineer

Sr. Big Data Data Engineer

Horsham, PA/Remote/Hybrid

C2H role Work hours (ex. 9am-5pm day/night shifts rotating shifts etc)? 9AM - 5PM EST

Interview process – Initial Phone Screen – Team Interview – Hands-on Development Exercise

Must have skills – Data Analysis and Architecture, Spark, PySpark, Python, Sqoop, Pig, Hive, No SQL Data Stores, Object Store, Design and Development of APIs, Kafka

Role Summary: This role will be responsible for transforming extensive and complex data into consumable business capabilities. Create system architecture, design, and specification using in-depth engineering skills and knowledge to solve complex development problems and achieve engineering goals. Determine and source appropriate data for a given analysis. Work with data modelers/analysts to understand the business problems they are trying to solve, then create or augment data assets to feed their analysis. Acts as a resource and mentor for colleagues with less experience.

Core Responsibilities:

• Lead a team of data engineers to develop data products and tools, explore new technologies, and continually improve tools and processes.
• Hands-on experience working with Data Analysis and Architecture, Spark, PySpark, Python, Sqoop, Pig, Hive, No SQL Data Stores, Object Store, Design and Development of APIs, Kafka
• Create business value by leading the design, get your hands dirty, write code, and ultimately deploy big data and machine learning capabilities.
• Possess expert knowledge in performance, large-scale data distributed system scalability, system architecture, and data engineering best practices.
• Give the highest priority to operational excellence, evaluate system performance, security, design system metrics, and driving quality improvements.
• Provide leadership, work collaboratively, and be a mentor in a fantastic team.
• Your expertise is deep and broad; you’re hands-on, producing both detailed technical work and high-level architectural designs.

5+ years of recent hands-on in an object-oriented language (Java, Scala, Python). 5+ years of experience designing and building data pipelines and data-intensive applications. Experience using Big Data frameworks (e.g., Hadoop, Spark), databases for complex data assembly and transformation. Experience working with Healthcare data is a plus.