Lead Data Engineer

Job Title

This role will be responsible for transforming extensive and complex data into consumable business capabilities. Create system architecture, design, and specification using in-depth engineering skills and knowledge to solve complex development problems and achieve engineering goals. Determine and source appropriate data for a given analysis. Work with data modelers/analysts to understand the business problems they are trying to solve, then create or augment data assets to feed their analysis. Acts as a resource and mentor for colleagues with less experience.

Core Responsibilities:

• Lead a team of data engineers to develop data products and tools, explore new technologies, and continually improve tools and processes.
• Hands-on experience working with Data Analysis and Architecture, Spark, PySpark, Python, Sqoop, Pig, Hive, No SQL Data Stores, Object Store, Design and Development of APIs, Kafka
• Create business value by leading the design, get your hands dirty, write code, and ultimately deploy big data and machine learning capabilities.
• Possess expert knowledge in performance, large-scale data distributed system scalability, system architecture, and data engineering best practices.
• Give the highest priority to operational excellence, evaluate system performance, security, design system metrics, and driving quality improvements.
• Provide leadership, work collaboratively, and be a mentor in a fantastic team.
• Your expertise is deep and broad; you're hands-on, producing both detailed technical work and high-level architectural designs.

Skills:

• Must have skills include Data Analysis and Architecture, Spark, PySpark, Python, Sqoop, Pig, Hive, No SQL Data Stores, Object Store, Design and Development of APIs, Kafka
• 5+ years of recent hands-on in an object-oriented language (Java, Scala, Python).
• 5+ years of experience designing and building data pipelines and data-intensive applications.
• Experience using Big Data frameworks (e.g., Hadoop, Spark), databases for complex data assembly and transformation.
• Experience working with Healthcare data is a plus.

Required Skills: Person with strong Python libraries (pandas, nmph) Strong with PySpark/SparkQL – someone who has done Spark implementation in the past. Create API's via Java, Python Able to handle Kafka calls – MQ messages. NoSQL – Hbase or MongoDB or similar Basic Qualification: Additional Skills: Background Check: Yes

Location: Horsham, PA (will need some occasional travel to client site; once or twice a quarter) - must live in Northeast/Mid-Atlantic areas.