Senior Data Engineer - Encounter Processing & Data Platform || Remote || 12+ months || Video

Title: Application Data Developer / Senior Data Engineer Encounter Processing & Data Platform



Location: Lincoln, NE Remote, Local Candidates Only Duration: Long Term



Interview: Webcam

Visa: Any visa except OPT, CPT, and H-1B

Position Overview

Client is seeking a Senior Data Engineer / Application Data Developer to serve as the primary technical owner for an enterprise data processing framework supporting Medicaid Encounter Processing and enterprise data ingestion .

This is a mission-critical platform that processes provider, member, reference, and encounter data used by downstream Medicaid systems and reporting applications.

The application framework is built using Scala, Apache Spark, Apache Hive, and Drools and operates on the Cloudera Data Platform (CDP) .

The selected candidate will be responsible for the ongoing development, maintenance, enhancement, production support, troubleshooting, performance optimization, and operational reliability of the platform.

The role requires strong hands-on expertise across the complete technology stack and the ability to independently own the application framework and its production operations.

Key Responsibilities

Application & Data Engineering

• Serve as the primary technical owner of the Scala/Spark application framework supporting Medicaid Encounter Processing.
• Develop, maintain, and enhance enterprise-scale distributed data processing applications.
• Develop and maintain Spark-based batch processing workflows.
• Develop and manage Hive tables and data processing workflows.
• Own enterprise data ingestion processes for master data and source system files.
• Implement business and regulatory changes within the data processing framework.
• Develop and maintain business rules using the Drools Rules Engine .
• Perform SQL development, query optimization, and data analysis.

Production Support & Operations

• Provide ongoing production support for mission-critical data processing applications.
• Investigate and resolve application defects and production issues.
• Troubleshoot distributed Spark applications and data processing failures.
• Monitor application performance and identify opportunities for optimization.
• Perform application performance tuning and optimization.
• Coordinate production releases, version management, and deployments.
• Support Linux-based production environments.
• Perform basic operational administration and monitoring of the Cloudera Data Platform .
• Coordinate with infrastructure and cloud operations teams when required

Documentation & Collaboration

• Maintain technical documentation, operational procedures, and support documentation.
• Document application architecture, workflows, processes, and troubleshooting procedures.
• Provide knowledge transfer to technical teams and stakeholders.
• Work with cross-functional teams to implement business and regulatory requirements.
• Participate in Agile development processes and project activities as needed.

Required Qualifications

• 7+ years of experience developing enterprise-scale distributed data processing applications.
• Strong hands-on development experience with Scala .
• Strong hands-on experience with Apache Spark 2.x and/or Spark 3.x .
• Experience developing and maintaining applications on Cloudera Data Platform (CDP) 7.x or equivalent enterprise Hadoop distributions.
• Experience with Apache Hive 3.x , including data processing and Hive table management.
• Hands-on experience implementing business rules using the Drools Rules Engine .
• Strong SQL development and query optimization skills.
• Experience supporting Linux-based production environments .
• Experience troubleshooting distributed Spark applications in production .
• Experience with Git and modern version control practices.
• Strong production support, troubleshooting, and problem-solving skills.
• Ability to take ownership of a complex enterprise data platform and work independently.

Strong Plus:

• Experience in Medicaid or healthcare environments.
• Experience with Medicaid Encounter Processing .
• Experience with Cloudera Manager, HDFS, and YARN .
• Experience integrating data processing applications with IBM DataStage .
• Familiarity with AWS infrastructure supporting Cloudera .
• Experience working in Agile/Scrum environments.
• Experience with Jira and Confluence .

Technology Stack

Primary Technologies:

• Scala
• Apache Spark 2.x / 3.x
• Apache Hive 3.x
• Drools
• Cloudera Data Platform (CDP) 7.x
• SQL
• Linux
• Git
  
  
  
  

Additional Technologies:

• Cloudera Manager
• HDFS
• YARN
• IBM DataStage
• AWS
• Jira
• Confluence
  
  
  
  

Ideal Candidate Profile

The ideal candidate is a Senior Data Engineer / Big Data Engineer with strong hands-on experience building and supporting distributed data processing platforms.

The candidate should be able to independently work across Scala, Spark, Hive, Drools, and Cloudera rather than being specialized in only one area.

Healthcare/Medicaid experience is highly desirable, particularly experience with Medicaid Encounter Processing , but the core technical requirements are the priority.



Aditi Sikarwar

Senior Talent Acquisition

E- View email address on us.fitly.work

STELLENT IT A Nationally Recognized Minority Certified Enterprise