Data Engineer

Data Engineer

Growing healthcare organization is hiring a Data Engineer to join a team of talented Data Engineers in their Digital Products and Services team. Their number one job responsibility is to deliver the most remarkable patient experience, in every dimension, every time, and understand how to contribute to the health system's vision of achieving that commitment to patients and families. The Data Engineer will support software developers, database architects, data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. Digital Products and Services team members are responsible for securely managing information systems throughout their lifecycle, including knowing what information systems are within their scope of responsibility, understanding what sensitive data is stored, transmitted, or processed on those information systems, enforcing the security principles of least privilege and least functionality, knowing what events may constitute a cybersecurity incident, and understanding their role in security incident response activities.

MUST LIVE IN THESE STATES ONLY: Delaware, Virginia, NC, SC, Georgia, Florida, Alabama, Mississippi, Louisiana, Michigan, Indiana, Oklahoma, Wyoming, Nevada

MUST HAVE PYTHON EXPERIENCE

Additional Skills/Requirements (required):

• Capable of building and optimizing highly scalable data sets, 'big data' pipelines and architectures to support structured and semi-structured data processing.
• Capable of performing root cause analysis on processes and data to identify opportunities for improvement
• Capable of building processes supporting data transformations, data structures, metadata, dependency, and workload management.
• Capable of writing scalable automated unit test and data validation frameworks.
• Capable of working with business partners to design and implement automated data validations to maintain data quality
• Capable with data warehousing concepts and design.
• Capable of writing and optimizing SQL
• Capable of writing code in 2 or more object function/object-oriented scripting languages including Python, Scala, C++, or Java
• Capable of working with ADLS, Delta Lake, SQL Server, Oracle and NoSQL databases.
• Capable of implementing lambda data-processing architecture (batch and streaming) such as Azure Event Hubs, Apache Spark (including structured streaming), Apache Kafka
• Capable of working independently or within a team and taking ownership of tasks.
• Capable of explaining and mentoring junior team members in the above concepts and techniques.
• Capable of building big data management processes to manage archival, retention, dataset time- travel at scale.
• Capable of working in a fast-paced environment and able to handle multiple tasks with minimal supervision.
• Capable of DataOps or DevOps and agile methodologies

Additional Skills/Requirements (preferred):

• Understanding of Healthcare data is desired.
• MUST HAVE Experience with HL7 or FHIR
• Experience with leveraging microservices and API-based solutions such as Azure Functions
• Capable of working with message queuing and stream processing.
• Experience with Trino or other data virtualization tools
• Experience with Informatica Powercenter.
• Capable of orchestration of complex data pipelines to process batch and streaming data using Airflow, Azure Event Grid.
• Capable of building capabilities to implement row level filtering and column data masking.

Essential Functions:

• Process: Extracts, tests, loads, and validates data. Analyze the data flow requirements and develops a scalable architecture for staging and loading data.
• Interprets the transformation rules for all target data objects and develops the software components to support the transformation.
• Assist with design/implementation of data staging methods. Prepare/implement data verification and testing methods. Create scripts/processes required to extract, transform, clean, and move data and metadata from a source application so they can be loaded into a data warehouse, data mart, or operational data store.
• Must be able to troubleshoot the existing code and develop new code, stored procedures, views, and functions.
• Ability to be on-call and respond to production issues in a timely manner.
• Documentation: Create and maintain ETL, data orchestration, and system documentation. Create source to target data lineage documentation. Maintain the development, test, user acceptance, and production ETL and data orchestration environments.
• Compliance: Adheres to the principles of the organization's Health Code of Ethics Compliance Plan in every encounter with customers, vendors, regulating agencies, the public and fellow employees. Adheres to corporate policies, procedures and standards. Follows SDLC standards.
• Communication: Effective in giving and receiving the information required to perform the job. Proficient in verbal and written communication skills.

Education:

• 2 Year / Associate Degree Required, Computer Science or related field
• 4 Year / Bachelor's Degree Preferred, Computer Science or related field

Experience:

• 7 years Preferred, Experience in Information Technology
• 4 years Required, Experience with at least one contemporary data transformation tool

Certifications:

• Microsoft, Databricks, AWS or GCP Certified Data, Preferred Engineer Associate