Big Data Engineer

Big Data Engineer

Hybrid - Malvern, PA

Responsibilities

• Writes ETL (Extract / Transform / Load) processes, designs database systems, and develops tools for real-time and offline analytic processing.
• Troubleshoots software and processes for data consistency and integrity.
• Integrates data from a variety of sources for business partners to generate insight and make decisions.
• Translates business specifications into design specifications and code.
• Responsible for writing programs, ad hoc queries, and reports.
• Ensures that all code is well structured, includes sufficient documentation, and is easy to maintain and reuse.
• Partners with internal clients to gain a basic understanding of business functions and informational needs.
• Gains working knowledge in tools, technologies, and applications/databases in specific business areas and company-wide systems.
• Participates in all phases of solution development. Explains technical considerations at related meetings, including those with business clients.
• Tests code thoroughly for accuracy of intended purpose.
• Reviews end product with the client to ensure adequate understanding. Provides data analysis guidance as required.
• Provides tool and data support to business users and fellow team members.
• Tests and implements new software releases through regression testing. Identifies issues and engages with vendors to resolve and elevate software into production.
• Participates in special projects and performs other duties as assigned.

Qualifications

• 1-3 years Data Engineering Experience with SQL, Python & AWS
• Experience with CI/CD and Jira. Experience with Data Quality
• Experience with setting up lineage
• Undergraduate degree or equivalent combination of training and experience.