Streaming Data Lead (Flink)

Java/Apache Flink Engineer

Remote - Must be local to a hub (Minneapolis, Atlanta, Chicago, San Francisco, Dallas)

12 Month+ Contract

Overview: This role will be responsible for working alongside a FTE + 1 other Engineer to further develop and manage a custom-built Flink framework. Legacy applications like Informatica PowerExchange are being sunset, need support migrating those data movements and workflows into Flink.

Flink framework being used for CDC (Change Data Capture) for MongoDB and SQL Server. Sachi has established a CDC COE within his group.

Team: Currently a 2-person team needing more support, increased velocity of tooling migrations to Flink framework + additional data sources needing integration (6-7 ongoing projects involving the Flink framework).

• Needs someone with experience in Flink, real-time data pipeline development (i.e. Flink SQL, DataStream API)
• They are primarily a Java environment, but if someone only has experience in Python (and is a Python "guru" that is acceptable.)
• Needs core engineering experience and fundamentals, CI/CD (Azure DevOps preferred), Github
• Cloud experience needed, Azure preferred.
• Kafka experience preferred

Experience in building AI agents also preferred for a separate initiative: they own US Bank's Kafka instance with 1000+ topics; they are lacking a schema registry and seeing an increased demand for one to understand lineage breakdown and metadata around their Kafka implementation and footprint to date. Looking to build a small POC with Azure Open AI to read Kafka topics and generate JSON schemas for metadata classification. This is lacking funding to move forward so not an imminent need for the role, but would be nice to have from the longer-term team perspective