Data Engineer (Python/AWS)

Job Description

Job Description

** PLEASE NOTE THAT THIS IS AN IN OFFICE M-F APPOINTMENT. NO REMOTE, C2C OR SPONSORSHIP PROVIDED**

What you will do:

• Build, test, and operate production data pipelines in Python on our modern pipeline framework, orchestrated with Prefect
• Build and maintain warehouse models in Snowflake using dbt — clean, tested, documented, and cost-aware
• Migrate legacy pipelines and Oracle-based components onto current frameworks and standards, retiring technical debt as you go
• Work across the platform stack — Kafka, Redis, InfluxDB, Oracle, and Snowflake — building to the established pattern for each rather than reaching for the tool you already know
• Acquire data from external sources — vendor APIs, files, feeds, and web sources — and land it reliably
• Implement data quality, freshness, and reconciliation checks so problems surface before users find them
• Use AWS data services where our platform patterns call for them
• Support what you ship — monitoring and alerting on your components, and investigating when something breaks
• Contribute to making platform data AI-ready, and work with the catalog and steward teams so what you build is documented, classified, and findable
• Engage directly with analysts and desk users to check that what you are building solves the actual problem

 

Qualifications:

• 3+ years of hands-on data engineering experience building and operating production data pipelines
• Strong Python — clean, tested, maintainable code, not scripts that happen to run. Fluency with pandas and the wider data-handling ecosystem
• SQL depth — you can model, query, and tune, and you know what makes a query expensive before you run it
• Snowflake or a comparable cloud warehouse — dimensional modeling, performance tuning, and cost-aware design. dbt experience is a strong plus
• Breadth across data technologies — streaming, caching, timeseries, relational, and warehouse, with a view on where each belongs. Kafka, Redis, InfluxDB, Oracle, and Snowflake are what we run; comparable exposure matters more than an exact match
• Pipeline orchestration experience — Prefect preferred; Airflow, Dagster, or similar considered
• Sound engineering fundamentals — object-oriented design, design patterns, testing, code review, and version control as habits rather than requirements
• Working knowledge of AWS data services and how to compose them into something reliable
• A build-to-operate mindset — monitoring, alerting, and failure recovery are part of how you design, not something added later
• Clear communication — you can explain a technical trade-off to an analyst, and turn a vague request into the right set of questions
• Terraform, Docker, or API development (FastAPI, Flask) exposure is welcome — we use all three
• Experience in financial services, commodities, or energy trading data is a plus; the data volume, latency requirements, and stakes are real