Sr Data Engineer

Ideal/ Required

Skills

Actual Candidate skills, credentials, describe actual experience

Required

Currently lives in the commutable NYC area; where do you live?

8 yrs, Required

Data Engineering, with demonstrable seniority in design ownership - not just implementation. Owns the data models end to end deciding how complex financial instruments and their time-series behavior should be represented - and build the ingest, transform, and publish layers around them.

8 yrs, Required

Strong hands-on Databricks- Delta Lake, Spark (PySpark and Spark SQL), job/workflow orchestration, and performance tuning across partitioning, clustering, file sizing, and incremental processing. Production depth required; familiarity is not sufficient

Required

Demonstrated skill in data modeling and abstraction - you can explain why you structured a domain the way you did, and what you rejected

Carefully review source system's business logic, analyzing financial logic that is in a large, sparsely documented codebase

Required

Substantial experience processing and transforming financial time-series data, including the correctness concerns unique to it - point-in-time accuracy, as-of joins, bitemporal history, late-arriving and restated data, corporate actions, and calendar alignment.

Required

Full pipeline experience- ingest, transform, and publish - rather than depth in only one segment

Required

Experience with large-scale financial systems and the data volumes, auditability, and accuracy expectations that come with them

Required

Working knowledge of structured finance instruments and datasets - securitizations, ABS/MBS, loan-level and cashflow data etc

Required

Strong reading & reasoning about unfamiliar production code in order to extract requirements from it. Comfort operating without documentation, and the patience to work through a large legacy system methodically rather than rewriting on assumption

Required

Strong SQL and Python; comfort with shell scripting and version-controlled, CI/CD-driven delivery

Preferred

Experience migration of custom structured finance datasets from a legacy proprietary platform onto Databricks including parity testing and cutover.

Preferred

Familiarity with languages & platforms common to older proprietary financial systems (e.g. legacy SQL dialects and stored procedures, C++/C#/Java monoliths, or vendor-specific scripting).

Preferred

Databricks Unity Catalog, Delta Live Tables, and cost governance.

Preferred

Cloud data platform experience (AWS or Azure), including object storage, IAM, and encryption/key management.

Preferred

Orchestration tooling (Airflow, Databricks Workflows, or similar) and infrastructure-as-code.

Preferred

Exposure to preparing datasets for downstream analytics, quantitative research, or AI/ML consumption

Required

Proven ability to work independently with minimal supervision and ambiguous starting requirements

Required

Strong communication, collaboration, and influence with ability for technical leadership

Required

Ownership mindset - problem solver, not a task executor

Required

Work in fast-paced, dynamic environments and manage competing priorities.

Required

Clear English, strong written and verbal communications and teamwork skills

Required

Strong self-starter attitude, team-oriented and good communication skills

Preferred

Certs: Databricks, Cloudera, Hortonworks, etc. Describe if active