Senior Data Engineer

Job Title: Senior Data Engineer (Compliance Data Architecture & Engineering, Manager)

Location: Remote/Tampa, FL

Duration: Long term

Key Responsibilities:

Build & Operate the Compliance Data Foundation: Serve as a principal individual contributor responsible for further development of newly acquired compliance and security datasets, building the end-to-end Databricks ETL pipelines that transform raw data into highly refined, high-utility data products designed to measure and monitor control performance and effectiveness.

Implement Medallion Data Pipelines: Design, optimize, and maintain robust ETL orchestration and automated pipelines to curate Bronze, Silver, and Gold data layers. Ensure raw, conformed, and consumption-ready compliance data products are highly structured and standardized.

Normalize Core Security & Compliance Data: Develop and implement data normalization schemas for diverse, high-volume telemetry feeds ingested from core corporate tools, including ServiceNow, Microsoft Defender, CrowdStrike, Zscaler, Wiz, Microsoft Entra ID, SailPoint, and many others.

Transform Telemetry to Monitored Logic: Partner with the data source and control domain owners to translate data sets into monitoring enablement and programmatic control-testing logic built directly into the data layer.

Produce High-Insight Data Products: Develop and maintain specialized compliance data products—including Databricks dashboards, reports, and analytic applications—that provide clear, actionable insights into compliance control performance, coverage gaps, and operational effectiveness enabling future agentic capabilities

Support SDLC, Automation & Release Gates: Manage GitHub repositories, branching, and peer review standards. Establish automated job monitoring, exception handling, and code promotion processes to transition pipelines reliably into production.

Qualifications:

Education & Professional Experience: Bachelor’s degree with 5–8 years of experience spanning a hybrid of Cyber Security Operations (e.g., threat detection engineering, security telemetry analysis) and Security Compliance/GRC roles.

Databricks Engineering Expertise (or similar platform): Advanced technical experience building ETL pipelines, orchestrating data workflows, and developing analytics in Databricks. Prior experience developing, deploying, or managing custom Databricks applications or Databricks Agents is a strong plus.

Compliance Telemetry & Normalization: Proven capability to normalize, correlate, and test data structures sourced from security, identity, and governance platforms (ServiceNow, Defender, CrowdStrike, Zscaler, Wiz, Entra ID, and SailPoint).

Control Assurance & Data Strategy: Solid understanding of security compliance frameworks, policy-to-control translation logic, and the deployment of continuous control monitoring (CCM) strategies within a secure cloud data lake.

Agile Software Delivery (SDLC): Hands-on experience with modern software development workflows, including version control (Git/GitHub), peer review cycles, and release testing/approval gates.

Communication & Collaboration: Excellent verbal and written communication skills with a proven ability to bridge the gap between complex engineering requirements and business-facing risk management objectives.

Relevant Certifications include:

CISSP, CRISC, or Databricks Certified Data Engineer Associate/Professional