Data Engineer

Salary Range: $100,000 - $180,000 per year Hours: Monday - Friday, 9 am to 6 pm PST We are looking for a seasoned Data Engineer to play a dual role in accelerating our data capabilities and ensuring absolute data integrity. With 5+ years of hands‑on experience, you will be responsible for moving our data ecosystem away from third‑party visualization dependencies and building a centralized cloud data warehouse. A critical component of this role is owning our data quality assurance (QA) — you will develop infrastructure to ingest complex structured and unstructured data under tight API constraints, while simultaneously building automated testing frameworks to validate that data at every stage of the pipeline. Our Approach We don’t come to this role with a locked‑in technology stack — and that’s intentional. You’ll work closely with our team to evaluate options, weigh cost and scalability tradeoffs, and recommend the right solution for our specific needs. We’re looking for someone who can think critically about the build vs. buy decision, bring informed recommendations to the table, and help us arrive at a solution that is both technically sound and cost‑effective. Collaboration, transparency, and shared ownership of outcomes are core to how we work. Key Responsibilities Infrastructure Design Architect and build a formal cloud data warehouse ecosystem to establish a single, reliable source of truth, selecting the platform that best fits our scale and budget. Design custom ingestion systems and caching layers to seamlessly stream data from complex external APIs without hitting rate limits or dropping payloads. Unstructured Data Processing Build pipelines capable of parsing, structuring, and storing unstructured data sets for downstream business intelligence initiatives. Data Quality Assurance (QA) & Observability Automated Data Testing Implement QA frameworks using the tooling we collectively agree on to verify data accuracy, completeness, and freshness at every pipeline stage. Set up proactive alerting systems to capture data anomalies, null values, or schema drift before bad data impacts business logic or reports. Track down and debug data discrepancies between source APIs and final warehouse tables. Required Technical Skills & Qualifications 5+ years of hands‑on experience architecting and querying cloud data warehouses on a major cloud platform. Deep proficiency in Python and SQL for writing ETL/ELT pipelines, with experience using at least one modern orchestration or transformation framework. Strong understanding of managing high‑volume API requests, token management, pagination, and error‑handling strategies. Data QA & Validation Proven track record building test suites for data environments — validating schemas, row counts, uniqueness, and distribution constraints. Familiarity with how data formatting impacts downstream BI tools; experience migrating off third‑party visualization platforms into custom warehouse solutions is highly preferred. Core Competencies Cloud Data Warehousing Data Quality & Testing Python & SQL ETL / ELT Design Schema & Data Modeling Monitoring & Alerting BI Tool Integration What success looks like Engineers are no longer blocked by API rate limits — ingestion flows reliably with caching and error‑handling in place. The team has a clear, active migration path off third‑party BI platforms into a custom cloud warehouse with full data ownership. Every BI tool and report is built on a foundation of automatically validated, QA‑verified data. Pipeline anomalies, null values, and schema drift are caught proactively — before they ever impact business decisions. We have collectively landed on a cloud architecture and tooling stack that is scalable, maintainable, and cost‑effective for our needs. #J-18808-Ljbffr