Lead Data Engineer

This position collects, stores, processes and builds applications within our big data platform. Responsible for integrating these applications with the architecture used across the organization. Compensation Range: $131,000.00 - $220,000.00 . Offer may vary based on job level and geographic location. Relocation support is offered for eligible candidates. Eligible for H1 visa transfer. Primary Accountabilities Leads an engineering team to meet project deadlines and priorities. Supervises assigned data engineering team members and activities. Ensures the quality, completeness, security, privacy, and integrity of data throughout the data lifecycle. Documents critical workflows and operational support aspects of the team’s responsibilities. Develops deep understanding of data sources, granularity, availability, and limitations. Provides proactive technical oversight and advice to application architecture and development teams fostering re-use, design for scale, stability, and operational efficiency of data/analytical solutions. Creates maintainable, scalable code to load and manipulate data in the data warehouse. Facilitates communication upward and across project teams and business stakeholders. Specialized Knowledge & Skills Requirements Demonstrated experience providing customer-driven solutions, support, or service. In-depth knowledge of SQL or NoSQL and experience using a variety of data stores (e.g., RDBMS, analytic database, scalable document stores). Extensive hands‑on Python programming experience, with an emphasis on building ETL workflows and data‑driven solutions. Ability to employ design patterns and generalize code to address common use cases. Capability to author robust, high‑quality, reusable code and contribute to the division’s inventory of libraries. Expertise in big data batch computing tools (e.g., Hadoop or Spark), with demonstrated experience developing distributed data processing solutions. Applied knowledge of cloud computing (AWS, GCP, Azure). Knowledge of open‑source machine learning toolkits, such as sklearn, SparkML, or H2O. Solid data understanding and business acumen in data‑rich industries like insurance or finance. Applied knowledge of data modeling principles (e.g., dimensional modeling and star schemas). Strong understanding of database internals, such as indexes, binary logging, and transactions. Experience using infrastructure‑as‑code tools (e.g., Docker, CloudFormation, Terraform). Experience with software engineering tools and workflows (e.g., Jenkins, CI/CD, git). Practical experience authoring and consuming web services. Licenses: Not Applicable. Travel Requirements: Up to 10%. Physical Requirements: Work primarily involves sitting/standing. We provide comprehensive medical, dental, vision, and wellbeing benefits, a 401(k) contribution, a pension plan, an annual incentive, 9 paid holidays, paid time off (23 days accrued annually for full‑time employees), a student loan repayment program, and paid‑family leave. We are an equal‑opportunity employer. It is our policy to comply with all applicable federal, state, and local laws pertaining to non‑discrimination, non‑harassment, and equal opportunity. We also consider qualified applicants with criminal histories, consistent with applicable federal, state, and local law. American Family Insurance is committed to the full inclusion of all qualified individuals. Requests for reasonable accommodations to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment may be made by emailing View email address on click.appcast.io. #J-18808-Ljbffr SwiftCruit