Sr. Data Engineer

The Senior Data Engineer will be responsible for building and maintaining data pipelines and workflows that support ML, BI, analytics, and software products. This individual will work closely with data scientists, data engineers, analysts, software developers and SMEs within the business to deliver new and exciting products and services. The main objectives are to develop data pipelines and fully automated workflows to drive operational efficiency and effectiveness by enabling data-driven decisions across the organization. This includes fostering collaboration, building partnerships, co-developing products, sharing knowledge, and providing insights and valuable predictive information to business teams and leaders to highlight potential risks and opportunities that initiate the drive for change. A Day in the Life: This role has an impact on our data analytics and fleet management. What You’ll Do: Building, maintaining, optimizing, and automating scalable data pipelines using PySpark and Databricks across cloud and on-premise infrastructure. Data acquisition, data ingestion, data cleansing, data transformation, and data migration. Design, implement, and test data integrations and ETL processes using Databricks workflows, Spark SQL and PySpark, with detailed technical documentation. Improving data integrity and quality by implementing Delta Lake, Databricks notebooks, and PySpark-based validation frameworks to streamline existing processes. Identify trends in datasets and develop workflows and help with streaming analytics to take advantage of raw data flows. Identify opportunities for data acquisition from internal and external sources. Curate rich datasets and prepare data for predictive and prescriptive modeling using Databricks, PySpark, and feature engineering best practices. Strive to engineering excellence by simplifying, optimizing, and automating processes and workflows. Build and maintain stakeholder-facing reporting solutions and dashboards using Tableau, enabling business users to monitor key metrics and make data-driven decisions. Ensures data sets are thoroughly tested before release to production environments. Mentor junior level data Engineers and data analysts in best practices for data preparation and modeling. What We’re Looking For: A bachelor's degree or higher in Computer Science or related field or equivalent experience required, master’s degree preferred. Minimum of 8 years’ hands-on experience in data engineering or ETL development role required. Minimum 2 years of intensive experience with data lakes, preferably running on the AWS cloud platform required. Strong problem solving and critical thinking skills with a proven record for identifying and diagnosing problems and solving complex problems with simple, logical solutions required. Outstanding verbal, written, presentation, facilitation, and interaction skills, including ability to effectively communicate technical and non-technical issues and concepts to multiple organization levels required. Preferred Experience: Strong expertise with modern data lake tools such as Databricks or PySpark Experience with ETL tools such as Informatica Experience with event driven architectures and data streaming pub/sub technologies such as Kafka and Amazon Kinesis Strong capabilities in languages such as Python, R, Scala or Java Agile/Scrum experience This position starts at $120K, commensurate of experience. What You’ll Get: 40% off any standard Hertz Rental Paid Time Off Medical, Dental & Vision plan options Retirement programs, including 401(k) employer matching Paid Parental Leave & Adoption Assistance Employee Assistance Program for employees & family Educational Reimbursement & Discounts Voluntary Insurance Programs - Pet, Legal/Identity Theft, Critical Illness Perks & Discounts – Theme Park Tickets, Gym Discounts & more #J-18808-Ljbffr