Data Engineer III

Work Schedule Monday–Friday, 8 a.m.–5 p.m. Positions Supervised None Position Summary As a Data Engineer at Beusa, you will own the performance, reliability, and cost-efficiency of our Databricks environment. You’ll design and optimize Delta tables, pipelines, and SQL Warehouse workloads so teams get fast, trustworthy data at the lowest reasonable cost. You will collaborate with cross‑functional teams to translate requirements into high‑performing, production‑grade datasets and dashboards with clear SLAs. Essential Functions (The following duties and responsibilities are all essential job functions, as defined by the ADA, except for those that begin with the word "may.") Build and operate Lakehouse pipelines on Databricks (Bronze/Silver/Gold) using Delta Lake, Delta Live Tables (DLT), and/or Jobs. Own ingestion patterns (Autoloader, CDC, streaming) optimization Own data modeling, quality checks, and performance optimization. Own performance optimization – profile and tune Spark/SQL workloads: partitioning, clustering, constraints, liquid clustering (if applicable), manage table health. Engineer Delta tables for speed and cost: partitioning, Z‑Ordering or clustering, constraints, file sizing; manage table health with Auto Optimize, OPTIMIZE, and VACUUM. Implement incremental processing patterns (MERGE with Change Data Feed, APPLY CHANGES INTO) with idempotency and exactly‑once delivery. Partner with data consumers to deliver reliable, well‑documented datasets with clear SLAs. Design, build, and implement interactive dashboards and reports using Power BI and other visualization tools. Collaborate with business units (Operations, Field Engineers, Assets) to gather requirements and translate them into technical solutions. Develop and maintain data models, ensuring data quality and integrity. Integrate data from multiple sources, including real‑time field equipment and sensors. Continuously evaluate and adopt new data management and analytics technologies. Educate and support stakeholders on data tools and best practices. Advocate for Beusa’s digital transformation and data‑driven decision‑making. Engage with technology vendors and partners as needed. Occasional travel to meet with stakeholders. Performs other related duties as assigned to assist with successful operations and business continuity. Qualifications Position Requirements Successfully passes background check, pre‑employment drug screening, and any pre‑employment aptitude and/or competency assessment(s). Valid U.S. driver’s license and ability to pass a background check. Education/Experience Level Bachelor’s degree in MIS, Computer Science, Engineering, IT, or related field. 7‑10+ years of experience in a data‑focused role (Business Intelligence, Data Analysis, Data Management, etc.). Qualifications, Skills, Competencies, and Abilities 8+ years of experience in Data Engineering, Data Warehousing, or Analytics Engineering. 3+ years of hands‑on experience with Databricks and the Lakehouse Platform. Expert‑level proficiency with PySpark, Spark SQL, and distributed data processing concepts. Advanced SQL development skills with experience in relational database platforms (Microsoft SQL Server preferred). Experience designing and implementing Databricks Medallion Architectures (Bronze, Silver, Gold). Experience building and optimizing batch and streaming data pipelines. Experience designing, implementing, and maintaining CI/CD deployment strategies for data platforms. Experience defining and implementing testing frameworks, including Unit Testing, Data Quality Testing, Schema Validation, Integration Testing, and Regression Testing. Experience troubleshooting and optimizing Spark workloads for performance, scalability, reliability, and cost efficiency. Experience implementing data governance, security, and access control frameworks, including Unity Catalog or similar technologies. Experience with source control and DevOps practices using Git, Azure DevOps, or similar tools. Experience mentoring junior engineers and providing technical leadership across projects. Proven ability to work across multiple teams and stakeholders to deliver enterprise data solutions. Physical Requirements/Work Environment The physical demands and work environment described here are representative of those that must be met by an employee to successfully perform the essential functions of this job. Reasonable accommodations may be made to enable individuals with disabilities to perform the essential functions. Frequently required to walk, sit, climb, bend, reach and squat/kneel. The Business Intelligence Developer works primarily indoors and will be sitting prolonged periods of sitting at a desk and working on a computer. Must be able to access and navigate each department at the organization’s facilities. The Business Intelligence Developer may be required to lift heavy objects; therefore, The Business Intelligence Developer must be able to lift 25lbs. Work hours may include early morning, late afternoon/evening hours, and weekends in combination depending on job demands. The Company is committed to the cause of equal employment opportunity for all employees and applicants, thus abiding by all applicable state and federal laws. Our practices regarding employment, job promotion, compensation, training, and termination do not discriminate on the basis of race, color, religious creed, age, sex, national origin, veteran's status, disability, pregnancy, genetic information, or any other legally protected status. It is expected that all employees, both management and staff, will fully support these nondiscriminatory policies. The company has reviewed this job description to ensure essential functions and duties have been included. It is not intended to be an exhaustive list of all functions, responsibilities, skills, and abilities. #J-18808-Ljbffr