Data Engineer

Peak Performers has partnered with a Texas State Agency to assist in their search for a hybrid Data Engineer (Big Data Engineer) in Austin, TX. In this position, you will perform advanced data engineering duties, supporting data engineering, machine learning, and analytics initiatives using the Databricks Unified Analytics Platform. Applicants for employment with Peak Performers must possess work authorization that does not require sponsorship for a visa now or in the future. Major duties Designs and develops scalable data pipelines using Apache Spark on Databricks. Implements ETL/ELT workflows for both structured and unstructured data. Optimizes Spark jobs for performance and cost efficiency. Integrates Databricks solutions with cloud services, such as Azure Data Factory. Automates deployments using CI/CD tools and version control systems. Collaborates with cross-functional teams including data scientists, analysts, and stakeholders. Ensures data quality, governance, and security using Unity Catalog or Delta Lake. Designs and maintains data models, schemas, and database structures to support analytical and operational use cases. Evaluates and implements appropriate data storage solutions, including data lakes (Azure Data Lake Storage) and data warehouses. Implements data validation and quality checks to ensure accuracy and consistency. Contributes to data governance initiatives, including metadata management, data lineage, and data cataloging. Implements data security measures, including encryption, access controls, and auditing, while ensuring compliance with regulations and best practices. What are we looking for? 4+ years of experience implementing ETL/ELT workflows for both structured and unstructured data. 4+ years of experience automating deployments using CI/CD tools, DevOps, and version control systems. 4+ years of experience collaborating with cross-functional teams (data scientists, analysts, stakeholders) in agile, multicultural environments. 4+ years of experience designing and maintaining data models, schemas, and database structures. 4+ years of experience evaluating and implementing data storage solutions, including Azure Data Lake Storage and data warehouses. 4+ years of experience with data validation, quality checks, metadata management, data lineage, and data cataloging. 4+ years of experience implementing data security measures (encryption, access controls, auditing) and ensuring compliance. 4+ years of proficiency in Python and R programming languages. 4+ years of strong SQL querying and data manipulation skills. 4+ years of experience with the Azure cloud platform. 4+ years of strong troubleshooting and debugging capabilities. 3+ years of experience designing and developing scalable data pipelines using Apache Spark on Databricks. 3+ years of experience optimizing Spark jobs for performance and cost efficiency. 3+ years of experience integrating Databricks solutions with Azure Data Factory. 3+ years of experience ensuring data quality, governance, and security using Unity Catalog or Delta Lake. 3+ years of deep understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL. 3+ years of hands-on experience with Databricks notebooks, clusters, jobs, and Delta Lake. Knowledge of ML libraries (MLflow, Scikit-learn, TensorFlow) preferred. Databricks Certified Associate Developer for Apache Spark or Azure Data Engineer Associate preferred. Exciting Opportunity Details Rate: $64.50/hour W2 Contract Worksite Location: Austin, TX 78751 Estimated Start Date: 10/28/2026 through 08/31/2027 (opportunity for extension) The Peak Performers difference Excellent medical health, dental & vision insurance at 60 days Our employees may choose to participate in a 403(b) retirement plan Qualified applicants with chronic medical conditions and/or disabilities receive priority placement We pay weekly through direct deposit In business since 1994 and thousands of job placements All employment offers are contingent on the successful completion of a pre-employment criminal background check, in compliance with all applicable federal and state laws. Peak Performers is an equal opportunity employer and will consider all applicants without regard to race, marital status, sex, age, color, religion, national origin, veteran status, disability or any other characteristic protected by law. Peak Performers does not accept unsolicited resumes from headhunters, recruitment agencies or fee-based recruitment services. #J-18808-Ljbffr Peak Performers