Principal Data Engineer
United IT
Principal Data Engineer
Location: Fort Mill or Austin (Hybrid)
We are looking for a skilled Data Engineer to join our team and help build robust, scalable, and efficient data pipelines. The ideal candidate will have strong expertise in AWS, Python, Spark, ETL Pipelines, SQL, and Pytest. This role involves designing, implementing, and optimizing data pipelines to support analytics, business intelligence, and machine learning initiatives.
Key Responsibilities:
- Design, develop, and maintain ETL pipelines using AWS services, Python, and Spark.
- Optimize data ingestion, transformation, and storage processes for high-performance data processing.
- Work with structured and unstructured data, ensuring data integrity, quality, and governance.
- Develop SQL queries to extract and manipulate data efficiently from relational databases.
- Implement data validation and testing frameworks using Pytest to ensure data accuracy and reliability.
- Collaborate with data scientists, analysts, and software engineers to build scalable data solutions.
- Monitor and troubleshoot data pipelines to ensure smooth operation and minimal downtime.
- Stay up-to-date with industry trends, tools, and best practices for data engineering and cloud technologies.
Required Skills & Qualifications:
- Experience in Data Engineering or a related field.
- Strong proficiency in AWS (S3, Glue, Lambda, EMR, Redshift, etc.) for cloud-based data processing.
- Hands-on experience with Python for data processing and automation.
- Expertise in Apache Spark for distributed data processing.
- Solid understanding of ETL pipeline design and data warehousing concepts.
- Proficiency in SQL for querying and managing relational databases.
- Experience writing unit and integration tests using Pytest.
- Familiarity with CI/CD pipelines and version control systems (e.g., Git).
- Strong problem-solving skills and ability to work in a fast-paced environment.
Preferred Qualifications:
- Experience with Terraform, Docker, or Kubernetes.
- Knowledge of big data tools such as Apache Kafka or Airflow.
- Exposure to data governance and security best practices.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Principal Data Engineer in Austin, TX vacancy
$138.7k - $206.95k
...developing, and supporting Databricks-based pipelines, medallion-layer data products, and enterprise integrations that enable analytics,... ...You will partner closely with product owners, architects, data engineers, report and analytics teams, and source-system teams to define...SuggestedFull timeH1bLocal areaWork from homeRelocation packageFlexible hours$114.75k - $135k
Job Title Data Engineer Job Description Summary Key Objectives: Supports the development, optimization, and maintenance of Cushman & Wakefield’s commercial real estate (CRE) forecasting infrastructure across the Americas. This role is focused on engineering...SuggestedMinimum wageFull timeFlexible hours$162.5k - $260.4k
...matters - to you, to your community, and to the world. Progress starts with you. Job Description The Senior Manager, Data Engineering, leads teams in designing, building, and overseeing the deployment and operation of technology architecture, solutions, and software...SuggestedWork experience placementWork at officeLocal areaVisa
Austin, TX4 days ago$121k - $215k
...be valued and empowered, then we invite you to apply to our Data Engineering Manager position. This position may be filled remotely or in... ...after meeting eligibility requirements; and more. EOE/Minority/Female/Disabled/Vet. Principal Applicants Only. #J-18808-LjbffrSuggestedFull timeTemporary workRemote workFlexible hoursHusch Blackwell
Austin, TX1 day ago- ...maintain, and support large-scale and complex ETL and distributed Big Data applications. Work with platform architects to ensure technical... ...: Requires Bachelor's in Computer Science, Computer Engineering, Mechanical Engineering, or a related field and 60 months of progressive...SuggestedRemote work
Charles Schwab
Austin, TX3 days ago - ...work for a company that values your contributions and supports your growth, we would like to meet you. Key Responsibilities Data Engineering team leadership . Lead, mentor, and develop a team of data engineers supporting reporting, analytics, data science, AI, and data...Permanent employmentTemporary workWork at officeFlexible hours
- ...an exciting opportunity for a highly experienced and technically proficient Principal Electrical Engineer to engage in the design, fabrication, and integration of electrical systems for data center power and cooling products, as well as factory enablement projects. This...Contract workRemote work
Jabil Circuit, Inc.
Round Rock, TX9 days ago $128k - $249k
Senior Data Engineer Information Technology All Practice Areas Austin, TX, Boston, MA, Charleston, SC, Charlotte, NC, Chicago, IL, Dallas, TX, Durham, NC, Harrisburg, PA, Houston, TX, Irvine, CA, Kansas City, MO, Los Angeles, CA, Miami, FL, Nashville, TN, New York, NY,...Temporary workLocal areaRelocationFlexible hoursAfternoon shiftK&L Gates LLP incorporated
Austin, TX4 days ago- ...A federal service contractor is seeking a qualified CommVault Systems Engineer proficient in data protection and backup solutions. The ideal candidate will have experience in designing and managing complex backup systems, troubleshooting issues, and a strong knowledge...For contractorsFlexible hours
CGS Federal (Contact Government Services)
Austin, TX1 day ago - Description Within the Productization Data Tools team, you will assume the responsibility for developing and maintaining the software... ...of relevant experience Degree in Computer Science, Software Engineering, or a related field Preferred Qualifications Three or more years...
Apple
Austin, TX4 days ago - SII Group USA is seeking a Senior Data Migration Engineer to design and execute a large-scale migration of sensitive identity and credentialing data into AWS GovCloud. You will own the technical migration lifecycle, from source-data analysis and transformation specifications...3 days per week
SII Group USA
Austin, TX22 hours ago $71.06k
Data Engineer I This position focuses on expanding and optimizing the healthcare system’s data and data pipeline architecture, overseeing data flow and collection for cross‑functional teams. The Data Engineer design, builds, and maintains scalable data infrastructure to...Flexible hoursShift workPhase2 Technology
Austin, TX22 hours ago- ...execute video advertising campaigns. To achieve this, we need an amazing team. Here’s where you come in… We are looking to hire a Data Engineer with a strong work ethic and a passion for the fast‑paced startup life. The ideal candidate doesn't just build pipelines — they...Work at officeImmediate startRemote workVisa sponsorship
Cross Screen Media
Austin, TX2 days ago $135k - $155k
Role Summary Data Engineering is a key role in the development team and is responsible for building and maintaining the AI-ready data foundation that powers inKind's intelligent products, machine learning models, and large language model (LLM) applications. The position...Temporary workWork experience placementWork at officeRemote workinKind
Austin, TX3 days ago- The Senior Data Warehouse Data Engineer will develop,implementandmaintaindata platform solutions, ensuringoptimalperformance, reliability, and scalability for business and analytics teams. This role requires a strong background in data warehousing and data lakes, ETL processes...Work at officeWeekend workAfternoon shift
University Federal Credit Union
Austin, TX3 days ago $110k - $120k
...yr Responsibilities Candidate with strong understanding of the data ecosystem (Databricks/DBT/Redshift, etc), Distributed Systems,... ...Data Infrastructure, etc. Qualifications Mandatory Skills: Data Engineering Full Stack. Experience: 10+ Years. Seniority level Mid-Senior level...Full timeSmart IT Frame
Austin, TX2 days ago$40 - $50 per hour
...Collaborate strategically with Customer Delivery Executive (CDE) Technical Customer Success Managers (TCSM), AI Data Center Functional Leaders, Technical Engineers, and the Field Service Manager (FSM) to develop and implement recovery protocols * Conduct root cause...Contract workTemporary workTEKsystems
Round Rock, TX4 days ago- ...decision cultures at our clients. Everyone lives and breathes data an AI, so you’ll be surrounded by an inquisitive, technical, and... ...Snowflake, AWS, and Anthropic partner, and we’re looking for data engineering folks that love diagramming the big picture and then executing...
Applied Curiosity - Drive Insights and Decision Making
Austin, TX2 days ago $75 - $80 per hour
...your skills and experience — talk with your recruiter to learn more. Base pay range $75.00/hr - $80.00/hr We are seeking a Data Engineer to join a dynamic Agile team and support the build and enhancement of a large-scale data integration hub . This role requires...Contract workLanceSoft
Austin, TX23 hours ago$150k - $190k
...Advanced Solutions Group of SHI that is building new digital experiences for our internal users, customers and partners. The Senior Data Engineer will be responsible for the analysis, design, and development of solutions focused on data engineering and ETL workflows. As a...Work experience placementWorldwideFlexible hoursShift workSHI GmbH
Austin, TX1 day ago$144k - $187k
...SpyCloud a special place to work. If you’re driven to align your career with a fantastic mission, look no further! As a Senior Data Engineer at SpyCloud, you will lead the architectural transformation of our data systems, working closely with Product Management,...Temporary workLocal areaRemote workWorldwideVisa sponsorshipFlexible hoursSpyCloud
Austin, TX23 hours ago- ...Employment Type: Full-Time, Mid-level Department: Business Intelligence CGS is seeking a passionate and driven Data Engineer to support a rapidly growing Data Analytics and Business Intelligence platform focused on providing solutions that empower our federal customers...Full timeFlexible hours
CGS Federal (Contact Government Services)
Austin, TX2 days ago - ...place to grow, make an impact, and work with people who care, we'd love to meet you! About the Role We are looking for a Senior Data Engineer to design and build scalable data lakes, warehouses, and lakehouse architectures supporting a thematic research platform that processes...Work at officeRemote workVisa sponsorshipWork visaFlexible hours
AgileEngine
Austin, TX3 days ago - ...our customers get the high-quality gear they need to make the most of their adventures. We are BUILT FOR THE WILD™. The Senior Data Engineer will be part of a team that designs, builds, and operates YETI’s enterprise data platform with a strong emphasis on Azure Databricks...Local area
Yeti
Austin, TX1 day ago - Design and implement scalable batch and near-real-time data pipelines. Develop ETL/ELT workflows optimized for performance and cost... ...automation. Required Qualifications 6+ years of experience in data engineering for analytics or ML systems. Experience in Python, Scala, or...
Optomi
Austin, TX2 days ago $70 - $75 per hour
...experience — talk with your recruiter to learn more. Base pay range $70.00/hr - $75.00/hr Qualifications 6+ years of software engineering or data engineering experience with a strong focus on SQL and large datasets. Proficiency in at least one modern language such as...Contract workMindSource
Austin, TX23 hours ago- Data Engineer Habitat Energy is a fast growing technology company focussed on the physical and financial optimisation of energy storage and renewable generation assets globally through complex models and trading. By maximising the returns from these assets we aim to drive...Work at officeFlexible hours
Habitat Energy Limited
Austin, TX1 day ago - About the role Teza Technologies is looking for Data Engineers to join our data team. Data drives systematic trading and is critical to all aspects of the firm's business. This is a hands‑on position on a small team of data engineers with growth potential, as this team...InternshipWork at officeFlexible hours
Teza Capital Management LLC
Austin, TX2 days ago - ...ongoing commitment to democratize access to investing and financial planning. We are looking for an energetic and accomplished Data Engineer (individual contributor) with a strong technology and solution delivery acumen who will be responsible for the successful design...Full timeFor contractorsWork at office
Charles Schwab
Austin, TX1 day ago $96.88k
Job Description JOB ID MF110248: Data Engineer Metaforge IT Solutions INC, 106 Taylor St STE 200, Hutto, TX. Based Project Management & Consulting Services Firm has multiple openings for JOB ID MF110248: Data Engineer. Job duties include: Gather and document requirements...RelocationMetaForgeIT
Austin, TX2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Principal Data Engineer. Be the first to apply!
Related searches
- senior director engineering Austin, TX
- chief engineer Austin, TX
- civil engineer project manager Austin, TX
- senior principal engineer Austin, TX
- engineering director Austin, TX
- senior chief engineer Austin, TX
- director of product engineering Austin, TX
- director systems engineering Austin, TX
- principal infrastructure engineer Austin, TX
- data center chief engineer Austin, TX