Principal Data Engineer
United IT
Principal Data Engineer
Location: Fort Mill or Austin (Hybrid)
We are looking for a skilled Data Engineer to join our team and help build robust, scalable, and efficient data pipelines. The ideal candidate will have strong expertise in AWS, Python, Spark, ETL Pipelines, SQL, and Pytest. This role involves designing, implementing, and optimizing data pipelines to support analytics, business intelligence, and machine learning initiatives.
Key Responsibilities:
- Design, develop, and maintain ETL pipelines using AWS services, Python, and Spark.
- Optimize data ingestion, transformation, and storage processes for high-performance data processing.
- Work with structured and unstructured data, ensuring data integrity, quality, and governance.
- Develop SQL queries to extract and manipulate data efficiently from relational databases.
- Implement data validation and testing frameworks using Pytest to ensure data accuracy and reliability.
- Collaborate with data scientists, analysts, and software engineers to build scalable data solutions.
- Monitor and troubleshoot data pipelines to ensure smooth operation and minimal downtime.
- Stay up-to-date with industry trends, tools, and best practices for data engineering and cloud technologies.
Required Skills & Qualifications:
- Experience in Data Engineering or a related field.
- Strong proficiency in AWS (S3, Glue, Lambda, EMR, Redshift, etc.) for cloud-based data processing.
- Hands-on experience with Python for data processing and automation.
- Expertise in Apache Spark for distributed data processing.
- Solid understanding of ETL pipeline design and data warehousing concepts.
- Proficiency in SQL for querying and managing relational databases.
- Experience writing unit and integration tests using Pytest.
- Familiarity with CI/CD pipelines and version control systems (e.g., Git).
- Strong problem-solving skills and ability to work in a fast-paced environment.
Preferred Qualifications:
- Experience with Terraform, Docker, or Kubernetes.
- Knowledge of big data tools such as Apache Kafka or Airflow.
- Exposure to data governance and security best practices.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Principal Data Engineer in Austin, TX vacancy
- ...connections, and building confidence through expert guidance.We have an exciting opportunity for a motivated and energetic Principal Data Engineer to shape and lead our technical strategy, a role critical to driving the next phase of growth and innovation within our Data...SuggestedWork at officeLocal area
- ...Principal Data EngineerWe are looking for a skilled Data Engineer to join our team and help build robust, scalable, and efficient data pipelines. The ideal candidate will have strong expertise in AWS, Python, Spark, ETL Pipelines, SQL, and Pytest. This role involves designing...Suggested
InterSources
Austin, TX3 days ago $152k - $242k
...that power infrastructure, solutions, and data management our customers need most. Our... ...and reimagining how work gets done. Engineers define intent, author precise specifications... ...open roles and various levels (Senior, Principal and Senior Principal).Who We AreWe believe...SuggestedRelocation package$200k
...Responsibilities Responsible for designing & implementing data architecture that will help with scalability, performance & reliability... ...architecture at our company and work closely with the CTO & Engineering Leadership in order to solve the toughest technical challenges...SuggestedHired Recruiters
Austin, TX12 days ago- Job DescriptionAbout the RoleThe Data Engineering Manager will lead a team of data engineers focused on building and operating Customer data platforms supporting OnStar, Connected Services, Customer, Marketing, Personalization, AI, and Advanced Analytics.This is a highly...SuggestedFull timeLocal areaWork from homeRelocation package
$145k - $187k
...that power infrastructure, solutions, and data management our customers need most. Our... ...and reimagining how work gets done. Engineers define intent, author precise specifications... ...technology in Austin, Texas.Mechanical Principal Engineer Why Work with UsDell Technologies...$124k - $242k
...that power infrastructure, solutions, and data management our customers need most. Our... ...and reimagining how work gets done. Engineers define intent, author precise specifications... ...future of enterprise technology. Senior Principal Thermal Engineer-Data Center Solutions...Relocation package$151.2k - $204.6k
...we’re the people who keep the cloud running. We support all AWS data centers and all of the servers, storage, networking, power, and... ...You’ll join a diverse team of software, hardware, and network engineers, supply chain specialists, security experts, operations managers...Flexible hours$132.4k - $238.4k
...world's top brands, offering comprehensive engineering, supply chain, and manufacturing... ...seeking a highly experienced and innovative Principal Controls Engineer to lead the development... ...critical systems, with a strong emphasis on data center infrastructure, power/energy...Full timeTemporary workWork at officeLocal areaRemote workWorldwide- ...world's top brands, offering comprehensive engineering, supply chain, and manufacturing... ...experienced and technically proficient Principal Electrical Engineer to engage in the design... ...and integration of electrical systems for data center power and cooling products, as well...Full timeContract workWork at officeLocal areaRemote workWorldwide
- ...Head of Data Engineering About the Company Ambitious education company Industry Primary/Secondary Education Type Privately Held Founded 2020 Employees 201-500 About the Role The Company is seeking a Head of Data Engineering to spearhead...Remote workRelocation
- ...Head of Data Engineering About the Company Ambitious education company Industry Primary/Secondary Education Type Privately Held Founded 2020 Employees 201-500 About the Role The Company is seeking a Head of Data Engineering to spearhead...Remote workRelocation
- ...adventures. We are BUILT FOR THE WILD.About the RoleWe are actively searching for a Senior Director, Enterprise Integrations and Data Engineering to oversee YETI’s Data Warehouse, Analytics Platforms, MDM, and Integration Platforms. This role is responsible for defining...Full timeContract workLocal areaFlexible hours
$126.2k - $264.1k
...Organizational Effectiveness Lead business operations programs for Engineering & Infrastructure organizations in partnership with CIO-office... ...(QBRs), staff meetings, and leadership updates. Provide data-driven insights and recommendations to support strategic...Temporary workWork at officeFlexible hoursOracle
Austin, TX3 days ago$218k - $323.95k
...ourselves, each other, and our communities.Job Summary:This role is the technical authority for Venmo’s data platform infrastructure and performance engineering. The Principal Engineer sets the technology vision, defines architectural direction, and provides deep technical...Full timeWork at officeLocal areaImmediate startFlexible hours- ...Data Engineering Manager We are seeking an experienced Data Engineering Manager to lead the design, development, and delivery of scalable data platforms and data products that power personalized customer experiences across digital retail channels. This leader will...Work experience placement
- ...Head of Data Engineering, Logistics About the Company Ambitious education company Industry Primary/Secondary Education Type Privately Held Founded 2020 Employees 201-500 About the Role The Company is seeking a Head of Data Engineering for...Remote workRelocation
$134.5k - $265.1k
...combining Deloitte's sports industry experience with proprietary data, AI-native decision intelligence, and reusable Converge... ...organizations. The team works across Product Management, Data Engineering, Data Science, Software Engineering, Forward Deployed Engineering...Local area- ...IAM, Cloud Monitoring/Logging)Expert Python for production-grade data and backend engineeringStrong SQL and data modeling for... ...least privilege, auditability)Strong observability and reliability engineering skills (monitoring, alerting, incident response, SLAs/SLOs)Fundamental...Full timeWork at office
- ...implementing database schemas, including tables, indexes, views, stored procedures, and triggers to meet business requirements and ensure data integrity.Data Modeling: Creating data models that represent the structure of the data, relationships between entities, and how data...
$100k - $125k
...all.Who You’ll Work WithAt Slalom, we co-create custom software, data, AI, and cloud solutions with clients who are ready to... ...products, experiences, and organizations.Our Data Architecture & Engineering team sees data as the foundation for decision-making, artificial...Temporary workWork at officeLocal area- Job DescriptionThe RoleWe are seeking a Senior Data Engineer to join our Motorsports Data Engineering team, building next-generation data platforms that power high-performance racing programs across Formula 1, NASCAR, IndyCar, IMSA, and beyond.This role is responsible for...Full timeLocal areaWork from homeRelocation package
- ...skills, and the ability to multitask in a fast-paced environment.ACADEMIC CREDENTIALS:Bachelor’s or Master’s degree in computer engineering, Electrical Engineering, Computer Science Technical certifications in relevant software systems are highly desirableThis role is...Remote work
- ...create solutions in client context.•Experience of BI, Reporting and Data Warehousing.Ability to play a Techno Functional role.•... ...of data would be an advantage.•Expected to lead offshore data engineers.Preferred Skill and Experience•Experience in Snowflake & PL SQL...Full timeTemporary workWork experience placementRelocation
$100.79k - $160.26k
...DLA Piper is a place you can engage in meaningful work and grow your career. Let’s see what we can achieve. Together.SummaryThe Data Engineer, Solutions & Data role designs, builds, and operates data pipelines and data integration processes that translate raw data into...Full timeWork at officeRemote workVisa sponsorship$154.6k - $209.1k
Are you passionate about building the data infrastructure that powers intelligent, autonomous systems at enterprise scale? Amazon Web Services is seeking a Senior Data Engineer to own the governed data platform and agentic infrastructure that drives operational intelligence...Flexible hours- ...SolarWinds and grow with us!We are building our People Analytics platform from scratch on a modern data stack (GCP/BigQuery, CData, dbt). We are looking for a Senior Analytics Engineer to lead the architecture, implementation, and ongoing evolution of our new People Analytics...
$63 - $68 per hour
...| USD 63-68 / Hourly (C2C)Employment Type: Full TimeClient: TechMContact: Meghana GorusuCompany: SRI Tech SolutionsJob Title: Data Engineering & AnalyticsLocation: Cupertino, CA (Hybrid) / Austin, TX / Seattle, WAAbout the Role:We build data systems that power innovation...Hourly payFull time$147k - $181k
...our team as we help shape a brighter way forward. About JLL and Data PlatformJLL is a leading professional services firm that specializes... ...pipelines of their own.What this job involvesAs a Senior Data Engineer on the Data Consumption team, you will own the design and...Full timeContract workLocal areaRemote work$40 - $50 per hour
...Collaborate strategically with Customer Delivery Executive (CDE) Technical Customer Success Managers (TCSM), AI Data Center Functional Leaders, Technical Engineers, and the Field Service Manager (FSM) to develop and implement recovery protocols• Conduct root cause analysis...Contract workTemporary work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Principal Data Engineer. Be the first to apply!
Related searches
- director data engineering Austin, TX
- principal engineer Austin, TX
- senior chief engineer Austin, TX
- principal infrastructure engineer Austin, TX
- civil engineer project manager Austin, TX
- principal data engineer Austin, TX
- senior civil engineer project manager Austin, TX
- chief engineer Austin, TX
- project engineer assistant project manager Austin, TX
- director quality engineering Austin, TX