Specialist - Data Engineering
LTM
Role description Job Title: Pyspark Developer Work Location : Irving, Texas Job Summary
- We are seeking a highly skilled and motivated Data Engineer to play a pivotal role in designing building and optimizing our next generation scalable data pipelines This position requires expertise in processing massive datasets using cutting-edge technologies like Apache Spark PySpark and Hive within a dynamic cloud environment Your primary objective will be to ensure the utmost data reliability speed and efficiency providing a robust foundation for downstream business intelligence and advanced analytics initiatives
- Key Responsibilities
- Data Pipeline Development Maintenance Design build and maintain highly scalable and efficient ETLELT data pipelines utilizing PySpark and Spark SQL for complex data transformations
- Cloud Data Infrastructure Management Deploy manage and scale critical data infrastructure components on leading cloud platforms such as Amazon Web Services AWS eg EMR Glue Microsoft Azure eg Databricks Synapse or Google Cloud Platform GCP
- Data Warehousing Storage Optimization Strategically manage data layout partitioning and indexing within Apache Hive and various cloud data lake solutions to optimize performance and accessibility
- Performance Tuning Optimization Proactively identify and resolve performance bottlenecks in Spark jobs leveraging Spark UI for indepth analysis effectively managing data skewness and optimizing memory utilization
- Diverse Data Integration Develop robust solutions for ingesting highvolume and diverse datasets from both structured relational databases and unstructured flat files into our data ecosystem
- Automated Workflow Orchestration Implement and manage automated data workflows using industrystandard scheduling tools like Apache Airflow or platformnative schedulers ensuring timely and reliable data delivery
- Strategic Collaboration Partner closely with data scientists business analysts and crossfunctional enterprise teams to translate complex business requirements into technically sound and efficient data solutions
- Required Core Technical Skills
- Big Data Frameworks Expertise Demonstrated high proficiency in Apache Spark architecture including a deep understanding of drivers executors and Directed Acyclic Graphs DAGs
- Advanced Programming Exceptional coding skills in Python and extensive experience with the PySpark API for developing intricate data transformations and processing logic
- Querying Schema Management Strong command of HiveQL and ANSI SQL coupled with expertise in data partitioning techniques and effective schema definition
- Optimized Storage Formats Indepth understanding and practical experience with optimized big data storage file formats such as Parquet ORC and Avro
- Cloud Ecosystem Development Handson development experience utilizing cloudnative big data utilities eg AWS EMR Azure Databricks within major cloud platforms
- Data Warehousing Fundamentals Solid foundation in Dimensional Data Modeling including Star and Snowflake schemas and practical experience with Data Lakes concepts and implementation
- Preferred Qualifications
- CICD DevOps Automation Experience with Continuous IntegrationContinuous Deployment CICD practices and automation tools like Git Jenkins or Ansible
- NoSQL Database Integration Exposure to and experience with NoSQL databases such as HBase Cassandra or MongoDB
- Professional Cloud Certifications Relevant professional cloud certifications eg AWS Certified Data Engineer Microsoft Certified Azure Data Engineer Associate are highly valued
Vacancy posted 13 hours ago
Similar jobs that could be interesting for youBased on the Specialist - Data Engineering in Irving, TX vacancy
- ...Azure Data Engineer Location: Irving TX onsite Desired Skills: 1. Possess a minimum of 5 years of experience in data engineering and development 2. Strong experience in designing and implementing data solutions using Azure data services Azure Databricks Azure...Suggested
Futran Tech Solutions Pvt. Ltd.
Irving, TX3 days ago - ...Job Summary ~ Seeking a Senior Specialist with 7 to 11 years of experience in Python... ...maintain robust Python applications focused on data processing and analytics Implement... ...for Python projects Collaborate with data engineers analysts and stakeholders to ensure...Suggested
LTM
Irving, TX1 day ago - ...Description Work closely with source data application teams and product owners to design implement and support analytics solutions... ...insights to make better decisions Implement data migration and data engineering solutions using Azure products and services Azure Data Lake...Suggested
Futran Tech Solutions Pvt. Ltd.
Dallas, TX5 days ago - ...Role description Title - Azure Data Engineer Location - Fortworth, TX This is Hybrid role and there will be in-person client interview at Fortworth, TX, if shortlisted. TOP requirements: Azure Cloud Databricks Pyspark SQL WHAT YOU'LL DO...Suggested
LTM
Dallas, TX4 days ago - Overview Looking for Fulltime profiles who are open to relocate to Dalla, TX Description Seeking a data engineer with 6 to 10 years of experience to design develop and maintain scalable data engineering solutions using Python and Azure Data Factory within the Azure Data...SuggestedFull timeTemporary workFor contractorsLocal areaRelocationOverseas
- ...democratize access to investing and financial planning. Join a high-impact QA organization where you will lead quality engineering efforts across a portfolio of data engineering initiatives. In this role, you will partner closely with Product Owners and development teams from...Work at office
- ...Scala , Java , Bash , Shell Scripting Cloud Platforms: Azure Data Factory , Databricks , PySpark , Synapse , Event Hubs , ADLS Gen... ...BigQuery , Cloud Storage , Cloud Functions , PubSub , Looker Data Engineering ETL: Apache Spark , PySpark , Structured Streaming , Databricks...Temporary workLocal area
$149.78k
...DevOps Engineer – Job ID 11993 Job duties include: Participate in designing, developing, and maintaining infrastructure and tools to operate... ...the United States may be required. Equal Opportunity Employer. Data Engineer – Job ID 11992 Job duties include: Support in the...RelocationSlesha Inc
Irving, TX2 days ago$75.17k - $130.5k
...NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive... ..., adaptable, and forward‑thinking organization, apply now. Data Engineer - Hybrid – Irving, Texas, United States We are looking for a...Temporary workRemote workFlexible hoursNTT DATA
Irving, TX5 days ago$92.7k - $222.48k
...– helping to simplify health care one person, one family and one community at a time. Position Summary We are seeking a Senior Data Engineer with a proven track record in delivering scalable cloud‑native data pipelines, microservices, and AI‑driven workflow orchestration...Hourly payFull timeTemporary workLocal areaCVS Health
Irving, TX4 days ago$114.38k - $185.4k
...Position Summary Caremark LLC, a CVS Health company, is hiring in Irving, TX for a Sr. Data Engineer. The role involves developing, building and managing large‑scale data structures, pipelines and efficient ETL workflows to address complex problems and support business...Temporary workLocal areaHispanic Alliance for Career Enhancement
Irving, TX3 days ago$114.38k - $144.2k
...Position Summary: Caremark LLC, a CVS Health company, is hiring a Data Engineer in Irving, TX. The role involves analyzing data engineering problems and developing, building, and managing large‑scale data structures, pipelines, and efficient ETL workflows to support complex...Temporary workLocal area7505 Caremark, L.L.C.
Irving, TX2 days ago- ...Job Title: Data Engineer – MEM SQL Location: New Jersey / Irving, TX / Tampa, FL Job Description We are looking for an experienced Data Engineer with strong expertise in MEM SQL (SingleStore) to join our team supporting Incedo projects. The ideal candidate will be responsible...
Qode
Irving, TX6 days ago - ...Work Model: 4 days on-site, 9:00am-5:00pm The Full-Stack AI/ML Engineer will work on high-impact initiatives supporting the company's digital... ...optimization and sales forecasting , helping the business make data-driven decisions and improve profitability. Collaborating cross-...Contract work
Addison Group
Irving, TX2 days ago - ...We are seeking a Senior Data Engineer to join a multidisciplinary team focused on building scalable data platforms and solutions that enhance customer experience and drive business growth. This role involves designing and optimizing data pipelines, mentoring junior engineers...
Compunnel
Irving, TX2 days ago - ...Position Overview We are looking for an experienced and versatile Data Engineer to join our dynamic and fast-growing team. If you are passionate about data, solving complex problems, and working directly with enterprise stakeholders to translate business needs into...Work at officeRemote workWork from home
Medium
Irving, TX2 days ago - ...We are seeking a Lead Snowflake Engineer to design, build, and operate robust data pipeline infrastructure that ingests from diverse source systems, normalizes data into consumption-ready models in Snowflake, and delivers clean, reliable datasets for Power BI reporting...
Resolve Tech Solutions
Irving, TX2 days ago - ...minimizing exposure and risks on the project or closure of escalations. Required Skills and Experience 9+ years of experience in Data Engineering and Data Analytics. Expert in Big Data Technologies including Spark, Kafka, Kudu, Impala, HBase, and Spark Scala programming....For contractors
TechDigital Group
Irving, TX2 days ago $55 - $60 per hour
...Job Title and Location Data Engineer – Irving, Texas (US-TX), United States (US) Day to Day Job Duties Design and build robust data pipelines to ingest structured and unstructured data from multiple sources. Build and optimize data storage solutions (relational/NoSQL databases...Hourly payTemporary workFlexible hoursNTT Data Americas, Inc.
Irving, TX4 days ago- ...Job Title Big data Developer Work Location: Irving, Texas Job Summary Seeking a Senior Python Data Engineer with 3 to 5 years of experience specializing in Apache Spark and Python to design and implement scalable data solutions Job Description Develop and maintain large...Temporary workLocal area
LTM
Irving, TX1 day ago - ...Hilltop Holdings is looking for a Senior Data Engineer! The Senior Data Engineer is a key technical leader responsible for the design, maintenance, and optimization of enterprise data systems across on-premises and cloud environments. This hybrid role combines traditional...Full timeWork experience placement
Hilltop Holdings
Irving, TX3 days ago - ...degree or equivalent in Computer Science, Mathematics, Software Engineering, Management Information Systems, Computer Engineering/... ...or Statistics. Minimum 3-5 years of relevant work experience in data engineering, with experience in data modeling and analytics. Strong...Work experience placement
CBTS
Irving, TX2 days ago - ...to building quality homes and delivering exceptional customer service while fostering opportunity and growth for associates. Sr Data Engineer A Sr. Data Engineer is an experienced professional responsible for leading the development of complex data architectures. The role...Local area
Lennar
Irving, TX1 day ago $114.38k - $185.4k
...one community at a time. Position Summary Caremark LLC, a CVS Health company, is hiring for the following role in Irving, TX: Sr Data Engineer to develop, build and manage large-scale data structures, pipelines and efficient Extract/Load/Transform (ETL) workflows to...Hourly payFull timeTemporary workInternshipLocal areaRemote workCVS Health
Irving, TX14 hours ago- ...Data Engineer Working at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you'll have the opportunity to grow your career, give back to your community and make a real impact...
Citigroup Inc
Irving, TX3 days ago $119.77k - $140.9k
...job – they want to make a difference! U.S. Bank is seeking a Software Engineer who will contribute toward the success of our technology initiatives in our digital transformation journey. The Senior Data Engineer will be responsible for the analysis, design, testing,...Temporary workWork experience placementLocal area3 days per weekU.S. Bank
Irving, TX4 days ago$100k - $120k
-Title: Data Engineer -Pay Range: $100-$120K -Location: Dallas-Fort Worth, TX -Benefits: Medical, Dental, Vision, 401(k) We are looking to bring on a 3-6 year Data Engineer to our growing team. What we will need is someone who brings...Addison Group
Irving, TX5 days ago- ...Role Sr Data Engineer Location Canada Toronto Requirements ~6-8 years of experience with Ab Initio in ETL projects ~ Expertise as an Ab Initio Technical Expert ~ Strong problem-solving skills: Able to analyze issues, identify root causes, and...
United IT Solutions
Irving, TX1 day ago - ...Data Engineer We are seeking a skilled and detail-oriented Data Engineer to join our growing data team. The ideal candidate will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure to support analytics, reporting, and data...Contract work
Publicis Groupe Holdings B.V
Irving, TX4 days ago - ...Sr. Data Engineer Required Skills/Abilities/Attributes: ~5 years of experience in data architecture, business system designs, and ETL architecture. ~3 years of experience with hands-on experience designing and constructing data platforms and/ or data pipelines...
3B Staffing LLC
Irving, TX4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Specialist - Data Engineering. Be the first to apply!
Related searches
- mental health specialist Irving, TX
- assessment specialist Irving, TX
- program specialist Irving, TX
- art specialist Irving, TX
- remote outreach specialist Irving, TX
- information technology specialist Irving, TX
- adoption specialist Irving, TX
- licensing specialist Irving, TX
- accessibility specialist Irving, TX
- equal employment specialist Irving, TX