Data Engineer - AWS & PySpark
Cliff Services Inc
Data Engineer AWS & PySpark
Location: Dallas, TX or Richmond, VA
Client: Confidential Job Summary
We are looking for an experienced Data Engineer with strong hands-on expertise in AWS and PySpark . The ideal candidate must have prior professional experience working with Banking domain and be capable of developing scalable data pipelines and data processing solutions in a cloud environment.
Key Responsibilities- Design, develop, and maintain scalable data pipelines using PySpark and AWS .
- Develop and optimize large-scale data processing and ETL workflows.
- Work with AWS cloud services to build reliable and high-performance data solutions.
- Perform data transformation, cleansing, validation, and integration.
- Optimize PySpark jobs for performance, scalability, and cost efficiency.
- Collaborate with data architects, analysts, developers, and business stakeholders.
- Troubleshoot data pipeline issues and ensure data quality and reliability.
- Follow engineering best practices for code development, testing, deployment, and documentation.
- Strong hands-on experience with PySpark .
- Strong experience with AWS cloud services .
- Experience developing ETL/data pipelines and processing large datasets.
- Strong Python and SQL skills.
- Experience with data transformation, integration, and data quality.
- Mandatory: Prior Capital One project/client experience.
- Strong communication and problem-solving skills.
- Experience with AWS data services such as S3, Glue, EMR, Lambda, Redshift , or similar.
- Experience with distributed data processing and cloud-based data platforms.
- Financial services/banking domain experience.
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - AWS & PySpark in Dallas, TX vacancy
- ...Type: C2CIndustry: Computer SoftwareClient: WiproContact: Meghana GorusuCompany: SRI Tech SolutionsRole: Data Engineer - Lakehouse (PySpark / Databricks / Airflow / AWS)Location: Dallas, TX (Day 1 Onsite)Job Description:Data Engineer - Lakehouse (PySpark / Databricks /...Amazon Web Service
- ...is currently seeking a Sr. Associate, Data Science Engineer to join our Consulting practice.Responsibilities... ...within AI service providers (e.g., AWS, GCP, Azure) to provide a first-in-... ...MLOps, with hands-on expertise in Python, PySpark, SQL, Databricks, and Snowflake, while...Amazon Web ServiceH1bLocal area
$100k - $105k
Tata Consultancy Services is looking for a Data Engineer to design, develop, and maintain data pipelines using PySpark in a Big Data environment. You will ensure data quality... ..., along with expertise in cloud platforms like AWS and Databricks. Strong problem-solving skills...Amazon Web Service- ...leading financial services organization is seeking a Senior Data Engineer to help build and scale its modern cloud data platform.... ...and business stakeholders.Key Skills:Snowflake dbt Fivetran AWS Python / PySpark SQL Terraform CI/CDWhat You'll Do:Design and optimize...Amazon Web Service
- ...Company: Addison GroupTitle: Senior Data EngineerLocation: Coppell,... ...looking to bring on a Senior Data Engineer to our team due to growth. What... ...based environment (Azure and/or AWS), Azure DevOps (ADO), SQL (ETL, SSIS packages), and PySpark or Scala automation. Architecture...Amazon Web Service
- Job Title: Sr. DATA Engineer Work LocationTampa FLJob Description:Mandatory Karat InterviewRole... ...seeking a highly skilled and motivated AWS Certified Engineer to design build and optimize... ...expertise in big data processing using PySpark and a deep understanding of data...Amazon Web Service
- ...Data Engineer Location: Dallas, TX OR McLean, VA OR New York (Onsite from Day 1 in Hybrid Model) Duration: 6-12+ Months (With... ...further ext. possibilities) Must Haves Python PySpark, Hadoop AWS Services Microservices Data Lakes experience...Amazon Web Service
Samprasoft
Dallas, TX5 days ago - ...top-tier projects. Job DescriptionThe Data Solutions Engineer IV works in close collaboration to design... ...governanceDefine and drive Python, PySpark, and SQL standards for data engineering... ...Fivetran, dbt, Snowflake, Kafka ecosystem, AWS services) with a focus on scalability,...Amazon Web Service
$75.17k - $130.5k
Req ID:376258NTT DATA strives to hire exceptional, innovative and... ...are currently seeking a Data Engineer - Hybrid to join our team in Irving... ...building data pipelines using PySpark, Django, or similar frameworks... ...in cloud environments such as AWS, Google Cloud, or Azure.Strong...Amazon Web ServiceTemporary workWork at officeRemote workFlexible hours- ...Big Data EngineerLocation: RemoteRole SummaryExcellent Scala, PySpark, Java coding experience for REST API and SOAP webservicesExcellent AWS experience (Batch and API microservices) using EMR/EC2/ECS/S3/Airflow/Step Functions, AWS API GatewayExcellent Batch Application...Amazon Web Service
United IT
Irving, TX8 hours ago - ...Senior Data EngineerWe are seeking a highly skilled Senior AWS Data Engineer with expertise in Fivetran, AWS Glue, Python, SQL, and cloud-native data engineering. The... ...Fivetran.Develop ETL/ELT workflows using AWS Glue and PySpark.Build and optimize data lake solutions on AWS....Amazon Web Service
Diverse Lynx
Dallas, TX4 days ago - ...Title: Data Engineer Location: Sunnyvale CA or Austin TX or Elk Grove CA /3 days in a week... ...Comfortable working with cloud platforms (e.g., AWS) and tools like Snowflake 4. Familiar... ...manipulation (e.g., pandas, pyarrow, pyspark) Data Infrastructure &...Amazon Web Service3 days per week
RIT Solutions, Inc.
Dallas, TX1 day ago $140k - $160k
...generation of its enterprise data platform. The team is building... ...effort where you\'ll help define engineering standards, influence... ...Snowflake Modern cloud experience (AWS preferred) Experience building... ...Terraform or Infrastructure as Code PySpark Kafka or streaming data...Amazon Web ServiceHarnham
Richardson, TX5 days ago- ...Required Skills Data Science, Machine Learning, AI, Big Data/ Hadoop, Python, Cloud- AWS/GCP. What were looking for... You are an... ...Statistics, Math, Economics, Engineering or Data Science related fields... ...statistical models using Python, PySpark, Machine Learning libraries,...Amazon Web ServiceWork experience placement
Rapinno Tech
Irving, TX1 day ago - ...Data Engineer We are seeking a highly skilled and motivated Data Engineer to play a pivotal... ...cutting-edge technologies like Apache Spark, PySpark, and Hive within Cloudera Platform. Your... ...cloud-native big data utilities (e.g., AWS EMR, AWS Databricks) within major cloud...Amazon Web Service
United IT
Irving, TX4 days ago $125.76k - $188.64k
...a highly skilled and experienced Senior Data Engineer to join our dynamic and innovative technology... ...scalable ETL/ELT pipelines using PySpark, Spark SQL, and Delta Lake on Databricks... ...data solutions on cloud platforms (e.g., AWS, GCP, Azure). Leverage cloud-native services...Amazon Web ServiceFull time$60 - $65 per hour
...AWS Data EngineerOur esteemed client is seeking an experienced AWS Data Engineer with strong expertise in ETL testing, cloud migration, and Python programming in a production... ...Python (primary language) along with SQL and PySpark to develop data processing and automation...Amazon Web ServiceHourly paySelectMinds
Dallas, TX8 hours ago- ...Dallas for a hybrid position. Senior Data Engineer About the Role We are seeking... ...expertise in Snowflake, Fivetran, dbt, AWS Glue, Python, and SQL. Responsibilities... ...workflows using dbt, AWS Glue, Python, and PySpark. Architect and support data lake and...Amazon Web Service
The Judge Group
Dallas, TX3 days ago $130k - $150k
$130,000-$150,000 Senior Data Engineer Position Summary The Lead Data Solutions Engineer... ...data pipelines using SQL, Python, and PySpark. Build solutions that accommodate... ...Preferred Experience Experience with AWS and/or Azure cloud environments. Exposure...Amazon Web ServiceHarnham
Dallas, TX2 days ago- ...way. Critical Skills (Must-Have) Advanced PySpark Optimization: Beyond simple... ...currently hitting performance ceilings. AWS Data Ecosystem (EMR & Glue): Practical, production... ...considered "table stakes" for a modern engineer in this environment. Consultative Mandate...Amazon Web Service
InRhythm Inc
Dallas, TX1 day ago - ...Week onsite in Westlake, TX Our client seeks a Data Engineer to build a modern data platform using Apache Iceberg on AWS with Spark, Kafka, and Python. The role emphasizes... ...distributed engines such as Spark, Flink, or PySpark. Fluency in Python for data pipelines,...Amazon Web ServiceHourly pay
Eliassen Group
Dallas, TX1 day ago $114.38k - $185.4k
...Sr. Data Engineer CVS Shared Services Resources LLC, a CVS Health company, is hiring for the following... ...technologies: Azure, Amazon Web Services (AWS), or Google Cloud Platform (GCP); SAS or SQL programming languages; Spark, PySpark, or Scala; Machine learning, statistical...Amazon Web ServiceHourly payFull timeTemporary workWork at officeRemote workOak St. Health
Irving, TX5 days ago$128.25k - $266.88k
...Us The Mail Analytics Infrastructure & Data Engineering team at Yahoo is the backbone of Yahoo Mail... ...Mastery: Deep expertise in SQL, PySpark, or Scala. Scale: Proven track record of... ...at least one major cloud provider (GCP, AWS, or Azure) and a strong command of GitOps...Amazon Web ServiceWork at officeFlexible hoursYahoo Holdings Inc.
Richardson, TX1 day ago- ...Lead Data Engineer, Technology | Data & AnalyticsWe're looking for a Lead Data Engineer to own... ...complex, high-impact data initiatives using AWS, Snowflake/Redshift, Spark, and modern... ...programming skills in Python and/or PySpark.Experience with cloud data warehouses like...Amazon Web Service
RIT Solutions
Dallas, TX9 hours ago - ...We’re looking for a Lead Data Engineer to help design, build, and scale a modern enterprise data platform on AWS. This is a hands‑on role for someone who enjoys solving complex... ...architectures. Strong skills in Python, PySpark/Scala, and advanced SQL. Proven experience...Amazon Web Service
Harnham
Dallas, TX1 day ago $141.34k - $144.2k
...Data EngineerWe're building a world of health around every individual... ...role in Irving, TX: Data Engineer to analyze data engineering problems... ...: Azure, Amazon Web Services (AWS), or Google Cloud Platform (... ...PowerBI or Tableau; Spark, PySpark, or Scala; Extract/Transform/Load...Amazon Web ServiceFull timeWork at officeLocal areaRemote workCVS Health
Irving, TX1 day ago- ...Lead Data Engineer Data team builds and operates the data marts and pipelines that power HR... ...Engineering, with strong hands-on Spark (PySpark/Scala), SQL, and Python. • Experience... ...operating ETL/ELT pipelines on cloud platforms (AWS EMR/S3, Databricks, or equivalent);...Amazon Web Service
United IT
Dallas, TX4 days ago - ...advanced technologies such as AI, cloud, and data-led innovation to help our clients... ...Required Skill and Experience• Experience with AWS (EC2, ECS, ECR, EFS, EMR, Lambda, Step functions... ...Skill and Experience• Experience with Pyspark• Experience with Fast API/Flask frameworks...Amazon Web ServiceFull timeTemporary workRelocation
- ...Synchrony Corp. Our client is looking for Sr. Data Engineer . Please find the Job description below... ...expertise in Fivetran, Snowflake, Dbt, AWS Glue, Python, SQL, and cloud-native data... .../ELT workflows using Dbt, AWS Glue and PySpark. Build and optimize data lake...Amazon Web ServiceFull time
Siri InfoSolutions Inc
Dallas, TX3 days ago - ...: ~ We are seeking a highly skilled Senior Data Engineer with expertise in Fivetran, Snowflake, Dbt, AWS Glue, Python, SQL, and cloud-native data engineering... ...Develop ETL/ELT workflows using Dbt, AWS Glue and PySpark. Build and optimize data lake solutions on...Amazon Web ServiceFull time
Siri InfoSolutions Inc
Dallas, TX4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - AWS & PySpark. Be the first to apply!
Related searches
- data center engineer Dallas, TX
- senior cloud data engineer Dallas, TX
- data infrastructure engineer Dallas, TX
- sr information security engineer Dallas, TX
- hadoop big data developer Dallas, TX
- data visualization developer Dallas, TX
- remote data engineer Dallas, TX
- etl data engineer Dallas, TX
- data engineer analytics Dallas, TX
- sr data engineer Dallas, TX