Big data Pyspark Developer

Futran Tech Solutions Pvt. Ltd.

Irving, TX

Big Data Pyspark Developer

Irving, Texas

Fulltime

Job Summary

Exp 6 Years Must have good technical experience and should be able to provide technical solutions for multiple modules in parallel on need basis and bring the task to closure on time
Unix SQL and Shell Scripting experience is a must have
Expertise in Designing and developing scalable Apache spark ETL based Data processing pipelines
Strong commandline knowledge in UnixLinux with Shell scripting using Bash Kornshell or Perl and File processing using awk scripts
Expertise in SQL querying and complex joins
Implementing comprehensive Spark based Data validation frameworks transforming large volumes of Financial data within the Project lifecycle
Expertise with complex Data workflows with Apache AirFlow managing task dependencies SLAs etc to ensure timely data delivery and corresponding automated validation controls
Strong Analytical skills and expertise on SparkSQL for Data analysis and validation ensuring the delivery of clean queryready datasets for business consumption
Expertise in Data quality checks and monitoring
Quality Engineering team where 70percent of effort will be for developing automation frameworks for testing Remaining 30percent effort will be on manual testing until its fully automated
Handson with Automation Framework Design for ETL and API
SME in Data Analysis Database testing Messaging queues
Experience with coding standards code reviews source management build processes CICD pipeline

Mandatory Skills: Apache Spark, Big Data Hadoop Ecosystem, Python, Python for DATA, SparkSQL

Apply

Vacancy posted 15 hours ago

Similar jobs that could be interesting for youBased on the Big data Pyspark Developer in Irving, TX vacancy

PySpark Data Engineer
$120k - $140k
...Job : PySpark Data Engineer Location : Irving, TX(Onsite) Fulltime Only Salary Range... ...you will be responsible for designing, developing, and maintaining data solutions for data... ...generation, collection, and processing in Big Data environment using predominantly...
Big data
Full time
Work experience placement
Diverse Lynx
Irving, TX
1 day ago
Sr. Pyspark Data Engineer
...Role: Sr. PySpark Data Engineer - Fulltime Location: Irving, TX Job Description... ...ideal candidate will have expertise in big data processing, ETL pipeline... ...scientists, and software engineers to design, develop, and optimize scalable data processing solutions...
Big data
Full time
SARIAN Co
Irving, TX
2 days ago
Data Engineer - Python/PySpark
$125k - $135k
...Data Engineer - Python/PySpark Location: Dallas/Irving TX (3 Days onsite/week) Fulltime Salary... ...shuffles, optimization). Experience with big data frameworks such as Hadoop and... ...into technical solutions. Design, develop, and maintain robust, scalable ETL/...
Big data
Full time
3 days per week
Diverse Lynx
Dallas, TX
4 days ago
Big Data Developer
...Position: Big Data Developer Location: Irving, TX Duration: Long Term Responsibilities: 1). Develop software using Hadoop technologies like Spark, Scala, Python, Hbase, Hive, Cloudera. 2). Fine tune application performance...
Big data
3B Staffing LLC
Irving, TX
4 days ago
Big Data Developer
...Skill: Big Data Developer Must Have Technical/Functional Skills: Bigdata Hadoop & ecosystem, Scala/Python, PySpark. Oracle PL/SQL, CI/CD, Excellent communication, Data Lake, Informatica, Teradata. Roles & Responsibilities: Strong experience...
Big data
Jconnect Inc
Dallas, TX
a month ago
Big Data Engineer in Irving TX
...Role: Big Data Engineer Location: Irving TX Duration: 5+ Months Job Description: Minimum 5+ years of experience in IT industry... ...Flume HBase and Zoo Keeper. Extensive experience working on Pyspark and Spark in performing ETL using Spark Core Spark-SQL and Real...
Big data
Work experience placement
Avani Technology Solutions
Irving, TX
15 days ago
Spark ETL Engineer - Big Data Pipelines & Validation
...LTM is seeking a Big Data Pyspark Developer in Irving, Texas, to design and develop scalable Apache Spark ETL pipelines, while managing complex Airflow workflows. The ideal candidate will have over 6 years of relevant technical experience. You will be responsible for...
Big data
LTM
Irving, TX
16 hours ago
Big Data Developer - II
...Big Data Engineer As a Big Data Engineer on the Google Cloud Platform the candidate will help design, develop, deploy, maintain code in Google Cloud Platform using services such as Big Query, Pub Sub, Data Flow, Data Fusion, Google Cloud Storage, Composer, and Looker...
Big data
Work experience placement
Samprasoft
Irving, TX
2 days ago
Senior Data Developer Tech Lead - Vice President
...Senior Data Developer Tech Lead - Vice President Working at Citi is far more than just a job... ...possess deep architectural expertise in big data technologies, including real-time data... ...and architectural knowledge of Python, PySpark, Unix, and SQL. ~ Familiarity with...
Big data
Citi
Irving, TX
5 days ago
Data Engineer
...Big Data Engineer Location: Irving, TX (Day 1 Onsite-Hybrid) Hexaware Primary Skills: Hadoop, Hive & Spark Framework Responsibilities... ...workflows. Troubleshoot and resolve issues related to PySpark applications and workflows. Understand source, dependencies...
Big data
InterSources
Irving, TX
1 day ago
Data Engineer - IAM
...Data Engineer – IAM Data Lake (Google Cloud Platform) Location... ...scalable data lake solutions, developing data ingestion pipelines, and... ...initiatives using modern cloud and big data technologies. The ideal... .... Develop and maintain PySpark-based data processing applications...
Big data
Contract work
Leading Utilities Organization
Irving, TX
1 day ago
IAM Data Lake -Data Engineer
...3 This position is focused on IAM Data Lake -Data Engineering responsibilities. We... ...Data Lake in Google Cloud Platform using big data tools and technologies. Experience with... ...2 - 4 Years Remove xms-USIT pyspark 1 4 - 6 Years Remove...
Big data
Mindlance
Irving, TX
4 days ago
Senior PySpark Developer
...Senior PySpark Developer Location: Irving, TX (Onsite) Duration: Full Time Job Description: As a Data Engineer, you will be responsible for designing, developing, and maintaining... ..., collection, and processing in Big Data environment using predominantly PySpark...
Big data
Full time
Immediate start
Relocation
JConnect Infotech
Irving, TX
1 day ago
Information Security - Information Security Engineer 3 - Contingent
...This position is focused on IAM Data Lake -Data Engineering responsibilities. We are seeking... ...Data Lake in Google Cloud Platform using big data tools and technologies. Experience... ...Hadoop Ecosystem Yes 2 2 - 4 Years xms-USIT pyspark Yes 1 4 - 6 Years EEO: "...
Big data
Mindlance
Irving, TX
5 days ago
Big Data (PySpark) Tech Lead
...I, Salman Shaikh would like to share a job opportunity as Big Data (PySpark) Tech Lead based in Irving, TX / Jacksonville, FL / Jersey... ...knowledge on majority of Hadoop and NoSQL databases as well. • Develop and execute data pipeline testing processes and validate...
Big data
Full time
Work visa
Syntricate Technologies
Irving, TX
2 days ago
Senior Java PySpark Developer
...Job Title: Senior Java PySpark Developer Location: Irving, TX (ONSITE) F2F Interview... ...We are seeking a seasoned Senior Lead Data Engineer with a dual-threat background in... .... • Hands-on experience with Big Data Environment. • Familiarity with CI...
Big data
Full time
AceStack LLC
Irving, TX
3 days ago
Senior Data Engineer - Vice President
...Senior Data Engineer - Vice President Citi is seeking a highly... ...scalable ETL/ELT pipelines using PySpark, Spark SQL, and Delta Lake on... ...processing, and analytics. Big Data Technologies: Work... ...processing for cost and efficiency. Develop and optimize robust data...
Big data
Citi
Irving, TX
5 days ago
Data Solutions Engineer
...Data Solutions Engineer Working at Citi is far more than just... ...the design and development of Big Data solutions. Partner with... ...analysts, and data scientists to develop robust Big Data pipelines in... ...into modern languages like PySpark and Scala for execution on Hadoop...
Big data
Relocation
Citi
Irving, TX
1 day ago
Applied Analytics Engineer
...optimizing, and maintaining robust data pipelines that ingest, clean,... ...for analytical readiness, and developing, rigorously testing, and... ...technologies such as Python, SQL, and PySpark. Undertake meticulous... ...~ Practical experience with big data processing frameworks such...
Big data
Full time
Vulcan Materials
Irving, TX
5 days ago
AIA COMMS -BIG DATA ENGINEER
...Job title COGNIZANT LOOKING FOR BIG DATA ENGINEER Job summary BigData Engineer Primary Skill Set • 4+ years of experience working on data engineering teams related to BigData and Teradata • Expert knowledge of SQL NoSQL and Python • Good experience with Apache Spark ecosystem...
Big data
Shift work
Avani Technology Solutions
Irving, TX
a month ago
Big Data Engineer
...Overview: Job Title : Big Data Engineer Location : Irving TX (hybrid role) (Only local candidates) Duration : 12 Months... ...standards, collaborate with Product Managers, Architects and Senior Developers to establish the physical application framework (e.g....
Big data
Work experience placement
Local area
Guru Schools
Irving, TX
3 days ago
Sr. AWS/big data developer
...Sr. AWS/Big Data Developer Excellent knowledge of Spark, Spark MLib, Scala, HBase, Hive. MUST HAVE knowledge of AWS environment - EC2, S3, EBS. Knowledge of Cloud Formation template is a plus. Good knowledge of Cloudera CDH. Experience of Unix Shell scripting is a...
Big data
ClifyX
Dallas, TX
4 days ago
Principal Data Platform Engineer Vice President
$125.76k - $188.64k
...Full Stack Python Developer We are seeking an exceptionally skilled and motivated... ..., scalable, and high-performance data solutions. The ideal candidate will... ...analysis role with extensive Python and/or big data expertise ~ Python/Pyspark Mastery: Expert-level capability in...
Big data
Full time
Citigroup Inc
Irving, TX
1 day ago
Senior Data Engineer
...Role description Job Title: Big DATA Developer with Python Work Location : Tampa,FL Job Description: Seeking a Senior Python Data Engineer with 3 to 5 years of experience skilled in Apache Spark and Python to design and implement scalable data processing...
Big data
LTM
Irving, TX
5 days ago
Data Scientist
...Data Scientist Location: Irving/Dallas area, Basking NJ, Atlanta, GA We are looking... ...You are experienced and handy at analyzing big data, preparing data for modeling, and... ...applications with hands-on experience in Python, PySpark, machine learning libraries, SQL, and...
Big data
Work experience placement
Samprasoft
Irving, TX
2 days ago
Data Scientist
...Job Title: Data Scientist Location: Irving, TX Job Description... ...(Pandas, NumPy, Scikit-learn, PySpark) • Experience with time-... ...detection • Experience with SQL, big data platforms (Spark, Hadoop)... ...across network domains • Develop KPIs and dashboards to track...
Big data
Night shift
United IT Solutions
Irving, TX
4 days ago
Principal Engineer - Data Science
...Responsibilities: Targeted Years of Experience : 5-7 years • Experience with Big Data technologies and ecosystem, data modeling tools • Experience with Project / Process management • Experience writing technical documentation • Familiarity with core data...
Big data
3B Staffing LLC
Irving, TX
1 day ago
Data Engineer
...Job Title: Data Engineer – MEM SQL Location: New Jersey / Irving, TX / Tampa, FL Job... ...platforms. Responsibilities: Design, develop, and maintain scalable data pipelines and... ...platforms (AWS / Azure / GCP) Knowledge of Big Data technologies (Spark, Kafka, etc.)...
Big data
Qode
Irving, TX
2 days ago
Information Security Engineer
...basis. Preferred Skills & Qualifications Experience with IAM Data Lake - Data Engineering responsibilities. Hands-on experience building Data Lake in Google Cloud Platform using big data tools and technologies. Experience with Hadoop/HDFS is highly...
Big data
Immediate start
Artech
Irving, TX
2 days ago
SPARK Data Onboarding Engineer- NJ
...PySpark Data Engineer We are seeking a skilled PySpark Data Engineer to join our team and... ...PySpark development skills, experience with big data technologies, and the ability to... ...Key Responsibilities: Design, develop, and test PySpark-based applications to...
Big data
Photon
Dallas, TX
1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big data Pyspark Developer. Be the first to apply!