Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer (Spark/Scala/Python)

Omni Inclusive

Senior Technical Lead with apache spark,scala,python

Job Summary

The Senior Technical Lead will be responsible for leading technical teams and projects related to apache spark, scala, and python. The role involves overseeing the design, development, and implementation of scalable and efficient solutions using these technologies.

Key Responsibilities
1. Lead technical teams in the design and implementation of solutions using apache spark, scala, and python
2. Provide technical expertise and guidance to team members in resolving complex technical issues
3. Collaborate with stakeholders to gather requirements and define project scope
4. Ensure adherence to best practices in coding, testing, and deployment processes
5. Conduct code reviews and performance optimization activities
6. Troubleshoot and debug technical issues to ensure seamless project delivery
7. Stay updated with the latest trends and advancements in apache spark, scala, and python technologies
8. Mentor team members and facilitate knowledge sharing within the team

Skill Requirements
1. Strong proficiency in apache spark, scala, and python
2. Experience in leading technical teams and projects
3. Excellent problem-solving skills and ability to think critically
4. Solid understanding of software development life cycle
5. Good communication and interpersonal skills
6. Ability to work effectively in a collaborative team environment
7. Strong organizational and leadership abilities
8. Experience in performance tuning and optimization techniques
9. Knowledge of big data concepts and tools is a plus

Certifications : Relevant certifications in apache spark, scala, or python are a plus
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer (Spark/Scala/Python) in Houston, TX vacancy
  • A leading technology firm in the United States is seeking a Senior Technical Lead to oversee technical teams in the development of scalable solutions using Apache Spark, Scala, and Python. The ideal candidate will be proficient in these technologies and have experience... 
    Python

    TechDigital Group

    Houston, TX
    5 days ago
  •  ...Data Engineering Leader Key skill: DWH, ETL, Pyspark, ADB, ADF 10+ years of experience in data engineering, with at least 2 years...  ...and unstructured data at scale. Proficient in SQL, Python, and Spark-based data processing. Experience with CI/CD tools (e.g... 
    Python

    E-Solutions

    Houston, TX
    5 days ago
  •  ...Data Engineer Qualifications Required in the Job: Advanced working SQL...  ...to SaaS solutions), Hadoop, Spark, Kafka, Linux scripting, etc....  ...function scripting languages: Python, Java, etc. Data Engineer...  ...services (EC2/S3/etc.). Will be the lead on this team to provide... 
    Python

    Software Technology Inc

    Houston, TX
    1 day ago
  •  ...Data Engineer Develop and deploy production-grade services, and data infrastructure emphasizing performance, scalability, and self...  ...Expertise in of at least two of the following languages - Python, Go, Scala, Java Experience in handling Large Scale Time Series data... 
    Python

    Samprasoft

    Houston, TX
    1 day ago
  • Overview Title: Lead Data Engineer (AWS Lakehouse) Location: Dallas, TX or Houston, TX (hybrid) Duration: Direct Hire Work Requirements...  ...and performance characteristics Develop pipelines using Python, PySpark, and Spark on AWS Glue and/or EMR Orchestrate workloads using... 
    Python
    Work at office
    Local area
    Remote work
    Flexible hours

    INSPYR Solutions

    Houston, TX
    2 days ago
  •  ...Spark Developer Job Location: Houston, TX Position Type: Contract...  ...skills: Experience with Scala (or Java + another functional...  ...to open-source big data projects (please provide link...  ...experience Experience with Python Experience with HBASE or other... 
    Python
    Contract work
    For contractors

    Georgia IT Inc

    Houston, TX
    1 day ago
  • A leading data platform company is seeking a Technical Lead, Spark (Java) to design and enhance complex data processing systems...  ...strong expertise in Java, Scala, or Python with 8-10+ years of...  ...pivotal role in advancing data engineering solutions, all while enjoying... 
    Python
    Flexible hours

    Cloudera

    Houston, TX
    2 days ago
  •  ...Tech Lead, Data & Inference Engineer A fast moving and venture backed advertising technology startup based...  ...query optimization on large datasets) and Python skills. ~ Hands-on experience with distributed data technologies (Spark, Flink, Kafka) and modern orchestration... 
    Python
    Full time

    Catalyst Labs, LLC

    Houston, TX
    5 days ago
  • $140k - $155k

     ...The American Red Cross is seeking a Lead Data Engineer to enhance data capabilities and implement a modern data architecture. The role involves...  ..., 10+ years of experience, and expertise in AWS, SQL, and Python. The position offers a remote working model with occasional in... 
    Python
    Remote work

    American Red Cross

    Houston, TX
    3 days ago
  •  ...SAP, InforEAM, and development of additional data products in support of the Surface Platform....  ...experience is a HARD requirement) Preferred Python (Java, Scala is also acceptable but not required) Preferred PySpark (Spark Scala is also acceptable) SQL (Understand... 
    Python

    Samprasoft

    Houston, TX
    2 days ago
  •  ...Job Description: Lead Data Engineer – Enterprise Data & Analytics Team Overview The American Red Cross is seeking a Lead Data Engineer to modernize...  ...writing and optimization for large‑scale, complex datasets. Python programming. AI‑assisted tools for code development, testing,... 
    Python
    Temporary work
    Work experience placement
    Work at office
    Local area
    Remote work
    Visa sponsorship
    Work visa
    Flexible hours
    Weekend work

    American Red Cross

    Houston, TX
    3 days ago
  •  ...NAVA Software solutions is looking for a Lead Data Engineer Details: Lead Data Engineer /Integrations Location: Houston TX -...  ...consultative and/or IT capacity. ~5+ years of demonstrated use of Python, Java, C#, or high-level programming language. ~5+ years... 
    Python
    Full time
    Shift work

    Nava Software Solutions

    Houston, TX
    5 days ago
  •  ...Senior Data Engineer - (100% Remote) Fully Remote: Work from anywhere with flexible hours...  ...and optimize workflows with Apache Spark and Scala for batch and streaming data processing...  ...Preferred Skills: Knowledge of Python for data engineering tasks.... 
    Python
    Remote work
    Flexible hours

    Elios, Inc.

    Houston, TX
    1 day ago
  • A leading technology solutions provider is seeking a hands-on Lead Data Engineer to modernize an enterprise data platform using AWS Lakehouse architecture. This role focuses...  ...with AWS data services, and proficiency in Python. This position offers a hybrid work environment... 
    Python

    INSPYR Solutions

    Houston, TX
    2 days ago
  •  ...Data, Analytics & Ai Team Lead At GHD, we don't just believe in the power of commitment, we live and...  ...expertise in data architecture, data engineering, and advanced analytics. The Data,...  ...Expertise Strong proficiency in Python, SQL, and modern data platforms and... 
    Python
    Work at office
    3 days per week

    GHD

    Houston, TX
    1 day ago
  • $106.9k - $176.5k

     ...working world. Technology – Data and Decision Science – Data Engineering – Senior We are...  ...business objectives. Lead end-to-end data pipeline...  ...Databricks and experience with Spark for big data processing....  ...in languages such as Python, Scala, or SQL. Experience with... 
    Python
    Summer holiday
    Flexible hours

    EY

    Houston, TX
    5 days ago
  •  ...possible. Are you ready to lead in this new era of technology...  ...for a Sr. Consultant, Data Engineer to join our growing team of...  ...Exadata, Netezza, SQL Server, Spark) ~ Experience building ETL...  ...are nice to have as well) ~ Python Scripting, Scala is required. ~ Ability to... 
    Python

    IBM Computing

    Houston, TX
    1 day ago
  •  ...opportunities for success. We are currently seeking an experienced Data Engineer to join the Big Dat and Advanced Analytics department. The...  .... ~5+ years of in-depth programming experience in Python and SQL. ~5+ years in software development lifecycle experience... 
    Python

    Enterprise Products

    Houston, TX
    4 days ago
  •  ...Data Engineering Lead Manages one or more data engineering projects of moderate complexity. Leads or acts...  ...Subversion). Open Source Frameworks (Apache Spark, Hadoop etc.) Strong design and coding skills (e.g. Python, Scala, JavaScript). Experience with reporting/visualization... 
    Python
    Daily paid
    Local area

    Samprasoft

    Houston, TX
    2 days ago
  •  ...Data Engineering Lead Manages one or more data engineering projects of moderate complexity. Leads or acts as key team member in...  ...including Postgres and Cassandra. Experience with object-oriented/object function scripting languages: Python, SQL, Scala, Spark-SQL... 
    Python

    Samprasoft

    Houston, TX
    1 day ago
  •  ...is looking for a Principal Data Engineer Details: Principal...  ...such as Databricks, Apache Spark, Kafka, Flink, AWS Glue or...  ...analytical models, leveraging Python, scala or R for data processing...  ...demonstrating to stakeholders. Leads one or more data projects... 
    Python
    Full time

    Nava Software Solutions

    Houston, TX
    5 days ago
  •  ...people to transform complex data into clear and actionable...  ...enterprises. Business Area: Engineering Seniority Level: Mid-...  ...Cloudera is seeking a Technical Lead, Spark (Java) with strong...  .... Develop new features in Scala/Java/Python on modern platforms. Gain... 
    Python
    Work from home
    Flexible hours

    Cloudera

    Houston, TX
    2 days ago
  • $125.5k - $230.2k

     ...working world. Technology – Data and Decision Science – Data Engineering – Manager We are...  ...of Data Engineering to lead our team in designing and...  ...Databricks and experience with Spark for big data processing....  ...in languages such as Python, Scala, or SQL. Excellent... 
    Python
    Summer holiday
    Flexible hours

    EY

    Houston, TX
    5 days ago
  • $99k - $232k

     ...Specialty/Competency: Data, Analytics & AI...  ...in data and analytics engineering focus on leveraging advanced...  ...You are expected to lead with integrity and...  ...programming languages such as Python, Java, or Scala - Familiarity with...  ...like Hadoop, Spark, or Kafka is a plus... 
    Python
    Full time
    H1b

    PwC

    Houston, TX
    5 days ago
  • A leading technology firm is looking for a Senior Data Engineer to join their fully remote team. The ideal candidate will have over 5 years of experience in data...  ...engineering, with strong expertise in Apache Spark and Scala. Responsibilities include designing scalable data... 
    Remote work

    Elios, Inc.

    Houston, TX
    4 days ago
  •  ...Google Cloud Data Engineer Are you ready to step up to the New and...  ...level? Join and help transform leading organizations and...  ...combination with 3rd parties - Spark, Cloud DataProc, Cloud Dataflow...  ...data architecture, using Java, Python, Scala etc. Minimum 1 year of architecting... 
    Python
    Work experience placement

    ClifyX

    Houston, TX
    1 day ago
  •  ...Senior Data Engineer Key Accountabilities: Architects, designs, implements and maintains...  ...experience with: Databricks and using Spark for data processing Configuring Delta Lake on Azure Databricks Python, Scala, SQL C#, ASP.NET, MVC,.NET Core,.Net... 
    Python

    Software Technology Inc

    Houston, TX
    1 day ago
  •  ...Day In The Life As a Data Engineering Solutions Lead within the Trailblazer initiative...  ...such as Apache Spark, Hadoop, and Kafka. Work...  ...analytical models, leveraging Python or R for data processing scripts...  ...engineering (Python, Java, Scala) Required Hands on RDBMS... 
    Python
    Work at office

    The Friedkin Group

    Houston, TX
    2 days ago
  •  ...Azure Data Engineer Houston TX - Onsite from day 1 Project Overview: New project...  ...for Sapient, Microsoft. Could lead to 200+ more applications for us to work...  ...certified preferred or AWS certified • Scala or Python or Spark, #C/.NET- working knowledge of one of... 
    Python

    Netorbit

    Houston, TX
    1 day ago
  •  ...Job Title: ADF Data Engineer Location & Reporting Address: Houston, TX 77079 Duration:...  ...Mandatory skills: ADF, Azure Databricks, Python, SQL Detailed Job Description: Experience...  ...Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, cluster configuration and... 
    Python

    eTeam

    Houston, TX
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer (Spark/Scala/Python). Be the first to apply!