Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Sr Data Engineer with Python & Databricks

Phaxis LLC

Job Description

Job Description

Salary is 140k to 170k bonus

This role focuses on building efficient data pipelines, implementing best practices in data engineering, and ensuring system reliability and performance. The Data Engineer works closely with data scientists, analysts, and software engineers to deliver optimized data solutions for both batch and real-time processing.

Key Responsibilities
  • Design, develop, and maintain end-to-end data pipelines and ETL/ELT workflows using Python, SQL, and modern orchestration tools (e.g., Airflow, dbt).
  • Architect and optimize data storage solutions, including data lakes and data warehouses, using Databricks, Delta Lake, and cloud-native services.
  • Build scalable data processing solutions leveraging Databricks notebooks, jobs, and clusters for both batch and streaming data workloads.
  • Develop and manage Databricks workflows using Spark (PySpark, SQL, or Scala) to transform, cleanse, and aggregate large datasets.
  • Design, develop, and implement complex data integrations across Databricks, cloud platforms (AWS, Azure, or GCP), enterprise applications, APIs, and modern data ecosystems to enable reliable, scalable data exchange and processing.
  • Implement data quality checks, schema validation, and monitoring to ensure data accuracy and reliability.
  • Optimize Databricks cluster configurations and job performance to minimize cost and maximize throughput.
  • Collaborate with DevOps teams to automate deployments, CI/CD pipelines, and infrastructure-as-code (IaC) for data systems.
Qualifications
  • Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field.
  • Advanced proficiency in Python and SQL for data manipulation, transformation, and automation.
  • Deep experience with Databricks, including Spark optimization, Delta Lake management, job orchestration, and workspace administration.
  • Strong understanding of distributed data processing, partitioning strategies, and performance tuning in Databricks and Spark.
  • Demonstrated experience designing and implementing enterprise-scale data integrations across Databricks, cloud platforms (AWS, Azure, or GCP), enterprise applications, APIs, and modern data ecosystems.
  • Hands-on experience with cloud platforms (AWS, Azure, or GCP) and their data ecosystems (e.g., S3, ADLS, BigQuery, Snowflake).
Vacancy posted 24 days ago
Similar jobs that could be interesting for youBased on the Sr Data Engineer with Python & Databricks in Edison, NJ vacancy
  •  ...Job Description: Lead Data Engineer (GCP Dataproc & BigQuery) Location:...  ...skilled Lead Data Engineer with deep expertise in Google Cloud...  .... ~ Strong experience with Python, PySpark, or Spark-based data...  ...Experience with Snowflake, Databricks, or other modern cloud analytics... 
    Databricks
    Full time

    Tredence Inc.

    Edison, NJ
    4 days ago
  • Senior Big Data Engineer Location: New Jersey (hybrid) Duration: 6+ Months...  .... Your expertise in Java, Python, Spark cluster management, data...  ..., and knowledge of Databricks and Delta Lake will be essential...  ...Python, and Spark. Collaborate with data scientists, analysts, and... 
    Databricks
    Senior

    Zortech Solutions

    Elizabeth, NJ
    1 day ago
  •  ...comWe are seeking an experienced Data Engineer to design, build, and...  ...will have strong expertise in Python, SQL, Spark, ETL processes, and...  ...data ingestion pipelines.Work with structured and unstructured datasets...  ...(Hive, HDFS).Experience with Databricks.Experience with Apache Kafka.... 
    Databricks

    Prolim Global

    Edison, NJ
    11 hours ago
  • $100k - $115k

    DescriptionKforce has a client that is seeking a Data Engineer - Guidewire. This position is on-site...  ...client is looking for a Data Engineer with experience in the Property and Casualty...  ...optimize data solutions utilizing Azure Databricks, Azure Data Factory, Apache Spark, and... 
    Databricks

    KForce

    Woodbridge, NJ
    11 hours ago
  •  ...Azure, AWS, or Google Cloud. Data Engineering: Strong understanding of data warehousing...  .... Programming: Experience with programming languages like Python, Java, or Scala. Cloud Data...  ...experience with Cloud Data Platform like Databricks (preferred), Snowflake, Redshift,... 
    Databricks

    Saxon Global

    Iselin, NJ
    1 day ago
  •  ...Operations to shift from traditional BI to a modern data foundation. You will design scalable data...  ...operations. The role emphasizes collaboration with supply chain, QA, and manufacturing stakeholders, using Databricks, Python, React/Streamlit, and GitHub for robust... 
    Databricks
    Senior
    Shift work

    Merck & Co.

    Rahway, NJ
    1 day ago
  • Databricks Engineer Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days’ work from office) Duration...  ...: This position is for a Cloud Data / Reporting engineer with a background in SQL and data...  ...Leverage SQL, programming language (Python or similar) and/or ETL Tools (Azure... 
    Databricks
    Work at office

    Zortech Solutions

    Iselin, NJ
    1 day ago
  • AWS Data Engineer With Databricks And DBT Job Location: Iselin, NJ Skill Cluster: Data-AWS Glue with EMR, Spark, Redshift, Kinesis, S3, AWS Native...  ...Build Tool). Write complex and efficient SQL queries and Python/PySpark code for data manipulation, transformation, and validation... 
    Databricks
    Local area

    E-Solutions

    Iselin, NJ
    1 day ago
  •  ...major bank in Central NJ, is looking for hands-on Cloud Data Engineer with strong Python, SQL, ETL. This role is hybrid commute (1/2 days per...  ...integrity. • Design, develop, and deploy Spark program in Databricks environment to process and analyze large volumes of data... 
    Databricks
    Contract work
    2 days per week
    1 day per week

    Mizuho Bank

    Iselin, NJ
    21 days ago
  • $86.8k - $160.7k

    Remote position with the ability to travel to our NJ and NY locations up to 25% of the time. Position Summary : Data Engineer with previous experience in the insurance domain, design,...  ...years of hands-on experience with Azure Databricks, Azure data factory including... 
    Databricks
    Work experience placement
    Work at office
    Remote work

    GREATER NEW YORK MUTUAL INSURANCE COMPANY

    Edison, NJ
    1 day ago
  •  ...TitleSenior Director- Data EngineeringAbout your role...  ...lead talented data engineering and development teams to...  ...data products that align with Fiserv's strategic...  ...years of experience in Python, Java, or similar programming...  ....Experience with Databricks, including Delta Lake,... 
    Databricks
    Senior
    Full time
    Temporary work
    H1b
    Work at office
    Monday to Friday

    Fiserv

    Berkeley Heights, NJ
    3 days ago
  • Role : Data Platform engineer - Azure admin Location : Iselin, NJ (Hybrid) Contract Skills...  .... Experienced working on Databricks workspaces, workload optimisation...  ...automating tasks through scripting (Bash, Python, Powershell). Experience with configuration management (e.g.... 
    Databricks
    Contract work

    United IT Solutions

    Iselin, NJ
    1 day ago
  • $160k - $200k

     ...Join Mizuho as a Cloud Data Engineer! In this role, you will be responsible for overseeing...  ...domain teams, ensuring alignment with enterprise cloud data standards and...  ...initiatives. Strong expertise in Python, PySpark, SQL, Databricks, Snowflake, and cloud-based data... 
    Databricks
    Work at office
    Local area
    Remote work
    Worldwide

    Mizuho Bank Ltd

    Iselin, NJ
    3 days ago
  • $90k - $130k

     ...enable businesses to operate with greater efficiency and effectiveness...  ...the Role We're hiring a Data Engineer to join our team in the US....  ...pipelines Strong Python skills used for real pipeline...  ...Hands-on experience with Databricks or Microsoft Fabric (either is... 
    Databricks

    GEP Worldwide

    Clark, NJ
    8 days ago
  •  ...! We are seeking a Lead Data & AI Engineer to design, build, and advance...  ...: Work directly with stakeholders across Finance,...  ...or GCP. ~ Strong SQL and Python development skills. ~ Practical...  ...technologies such as Snowflake, Databricks, Microsoft Fabric, BigQuery,... 
    Databricks
    Local area

    Conexus

    East Brunswick, NJ
    9 days ago
  •  ...Edison, NJ is looking for a Data Engineer to join the team and grow....  ...ingestion. Write SQL and Python scripts for tasks. Run and...  ...Understanding of core data concepts Databricks, Spark, and cloud technology...  ...a plus Familiarity with Git or other version control... 
    Databricks

    Phaxis LLC

    Edison, NJ
    24 days ago
  • $166.75k

     ...ingestion, SCD processing, data quality assertion, and...  ...IaC managing all Databricks and AWS resources. Execute...  ...pipelines in parallel with new PySpark/Databricks...  ...Skills required: AWS, Python, PySpark, Databricks, Terraform...  ..., Technology, or Engineering (any) with 5 years of experience... 
    Databricks
    Full time
    Shift work
    Day shift

    iPolarity

    Piscataway, NJ
    1 day ago
  •  ...Description Job Description SR AWS Data Engineer – SQL | S3/Redshift | Glue | Databricks - 11 Exp = New Jersey (Hybrid –...  ...Seeking an AWS Data Engineer with strong hands on experience in SQL...  ...and pipelines Python/PySpark — Hands on programming... 
    Databricks
    Contract work
    Remote work

    Han IT Staffing

    Iselin, NJ
    24 days ago
  •  ...Description Mandatory Skills: Python, AWS Devops Experience: 1...  ...-on professional software engineering and technical leadership...  ...of architecting distributed data processing systems, schema design...  ...: Expert-level proficiency with Databricks, DBT and Apache Airflow.... 
    Databricks

    TechDigital Corporation

    Rahway, NJ
    22 days ago
  •  ...Associate Director to lead digital operations by building modern data foundations, automation, and AI-enabled workflows for pharmaceutical supply chain data. You will design data pipelines with Databricks, Python, and Dataverse, develop custom WebApps, and implement agentic... 
    Databricks

    Merck

    Rahway, NJ
    2 days ago
  • $115.2k - $174k

     ...transformation, modern engineering, and technical empowerment...  ...how real-time data flows through our global supply chain.As our Sr. Data Engineer, you are...  ...engineering, working side-by-side with our Data Architect to build...  ...blockers across Python, SQL, Snowflake, and dbt... 
    Senior
    Hourly pay
    Local area
    Relocation

    Colgate-Palmolive

    Piscataway, NJ
    2 days ago
  •  ...Job Description Job Description Position: Data Engineer -Snowflake Python (10 years exp) Location: Sunnyvale, CA/ Austin, TX (Onsite) FTE/...  ...databases like Snowflake, Oracle, Postgres Experience with Kafka and Redis Programming experience in building high... 
    Hourly pay
    Contract work
    Work experience placement

    Tanisha Systems

    East Brunswick, NJ
    22 days ago
  • Sr. Data Engineer Data Engineers who can build scalable, reliable data pipelines and infrastructure, requiring mastery in SQL, Python/Scala, cloud platforms (AWS), and big data technologies (Spark,...  ...data-driven solutions. Collaborate with cross-functional teams to... 
    Senior

    E-Solutions

    Rahway, NJ
    1 day ago
  •  ...Job Title: Data Engineer Reference ID: 24-04727 Location: Beaverton, MA Duration:8 months...  ...Preferred platform experience of working either in Databricks/AWS for building big data pipelines. ~ Experience with source control tools such as GitHub and... 
    Databricks
    Contract work
    Work experience placement
    Local area
    Work from home
    Visa sponsorship
    Work visa

    Sunrise Systems

    Edison, NJ
    a month ago
  •  ...Description Job Description Contract Cloud Data Engineer MUST be local to Iselin, NJ, Hybrid Schedule...  ...Develop and deploy Spark/PySpark programs and Databricks jobs for big data workloads. Write robust Python and SQL code; perform code reviews and troubleshoot... 
    Databricks
    Contract work
    Local area

    Shulman Fleming & Partners

    Iselin, NJ
    21 days ago
  • $150.7k - $251.2k

     ...all in to shape your future with confidence. We’ll help you...  ...and experienced Manager of Data Engineering to lead our team in designing...  ...solutions with a strong focus on Databricks. The ideal candidate will...  ...skills in languages such as Python, Scala, or SQL. Familiarity... 
    Databricks
    Immediate start

    EY

    New Brunswick, NJ
    1 day ago
  • $96k - $128k

     ...and one of the largest clinical trial data sets in the industry. More than 1 million...  ..., and governance. As a Senior Data Engineer, you will expand this mission by reinforcing...  ...features. We predominantly use Python and operate with a GitOps-driven mindset, running short... 
    Senior
    Hourly pay
    Work at office
    Local area
    Flexible hours

    Medidata Solutions

    Iselin, NJ
    1 day ago
  • $110k - $120k

     ...Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy...  ..., our company has been honored with multiple employer awards, recognizing...  ...scalable data solutions using Databricks, AWS Cloud, Python, AWS Lambda, Amazon S3, and MongoDB... 
    Databricks
    Full time
    Temporary work
    Flexible hours

    Synechron

    Piscataway, NJ
    7 hours ago
  • $102k - $170k

     ...responsible to our employees who work with us throughout the world. We...  ...their merit. Job Function: Data Analytics & Computational...  ...Job Sub Function: Data Engineering Job Category: Scientific/Technology...  ...in Azure, Microsoft Fabric, Databricks, Power BI, Data Mesh, Data... 
    Databricks
    Temporary work
    Local area
    Immediate start

    Johnson and Johnson

    New Brunswick, NJ
    1 day ago
  • Role : Cloud Data Developer Location : Iselin, NJ (Hybrid) Key Responsibilities: Design...  ...scalable data pipelines using Azure Databricks and Scala . Develop and optimize queries...  ...PostgreSQL and Elasticsearch . Collaborate with DevOps teams to build and maintain CI/CD... 
    Databricks

    United IT Solutions

    Iselin, NJ
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Sr Data Engineer with Python & Databricks. Be the first to apply!