Sr Data Engineer with Python & Databricks
Phaxis LLC
Job Description
Job Description
Salary is 140k to 170k bonus
This role focuses on building efficient data pipelines, implementing best practices in data engineering, and ensuring system reliability and performance. The Data Engineer works closely with data scientists, analysts, and software engineers to deliver optimized data solutions for both batch and real-time processing.
Key Responsibilities- Design, develop, and maintain end-to-end data pipelines and ETL/ELT workflows using Python, SQL, and modern orchestration tools (e.g., Airflow, dbt).
- Architect and optimize data storage solutions, including data lakes and data warehouses, using Databricks, Delta Lake, and cloud-native services.
- Build scalable data processing solutions leveraging Databricks notebooks, jobs, and clusters for both batch and streaming data workloads.
- Develop and manage Databricks workflows using Spark (PySpark, SQL, or Scala) to transform, cleanse, and aggregate large datasets.
- Design, develop, and implement complex data integrations across Databricks, cloud platforms (AWS, Azure, or GCP), enterprise applications, APIs, and modern data ecosystems to enable reliable, scalable data exchange and processing.
- Implement data quality checks, schema validation, and monitoring to ensure data accuracy and reliability.
- Optimize Databricks cluster configurations and job performance to minimize cost and maximize throughput.
- Collaborate with DevOps teams to automate deployments, CI/CD pipelines, and infrastructure-as-code (IaC) for data systems.
- Bachelor's or Master's degree in Computer Science, Data Engineering, or a related technical field.
- Advanced proficiency in Python and SQL for data manipulation, transformation, and automation.
- Deep experience with Databricks, including Spark optimization, Delta Lake management, job orchestration, and workspace administration.
- Strong understanding of distributed data processing, partitioning strategies, and performance tuning in Databricks and Spark.
- Demonstrated experience designing and implementing enterprise-scale data integrations across Databricks, cloud platforms (AWS, Azure, or GCP), enterprise applications, APIs, and modern data ecosystems.
- Hands-on experience with cloud platforms (AWS, Azure, or GCP) and their data ecosystems (e.g., S3, ADLS, BigQuery, Snowflake).
Vacancy posted 24 days ago
Similar jobs that could be interesting for youBased on the Sr Data Engineer with Python & Databricks in Edison, NJ vacancy
- ...Job Description: Lead Data Engineer (GCP Dataproc & BigQuery) Location:... ...skilled Lead Data Engineer with deep expertise in Google Cloud... .... ~ Strong experience with Python, PySpark, or Spark-based data... ...Experience with Snowflake, Databricks, or other modern cloud analytics...DatabricksFull time
- Senior Big Data Engineer Location: New Jersey (hybrid) Duration: 6+ Months... .... Your expertise in Java, Python, Spark cluster management, data... ..., and knowledge of Databricks and Delta Lake will be essential... ...Python, and Spark. Collaborate with data scientists, analysts, and...DatabricksSenior
- ...comWe are seeking an experienced Data Engineer to design, build, and... ...will have strong expertise in Python, SQL, Spark, ETL processes, and... ...data ingestion pipelines.Work with structured and unstructured datasets... ...(Hive, HDFS).Experience with Databricks.Experience with Apache Kafka....Databricks
$100k - $115k
DescriptionKforce has a client that is seeking a Data Engineer - Guidewire. This position is on-site... ...client is looking for a Data Engineer with experience in the Property and Casualty... ...optimize data solutions utilizing Azure Databricks, Azure Data Factory, Apache Spark, and...Databricks- ...Azure, AWS, or Google Cloud. Data Engineering: Strong understanding of data warehousing... .... Programming: Experience with programming languages like Python, Java, or Scala. Cloud Data... ...experience with Cloud Data Platform like Databricks (preferred), Snowflake, Redshift,...Databricks
- ...Operations to shift from traditional BI to a modern data foundation. You will design scalable data... ...operations. The role emphasizes collaboration with supply chain, QA, and manufacturing stakeholders, using Databricks, Python, React/Streamlit, and GitHub for robust...DatabricksSeniorShift work
- Databricks Engineer Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days’ work from office) Duration... ...: This position is for a Cloud Data / Reporting engineer with a background in SQL and data... ...Leverage SQL, programming language (Python or similar) and/or ETL Tools (Azure...DatabricksWork at office
- AWS Data Engineer With Databricks And DBT Job Location: Iselin, NJ Skill Cluster: Data-AWS Glue with EMR, Spark, Redshift, Kinesis, S3, AWS Native... ...Build Tool). Write complex and efficient SQL queries and Python/PySpark code for data manipulation, transformation, and validation...DatabricksLocal area
- ...major bank in Central NJ, is looking for hands-on Cloud Data Engineer with strong Python, SQL, ETL. This role is hybrid commute (1/2 days per... ...integrity. • Design, develop, and deploy Spark program in Databricks environment to process and analyze large volumes of data...DatabricksContract work2 days per week1 day per week
$86.8k - $160.7k
Remote position with the ability to travel to our NJ and NY locations up to 25% of the time. Position Summary : Data Engineer with previous experience in the insurance domain, design,... ...years of hands-on experience with Azure Databricks, Azure data factory including...DatabricksWork experience placementWork at officeRemote work- ...TitleSenior Director- Data EngineeringAbout your role... ...lead talented data engineering and development teams to... ...data products that align with Fiserv's strategic... ...years of experience in Python, Java, or similar programming... ....Experience with Databricks, including Delta Lake,...DatabricksSeniorFull timeTemporary workH1bWork at officeMonday to Friday
- Role : Data Platform engineer - Azure admin Location : Iselin, NJ (Hybrid) Contract Skills... .... Experienced working on Databricks workspaces, workload optimisation... ...automating tasks through scripting (Bash, Python, Powershell). Experience with configuration management (e.g....DatabricksContract work
$160k - $200k
...Join Mizuho as a Cloud Data Engineer! In this role, you will be responsible for overseeing... ...domain teams, ensuring alignment with enterprise cloud data standards and... ...initiatives. Strong expertise in Python, PySpark, SQL, Databricks, Snowflake, and cloud-based data...DatabricksWork at officeLocal areaRemote workWorldwide$90k - $130k
...enable businesses to operate with greater efficiency and effectiveness... ...the Role We're hiring a Data Engineer to join our team in the US.... ...pipelines Strong Python skills used for real pipeline... ...Hands-on experience with Databricks or Microsoft Fabric (either is...Databricks- ...! We are seeking a Lead Data & AI Engineer to design, build, and advance... ...: Work directly with stakeholders across Finance,... ...or GCP. ~ Strong SQL and Python development skills. ~ Practical... ...technologies such as Snowflake, Databricks, Microsoft Fabric, BigQuery,...DatabricksLocal area
- ...Edison, NJ is looking for a Data Engineer to join the team and grow.... ...ingestion. Write SQL and Python scripts for tasks. Run and... ...Understanding of core data concepts Databricks, Spark, and cloud technology... ...a plus Familiarity with Git or other version control...Databricks
$166.75k
...ingestion, SCD processing, data quality assertion, and... ...IaC managing all Databricks and AWS resources. Execute... ...pipelines in parallel with new PySpark/Databricks... ...Skills required: AWS, Python, PySpark, Databricks, Terraform... ..., Technology, or Engineering (any) with 5 years of experience...DatabricksFull timeShift workDay shift- ...Description Job Description SR AWS Data Engineer – SQL | S3/Redshift | Glue | Databricks - 11 Exp = New Jersey (Hybrid –... ...Seeking an AWS Data Engineer with strong hands on experience in SQL... ...and pipelines Python/PySpark — Hands on programming...DatabricksContract workRemote work
- ...Description Mandatory Skills: Python, AWS Devops Experience: 1... ...-on professional software engineering and technical leadership... ...of architecting distributed data processing systems, schema design... ...: Expert-level proficiency with Databricks, DBT and Apache Airflow....Databricks
- ...Associate Director to lead digital operations by building modern data foundations, automation, and AI-enabled workflows for pharmaceutical supply chain data. You will design data pipelines with Databricks, Python, and Dataverse, develop custom WebApps, and implement agentic...Databricks
$115.2k - $174k
...transformation, modern engineering, and technical empowerment... ...how real-time data flows through our global supply chain.As our Sr. Data Engineer, you are... ...engineering, working side-by-side with our Data Architect to build... ...blockers across Python, SQL, Snowflake, and dbt...SeniorHourly payLocal areaRelocation- ...Job Description Job Description Position: Data Engineer -Snowflake Python (10 years exp) Location: Sunnyvale, CA/ Austin, TX (Onsite) FTE/... ...databases like Snowflake, Oracle, Postgres Experience with Kafka and Redis Programming experience in building high...Hourly payContract workWork experience placement
- Sr. Data Engineer Data Engineers who can build scalable, reliable data pipelines and infrastructure, requiring mastery in SQL, Python/Scala, cloud platforms (AWS), and big data technologies (Spark,... ...data-driven solutions. Collaborate with cross-functional teams to...Senior
- ...Job Title: Data Engineer Reference ID: 24-04727 Location: Beaverton, MA Duration:8 months... ...Preferred platform experience of working either in Databricks/AWS for building big data pipelines. ~ Experience with source control tools such as GitHub and...DatabricksContract workWork experience placementLocal areaWork from homeVisa sponsorshipWork visa
- ...Description Job Description Contract Cloud Data Engineer MUST be local to Iselin, NJ, Hybrid Schedule... ...Develop and deploy Spark/PySpark programs and Databricks jobs for big data workloads. Write robust Python and SQL code; perform code reviews and troubleshoot...DatabricksContract workLocal area
$150.7k - $251.2k
...all in to shape your future with confidence. We’ll help you... ...and experienced Manager of Data Engineering to lead our team in designing... ...solutions with a strong focus on Databricks. The ideal candidate will... ...skills in languages such as Python, Scala, or SQL. Familiarity...DatabricksImmediate start$96k - $128k
...and one of the largest clinical trial data sets in the industry. More than 1 million... ..., and governance. As a Senior Data Engineer, you will expand this mission by reinforcing... ...features. We predominantly use Python and operate with a GitOps-driven mindset, running short...SeniorHourly payWork at officeLocal areaFlexible hours$110k - $120k
...Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, servicing an array of noteworthy... ..., our company has been honored with multiple employer awards, recognizing... ...scalable data solutions using Databricks, AWS Cloud, Python, AWS Lambda, Amazon S3, and MongoDB...DatabricksFull timeTemporary workFlexible hours$102k - $170k
...responsible to our employees who work with us throughout the world. We... ...their merit. Job Function: Data Analytics & Computational... ...Job Sub Function: Data Engineering Job Category: Scientific/Technology... ...in Azure, Microsoft Fabric, Databricks, Power BI, Data Mesh, Data...DatabricksTemporary workLocal areaImmediate start- Role : Cloud Data Developer Location : Iselin, NJ (Hybrid) Key Responsibilities: Design... ...scalable data pipelines using Azure Databricks and Scala . Develop and optimize queries... ...PostgreSQL and Elasticsearch . Collaborate with DevOps teams to build and maintain CI/CD...Databricks
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Sr Data Engineer with Python & Databricks. Be the first to apply!
Related searches
- data engineering intern summer Edison, NJ
- senior living Edison, NJ
- senior manager customer operations Edison, NJ
- senior full stack web developer Edison, NJ
- senior magento developer Edison, NJ
- senior software design engineer Edison, NJ
- senior international tax analyst Edison, NJ
- senior manager accenture Edison, NJ
- senior accountant remote Edison, NJ
- senior software engineer Edison, NJ





