Lead Databricks Engineer with Pyspark & Python Exp
Zortech Solutions
Role: Lead Databricks Engineer with Pyspark & Python Exp
Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office)
Duration: 6+ Months
Job Description:
Inperson interview is must
(Financial background is must)
Major Responsibilities:
Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office)
Duration: 6+ Months
Job Description:
Inperson interview is must
(Financial background is must)
Major Responsibilities:
- Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
- Design, develop, and deploy Databricks jobs to process and analyze large volumes of data.
- Collaborate with data engineers and data scientists to understand data requirements and implement appropriate data processing pipelines.
- Optimize Databricks jobs for performance and scalability to handle big data workloads.
- Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks.
- Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions.
- Demonstrated proficiency with Data Analytics, Data Insights
- Proficient writing SQL queries and programming including stored procedures and reverse engineering existing process
- Azure Synapse/Bigquery/Redshift is good to have.
- Perform code reviews to ensure fit to requirements, optimal execution patterns and adherence to established standards.
- 5+ years' - Strong experience in Finance / Banking industry - Capital markets, investment banking - Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo
- 10+ years - Enterprise Data Management
- 10+ years - SQL Server based development of large datasets
- 5+ years with Data Warehouse Architecture, hands-on experience with Databricks platform. Extensive experience in PySpark coding.
- Azure Synapse/Bigquery/Redshift experience is good to have
- 3+ years Python(numpy, pandas) coding experience
- Experience with Snowflake utilities such as SnowSQL and SnowPipe - good to have
- Experience in Data warehousing - OLTP, OLAP, Dimensions, Facts, and Data modeling.
- Previous experience leading an enterprise-wide Cloud Data Platform migration with strong architectural and design skills
- Capable of discussing enterprise level services independent of technology stack
- Experience with Cloud based data architectures, messaging, and analytics
- Superior communication skills
- Cloud certification(s)
- Any experience with Regulatory Reporting is a Plus
- Minimally a BA degree within an engineering and/or computer science discipline
- Master's degree strongly preferred
Vacancy posted 13 hours ago
Similar jobs that could be interesting for youBased on the Lead Databricks Engineer with Pyspark & Python Exp in New York, NY vacancy
- ...Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Duration: 6+ Months Job Description: Inperson interview is must (Financial background is must)...DatabricksPythonWork at office
Zortech Solutions
New York, NY1 day ago - ...Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Job Description: Inperson interview is must, Expenses will be paid for in-person if not local (Financial background...DatabricksPythonWork at officeLocal area
Zortech Solutions
New York, NY13 hours ago - ...Azure Databricks Architect Location:... ...Need someone with strong Databricks... ...week Minimum Exp level: 15+ years... ...experience in PySpark coding and Snowflake... ...~3+ years Python (NumPy, Pandas)... ...Previous experience leading an enterprise-... ...with data engineers and data scientists...DatabricksPython
Zortech Solutions
New York, NY2 days ago - ...Azure Databricks Architect With PySpark And Snowflake Exp Location: NYC, NY-100% day one onsite Duration: 6+ Months Job Description: This is a... ...analyze large volumes of data. Collaborate with data engineers and data scientists to understand data requirements...Databricks
Zortech Solutions
New York, NY1 day ago - ...Role: Databricks Engineer with PySpark exp Location: NYC, NY-Hybrid (3 days a week from office) Duration: 6+ Months Job Description: Total Exp level : 8 to 14 years Skills: Great hands-on experience with Databricks...DatabricksWork at office3 days per week
Zortech Solutions
New York, NY3 days ago - ...Lead Software Engineer We have an opportunity to impact your... ...high-quality code in Python and/or Java; conduct... ...operationalization in collaboration with Data Science and SRE/... ...~ Experience with PySpark, Kafka, Terraform,... ...platforms such as Databricks for data engineering...DatabricksPython
Chase
Jersey City, NJ2 days ago - ...analyze master data across domains. Expert proficiency in SQL , PySpark, Python for querying, validating, and transforming large datasets in Databricks. Exposure in querying MDM systems via REST APIs and working with cross-system data integrations. Ability to support SIT/UAT...DatabricksPythonFull time
Factspan Analytics
New York, NY3 days ago - ...Overview: **Mandatory skills / Required Experience**- Azure (Bigdata Azure), DataBricks , Scala, Java, Spark, Pyspark, Python,**Job Description: -**- 11+ years of experience with Azure Data Factory and Databricks with Big Data ecosystem tools (e.g. Hadoop, Hive) and...DatabricksPython
Guru Schools
New York, NY13 hours ago - ...Cards Integration Lead (Data, PySpark) Location NYC Domain Knowledge... ...Full Stack - Java 1.8 Data Engineering skills - Python, PySpark or Java on Spark. Experience on Databricks NO-SQL Database... ...relevant AWS tools Experience With Git and Github (good to have...DatabricksPython
My3Tech Inc
New York, NY1 day ago - ...Software Engineer Java + Data (PySpark) Georgian office; New York, NY, United States... ...development company with over two decades of experience... ...backend services using Java and Python Build and optimize data... ...Nice-to-have Databricks OR AWS EMR OR Hadoop Search...DatabricksPythonContract workWork at office
Lineate
New York, NY2 days ago - ...Lead Data Engineer - Palantir & PySpark (Lead Data engineer role where experience should... ..., and governance in line with industry and client standards... ...be proficient in SQL, Python, and working with large datasets... ...(e.g., S3, Snowflake, Databricks). Should be...DatabricksPythonRemote work
VBeyond
Jersey City, NJ4 days ago - ...Lead Software Engineer We have an opportunity to impact your career and provide... ...leading technology products. With a strong SRE mindset, you... ...hands-on expertise in Java, Python, and big data technologies (e.g., Spark/PySpark, Databricks, Snowflake) Hands-on practical...DatabricksPython
Chase
Jersey City, NJ2 days ago $190k - $245k
...Lead Software Engineer Altana is the network for trusted trade... ...You'll work closely with our Data Scientists on... ...Experience with Python Machine Learning toolsets... ...Languages: Python, PySpark Tools: Docker, Git... ...Celery Datastores: Databricks and Delta Lakes...DatabricksPythonFull timeTemporary workWork experience placementFlexible hoursAltana Technologies
Brooklyn, NY2 days ago- ...models, written by PBG Strats in python, into c++. Supporting model run in... ...(ie: QA, UAT and PROD) Work with WM Strats to fully understand the... ...Strong Python experience (python, pySpark) ~ Strong database experience (SQL, Databricks and Snowflake) ~ Grid computing...DatabricksPython
Infotree Global Solutions
New York, NY2 days ago - ...Senior Data Engineer with Databricks/PySpark Location: NJ, hybrid 3X a week. Final round in person interview... ...for ingesting of datasets • Lead design of ETL, data integration and data... ...REQUIREMENTS • Experience on Python scripting, Spark SQL PySpark is a must...DatabricksPythonWork experience placement
RIT Solutions, Inc.
New York, NY13 hours ago - ...Job Title: L2/L3 Lead Support Engineer Location: Jersey City, NJ Duration: Full Time Employment... ...of experience. L2/L3 Support experience. Expert in Java, Python, Spark, AWS ( Redshift, Aurora, SQS, Glue, Airflow, etc), DBX Databricks skills and experienceDatabricksPythonFull time
Lorven Technologies
Jersey City, NJ2 days ago - ...Overview: Senior-level engineer responsible for... ...applications using AWS and Databricks. Role involves complex... ..., and working closely with distributed agile... ...(DLT) SQL, Spark, Python/PySpark Java Databricks... ...Ability to lead technically and influence...DatabricksPython
Magicforce
Jersey City, NJ2 days ago - ...Lead Software Engineer We have an exciting and rewarding opportunity for you to take your software engineering career... ...support of the firm's business objectives. With us, you can enhance your skills in Python, PySpark, and cloud architecture, while working in an...Python
Chase
Jersey City, NJ4 days ago - ...looking for a skilled Data Engineer to lead technical delivery on innovative... ...will have expertise in Databricks, Apache Spark, Delta Lake, as... ...well as advanced skills in Python and SQL. You will design and... ...leadership while collaborating with clients. This role offers...DatabricksPython
- ...Job Title: Lead Software Engineer - Power BI Location: New York, NY (Onsite/Hybrid as applicable... ...-performance data models aligned with business needs. Collaborate... ...Synapse Analytics Databricks Exposure to Python for data analysis or automation....DatabricksPython
Lorven Technologies
New York, NY3 days ago $130k
...will join our Product Engineering team that shares a passion... ...line. As our Senior Lead Software Engineer, you... ...organizations operate with integrity and protect their... ...technologies: Python and SQL; PostgreSQL and... ..., such as Snowflake or Databricks Containerization and orchestration...DatabricksPythonNAVEX Global
New York, NY3 days ago- ...Lead Software Engineer We have an opportunity to impact your career and provide... ...'t just use AI — they think with it, build with it, and know... ...services ~ Proficiency in Python programming ~ Experience... ...tools and platforms, especially Databricks Familiarity with CI/CD...DatabricksPython
Chase
Jersey City, NJ4 days ago - ...Lead Software Engineer We have an opportunity to impact your career and provide... ...ETL data pipelines using Databricks (Spark, Delta Lake, Unity Catalog... ...and storage Collaborate with data product owner, business... ...(s), including Java or Python for data engineering tasks...DatabricksPythonWork at office
Chase
Jersey City, NJ2 days ago - ...C++/Pyspark Developer with Mortgage Risk Choosing Capgemini means choosing... ...us and help the world's leading organizations?unlock the... ...expertise in C++ , Python , and data engineering , along with a strong... ...and platforms such as Databricks and Snowflake . ~ Experience...DatabricksPythonPermanent employmentFull timeContract workLocal area
Capgemini
New York, NY6 days ago - ...Lead Software Engineer This is your chance to change the path of your career... ...latency data pipelines using Databricks and Apache Spark (Core, SQL,... ...lakehouse patterns with Delta Lake (ACID transactions... ...libraries, frameworks, and APIs in Python and/or Java; oversee unit, integration...DatabricksPythonWork at office
Chase
Jersey City, NJ2 days ago $152k - $215k
...to take your software engineering career to the next level... ...trusted, market-leading infrastructure and tools... ...pipelines. Collaborate with data scientists, ML... ...infrastructure Proficiency in Python and one or more ML... ...Familiarity with Databricks for scalable data...DatabricksPythonJPMorgan Chase Bank, N.A.
New York, NY13 hours ago$152k - $215k
...of what's possible. As a Lead Software Engineer at JPMorgan Chase within the... ...and technical troubleshooting with ability to think beyond... ...stability Advanced in coding in Python Experience coding in Java... ...native experience (AWS) Databricks experience LLM...DatabricksPythonJPMorgan Chase Bank, N.A.
Jersey City, NJ13 hours ago- ...Senior Lead Software Engineer Be an integral part of an agile... ...technical troubleshooting with ability to think... ...practical experience with Python, Java, RESTful APIs,... ...Management. Databricks and Spark: Experience... ...) and Apache Spark (PySpark/Scala) for batch and...DatabricksPython
Chase
Passaic, NJ2 days ago $171k - $260k
...impacts front office outcomes. You'll work closely with Front Office and Middle Office users and help shape the tech roadmap across Python/Java, Kafka, AWS + Databricks, and SQL/NoSQL. As a Senior Lead Software Engineer at JPMorgan Chase within Corporate Technology -...DatabricksPythonWork at officeJPMorgan Chase Bank, N.A.
Jersey City, NJ13 hours ago$300k - $350k
...Senior Staff/Technical Lead Machine Learning Engineer Senior Staff/Technical Lead Machine... ...and experience — talk with your recruiter to learn more... ...( Spark , SQL , Scala , or Python ). DevOps knowledge including... ...with cloud platforms (AWS, Databricks) and ML tooling (MLFlow,...DatabricksPythonFull timeInternshipRemote workHarnham
New York, NY3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Databricks Engineer with Pyspark & Python Exp. Be the first to apply!
Related searches
- lead support engineer New York, NY
- lead ios developer New York, NY
- lead solutions engineer New York, NY
- lead mobile developer New York, NY
- lead quality engineer New York, NY
- lead project engineer New York, NY
- lead network engineer New York, NY
- lead product engineer New York, NY
- lead web developer New York, NY
- lead software test engineer New York, NY