Senior Data Pipeline Engineer — Python, Spark & Kafka
Jobtailor
Jobtailor in Kentucky is seeking a Data Engineer to design, build, and maintain scalable data pipelines and ETL/ELT processes, leveraging Python, PySpark, and Java to transform large datasets. You will ingest structured and unstructured data via REST APIs, streaming platforms, and batch jobs, work with Kafka and Hadoop, and ensure data quality and governance in an Agile team environment. #J-18808-Ljbffr Jobtailor
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Data Pipeline Engineer — Python, Spark & Kafka in Brooklyn, NY vacancy
- ...3899Reference Number: 25-00728Title: Senior Data Engineer (Python)Location: Jersey City, NJPosted Date:... ...solutioning.Exp in designing and Implementing Kafka producers and consumers.Experience... ....Experience in building ETL Pipelines using PySpark or Data Factory.DW experience...Senior
- ...GorusuCompany: SRI Tech SolutionsRole: Data Engineer - Python/PySparkLocation: Jersey City, NJ (3... ...building scalable ETL/ELT pipelines in big data environments- Strong understanding... ...domains- Hands-on experience with Spark, Hadoop, Kafka, and cloud platforms- Solid knowledge...SuggestedFull time3 days per week
$126k - $210k
...you will Build robust batch/streaming data pipelines and transformations that ensure high... ...and dependencies using modern data engineering tools.Partner with analytics, AI,... ...data environments.Deep expertise in Spark/Databricks, Kafka, SQL, and orchestration tools such as...SeniorTemporary workWorldwideFlexible hours$176.72k - $265.08k
...harness the power of data to drive innovation... ...are looking for a Senior Data Engineering Lead to be a player... ...(e.g., using Python, Scala, SQL, and Spark).Lead & Mentor: Lead... ...Impala, Apache Spark, Kafka, or Flink)Proficiency... ...maintaining CI/CD pipelines for data applications...SeniorFull time$126.7k - $215k
...Make An Impact As a Senior Staff Data Engineer, you will join a highly... ...APIs and data pipelines. Lead high-level architecture... ...experience in Python or Java programming language... ...data pipelines using Spark, BigQuery, AWS EMR,... ...processing (like Kafka, Flink, Pub/Sub). Collaboration...SeniorTemporary workLocal areaWorldwide$95k - $150k
...environment? As an experienced Senior Data Engineer you will have the ability to... ...and optimize ETL/ELT pipelines using Azure Data Factory (ADF... ...Databricks Write and tune PySpark / Spark SQL notebooks for large-... .... Required Qualifications Python, PySpark, Spark SQL, SQL Server...SeniorPermanent employment- ...Hands-on experience with Azure Services: Databricks, Data Factory and DevOps Proficient in Python, Apache Spark, and PySpark Write advance and complex SQL queries with solid experience in database management Develop and maintain data models, data warehouses, and data...Senior
Inizio Partners Corp
Jersey City, NJ1 day ago - ...Skills 6–10 years of experience in Data Engineering or Software Engineering. Strong... .... Advanced programming skills in Python . Strong expertise in Apache Spark/PySpark . Advanced SQL development... ..., and maintaining scalable data pipelines. Experience working with structured...SeniorRemote workWorldwide
Dexian
Jersey City, NJ4 days ago - ...We are seeking an experienced Data Engineer with strong expertise in Databricks, Python, and Database engineering. The ideal... ..., and optimize scalable data pipelines, ensure data quality, and support... ...data pipelines using Python and Spark. Optimize data transformation...
Cloud Analytics Technologies LLC
Jersey City, NJ2 days ago - ...consultants bring deep expertise in Data Science, Machine Learning and... ...seeking an experienced Data Engineer to join our data team. In this... ...and maintaining scalable data pipelines, data integration processes,... ...using Databricks, Apache Spark, and SQL to support analytics,...SeniorLocal area
Tiger Analytics Inc.
Jersey City, NJmore than 2 months ago - ...ABOUT THE ROLE We are looking for a Senior Data Engineer to design and build scalable data... ...financial data daily. You will implement Python-based ETL/ELT pipelines, orchestrate workflows with Airflow... ...APIs, and work with Snowflake, Spark, and AWS to deliver high-...SeniorWork at officeRemote workVisa sponsorshipWork visaFlexible hours
AgileEngine
Jersey City, NJ3 days ago - ...About the Job Opportunity The Senior Data Engineer plays a leading role in... ...designing and building scalable pipelines, setting best practices, and... .../ELT pipelines using SQL, Python, Talend, and Databricks to support... ...MDM platforms; knowledge of Kafka or real‑time streaming....SeniorLocal area
PartsSource
Brooklyn, NY2 days ago $30 per hour
Mindrift is seeking a Senior Python Data Scraping Engineer to join the Tendem project. This freelance, part-time remote role focuses on web scraping, data extraction, and processing to deliver high-quality structured datasets. You'll work with tools such as BeautifulSoup...SeniorRemote jobHourly payPart timeFreelance- The Senior Technical Specialist will be responsible for providing expertise in Apache spark, scala, and python. They will play a key role in developing and... ...Develop and implement data processing frameworks using... .... Knowledge of data engineering concepts and methodologies...Senior
- ...technology products.As a Senior Lead Software Engineer at JPMorganChase within the... ...wide Marketing and Events Data Platform. We are looking for... ...years of Experience with Python 2+ years of Experience in... ...Data technologies (e.g. AWS, Spark, Kafka, Data Bricks etc.)...Senior
$164k - $193k
...s the role?Etsy's Data & ML Enablement organizations... ...up a team of engineers who will sit at... ..., and ML feature pipelines that turn platform... ...We work in Scala, Python, Java, and SQL,... ...systems built on Spark, Kafka, Airflow, Looker,... ...reporting to the Senior Engineering Manager...SeniorFull timeWork at officeLocal areaVisa sponsorshipFlexible hours- ...technology products.As a Senior Lead Software Engineer- Python Distributed Development and... ...operates the spec generation pipeline — Implement artifact... ...event sourcing, CDC (change data capture), and data pipelines... ...toolingExperience with Kafka, PostgreSQL, and container...Senior
$30 per hour
...Vibecode specialists to join the Tendem project ( and drive specialized data scraping workflows for real-world use cases. Mindrift is looking for highly skilled Senior Python Data Scraping Engineers to join the Tendem project and drive specialized data scraping...SeniorHourly payPart timeFreelanceRemote work10 hours per week- Senior Data Engineer Established in 2021, Independence Pet Holdings is a corporate... ...building the platforms and pipelines that enable smarter... ...platforms and pipelines. Strong Python and SQL expertise. Strong SQL... .../event‑driven systems (Kafka, Kinesis). Solid experience...SeniorTemporary workWork at officeRemote workFlexible hours
- Pandora in the United States is seeking a Senior Staff Data Engineer to own backend services, data products and reporting systems at scale... .... The ideal candidate has extensive experience with Python or Java, Spark and cloud data warehouses, with strong ownership, mentoring...Senior
$159k - $208.95k
...are seeking a Staff Data Engineer to lead the technical... ...and products. In this senior-level individual contributor... ...& streaming data pipelines and architectures... ...and frameworks such as Spark, Airflow, Kafka, Databricks, and... ...programming language (e.g., Python, Scala, or Java)...Temporary workLocal areaWorldwide- ...56017Reference Number: 25-00834Title: Python PySpark/ Data EngineerLocation: Columbus, OH and Jersey... ...designing and implementing data pipelines in a cloud environment is required.5+... ...building real-time data streams using Spark and Experience in Spark.Knowledge or experience...Work experience placement
$165k - $180k
...skilled and motivated Sr. AI Data Engineer with a proven track record in... ..., and maintain scalable data pipelines and workflows using Databricks... ...open-source tools such as Python, PySpark, and Apache Airflow... ...processing tools like Apache Kafka or Structured Streaming. Familiarity...SeniorWork experience placementH1bWork at officeLocal area$121.2k - $181.8k
...DevelopmentCompany: CitiSenior Java Big Data Developer with experience in designing and... ...that provides sensitive data to senior management. Application involves business... ...level knowledge of Java, Apache Spring, Spark, Kafka, Cloudera Ecosystem (Hadoop, HDFS, Hive,...SeniorFull time- ...Services.Job DescriptionJob Description:-Our Big Data capability team needs hands-on developers... ...in at-least one of Java, C++ or Python (preferred). Scala knowledge a strong advantage... ..., etc..Hands-on experience with Apache Spark and its components (Streaming, SQL, MLLib)...SeniorPermanent employmentFull timeH1b
- Oracle SQLPLSQL10 years experience with at least 3 years experience as a data analystDelivery focused technofunctional roleGood knowledge of SQL programming preferably with databases such as Oracle ExadataProgramming experience with PythonProgramming experience with Hadoop...Senior
$155.6k - $306.8k
...At Deloitte, Forward Deployed Engineers (FDE) don’t just build AI solutions, they... ...ends on 10/30/2026.Work you’ll doAs a Senior Microsoft AI&Data FDE, you will work side by side with senior... ...environments Data engineering experience with Spark, Airflow/dbt, streaming, data modeling...SeniorLocal area- ...what's possible.As a Lead Software Engineer at JPMorganChase within Corporate Technology... ...languages including Java, Python, or Scala.Good understanding of the... ...features.Extensive experience with big data technologies, including Apache Spark, for large-scale data processing...
$124.7k - $207.8k
Position Summary Sr. Python Developer Are you an... .... Mentor junior engineers and contribute to knowledge... ...solutions in software, data, AI, network, and hybrid... ...needed. Knowledge of Apache Spark and distributed data... ...-level employees to senior leaders, we believe there...SeniorFlexible hours- ...Title: Senior Data Engineer-85823Y0474 Client: NYC Agency Duration: 12 Months... ...enabling efficient and predictable data pipelines. Have hands-on development experience... ...etc. Programming experience with Python or Java Experience with Azure Monitoring...SeniorRemote work
RightTalents LLC
Brooklyn, NY1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Data Pipeline Engineer — Python, Spark & Kafka. Be the first to apply!
Related searches
- data center engineer Brooklyn, NY
- data developer Brooklyn, NY
- data science developer Brooklyn, NY
- finance data engineer Brooklyn, NY
- senior data center engineer Brooklyn, NY
- sr information security engineer Brooklyn, NY
- data engineer Brooklyn, NY
- etl data engineer Brooklyn, NY
- data engineer machine learning Brooklyn, NY
- data engineer analytics Brooklyn, NY