Data Engineer - Kafka / PySpark / Hadoop
Full-time
Acestack
- Job Title: Data Engineer Kafka / PySpark / Hadoop
- Location: Toronto, ON
- Work Model: Onsite
- Job Type: Full Time (FTE) Job Description
- Design, develop, and maintain scalable batch and real-time data pipelines .
- Develop data processing applications using Python and PySpark/Apache Spark .
- Build and support Kafka-based data ingestion and streaming pipelines .
- Work with Hadoop and related technologies to process large volumes of data.
- Develop and maintain ETL/ELT pipelines for data ingestion, transformation, cleansing, and integration.
- Perform data validation, reconciliation, and quality checks.
- Troubleshoot pipeline failures, data discrepancies, and performance issues.
- Optimize Spark/PySpark jobs and SQL queries for performance and scalability.
- Monitor data pipelines and resolve production issues.
- Collaborate with data architects, developers, analysts, and business teams.
- Participate in Agile development, testing, deployment, and production support activities.
- Strong hands-on experience with Python for data engineering and automation.
- Strong expertise in PySpark / Apache Spark .
- Hands-on experience with Apache Kafka for real-time data ingestion and streaming.
- Strong experience with the Hadoop ecosystem and distributed data processing.
- Strong SQL skills and experience working with large datasets.
- Experience developing and maintaining ETL/ELT data pipelines .
- Strong understanding of distributed computing and data processing concepts.
- Experience with data ingestion, transformation, cleansing, and integration.
- Strong troubleshooting and performance optimization skills.
- Hive
- Databricks
- AWS, Azure, or GCP
- Git and CI/CD
- Unix/Linux
- Airflow or Autosys
- Relational and NoSQL databases
We are seeking an experienced Data Engineer with strong hands-on expertise in Kafka, PySpark, Python, and Hadoop to design, develop, and support scalable batch and real-time data pipelines. The ideal candidate will have strong experience working with large-scale distributed data processing environments and enterprise data integration solutions.
Key ResponsibilitiesVacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - Kafka / PySpark / Hadoop in Ontario, CA vacancy
$104.04k - $137.84k
...opportunity to an authentic experience!The Data Engineer II assists in the implementation of... ...Factory, Synapse Analytics, Az Data Bricks, PySpark ETL. Develop and implement data... ...MonitorPowerShellPythonDevelopment experience using PySpark, Spark, Hadoop, Kubernetes, and RDMIS is highly...SuggestedTemporary workWork experience placementRemote workMonday to FridayFlexible hours- Cloud Data Engineer Location: Toronto, ON Work Model: Hybrid Employment Type: Full-Time Permanent (FTE) Job Summary We... ...data processing technologies such as Apache Spark, Kafka, or similar platforms . Experience with cloud data warehouse...SuggestedPermanent employmentFull time
Acestack
Ontario, CA2 days ago - Join the Clean Energy RevolutionBecome a Data Engineering, Senior Advisor at Southern California Edison (SCE) and build a better tomorrow. In this job, you’ll be responsible for developing data integration pipelines (ETL) sourcing from various databases and systems. The...SuggestedRemote workRelocation
$114.28k - $140k
901 Corporate Center Dr, Pomona, CA 91768, USA Description The Senior Data Engineer leads the design, modernization, and operational support of the enterprise data platform. This role is responsible for architecting, optimizing, and supporting enterprise data integration...Suggested- Join the Clean Energy RevolutionBecome a Data Engineering, Senior Specialist at Southern California Edison (SCE) and build a better tomorrow. In this job, you’ll be responsible for creating and supporting analytical solutions, by designing and building data integration...SuggestedRemote work
- ...Job Title: Data Platform Engineer Job Location: Toronto Ontario Job Type: Contract Job Description: Collaborate with internal stakeholders to translate business and analytical requirements into well-defined backend features and data products. Design...Contract workTemporary workLocal area
Staffingine LLC
Ontario, CA2 days ago $190k - $220k
...Using Owner should feel like having a team of great operators, engineers, and marketers working for you. Our vision We’re starting by helping... ...Why we are looking for you We are looking for a mission-driven Data Platform Enginer to build Owner's data platform - the...Work at officeLocal areaRemote workGrabJobs
Fontana, CA2 days ago$69.6k - $142.7k
Job Title: Data Scientist for Analytics (Engineer/Scientist Support)Job Category: Information TechnologyTime Type: Full timeMinimum Clearance Required to Start: SecretEmployee Type: RegularPercentage of Travel Required: Up to 10%Type of Travel: Continental US* * *The Opportunity...Contract workFor contractorsWork experience placementFlexible hoursShift work- ...Job Title: Platform Engineer Location: Toronto, Ontario (ON) Work Arrangement... ...to join a growing Wealth Management Data Management & Insights (DMI) team. The... ..., Sybase, SQL Server, MemSQL, MQ, Kafka, OpenShift, ELK, Hadoop, and Connect Provide technical...Full time
Acestack
Ontario, CA2 days ago $155k - $165k
...Evolve might just be the place for you. Why This Role At Evolve, data powers every aspect of how we transform the vacation rental... ...operational excellence, and future-forward innovation. As a Senior Data Engineer, you'll be a key technical leader responsible for building and...Full timeLive inWork at officeImmediate startRemote workGrabJobs
Fontana, CA3 days ago$115k - $140k
...offers fantastic Culture as well as competitive pay, 401K W/Match, Medical, Dental, & Vision benefits. What You’ll Do: The Senior Data Engineer will drive the growth, productivity, and management of our organization by sourcing and delivering information critical for...- ...Job Title: Senior Data Developer - ETL Location: Mississauga, ON Work Model: Onsite Job Type: Full Time (FTE) Job Description We are seeking an experienced Senior Data Developer - ETL with strong expertise in SQL Server, SSIS, database development...Full time
Acestack
Ontario, CA2 days ago - ...Description Job Description POSITION IS 100% ONSITE DAILY IN NORCO, CA Job Summary The Senior SQL Server Database Developer / Data Engineer serves as the technical lead for enterprise database architecture, SQL Server development, data warehousing, and data...Contract workTemporary workFlexible hours
VSolvit
Norco, CA12 days ago - The engineering team at Chainalysis is inspired by solving the hardest technical challenges and... ...automates entity resolution, optimizes data labeling pipelines, and creates predictive... ...the modern data stack: Spark, Databricks, Kafka, Delta Lake/Iceberg, some experience with...Flexible hours
- ...Job Description Job Description POSITION IS 100% ONSITE IN NORCO, CA Job Summary The SQL Server Database Developer / Data Engineer II independently designs, develops, and maintains enterprise SQL Server database solutions supporting mission-critical applications...Temporary workFlexible hours
VSolvit
Norco, CA12 days ago - ...Description Job Description POSITION IS 100% ONSITE DAILY IN NORCO, CA Job Summary The SQL Server Database Developer / Data Engineer I supports the design, development, implementation, and maintenance of Microsoft SQL Server databases and data integration...Temporary workFlexible hours
VSolvit
Norco, CA12 days ago - SUMMARYThis role positions data as a strategic asset in support of the company's goals and objectives. The Data Integration Engineer is responsible for designing, developing, and implementing enterprise-level business intelligence solutions, data integration processes,...
- ...Job Title: Data Office Developer Job Type: Full Time Location: Toronto, ON (Hybrid) Job Description We are seeking a skilled Data Office Developer with strong hands-on experience in SQL and Python to design, develop, and maintain scalable data...Full timeWork at office
Acestack
Ontario, CA2 days ago - ...Job title: Senior Data Science Developer Location: Toronto, ON M5G 2L3. ( 3 days On-site) Hybrid Duration: 6 Months Description: Background Information: The Agency Mental Health & Addiction program requires technical analytics support to deliver...Full timeWork at officeWork from homeHome office
Amicis Global
Ontario, CA2 days ago - ...Job Title: Senior Data Science Developer Location: Toronto, Ontario, Canada Work Model: Hybrid Duration: 5 months Extension: 1 Time Must Haves Background in health data and public-sector/government experience is required Technology...
Innosystech Inc
Ontario, CA2 days ago $61.8k - $165.33k
The Senior Data & BI Reporting Engineer is responsible for acquiring, cleansing, and analyzing large data sets using technologies such as Databricks, Snowflake, SQL, Python, and Tableau. The role requires collaborating with stakeholders to design and implement ad-hoc data...Temporary work- ...communication and problem-solving skills Bilingual in Spanish and English Preferred Skills Docker and Kubernetes Apache Kafka Event-driven architecture Cloud-native application development Agile/Scrum environment Experience with enterprise-scale...Full time
Acestack
Ontario, CA2 days ago $18 - $50 per hour
...university graduation and completion of the program Position Overview: Siemens Digital Industries Software is seeking an AI & Engineering Data Intern to support emerging Artificial Intelligence and Digital Thread initiatives within the Capital PreSales organization....Remote jobHourly payFull timeInternshipLocal areaSiemens
Pomona, CA2 days ago- ...Azure Support Analyst with strong Azure data platform and production support... ...Troubleshoot Databricks notebooks, PySpark/Spark SQL, and Delta Lake issues Monitor... ...Collaborate with development and data engineering teams to resolve production issues...Full time
Acestack
Ontario, CA2 days ago - Join the Clean Energy RevolutionBecome a Data Science Manager at Southern California Edison (SCE) and build a better tomorrow. In this... ...filings. As a trusted technical leader, you will partner with engineers, strategy owners, IT, risk management, and regulatory teams to...Remote workRelocation
- ...Job Title: Back-end Engineer - Python Job Location: Downtown Toronto ,Canada Job Type: Full-Time Job Description: Design... .... Familiarity with message brokers such as RabbitMQ, Kafka, or similar. Proficient in designing and consuming RESTful...Full timeFlexible hours
Staffingine LLC
Ontario, CA2 days ago $94.99k - $137.28k
Overview Prime Healthcare is an award-winning health system headquartered in Ontario, California. Prime Healthcare operates 54 hospitals and has more than 360 outpatient locations in 15 states providing more than 3.0million patient visits annually. It is one of the nation...Daily paidFull timeTemporary workPart timeLocal areaShift work- ...Marketing Data Analyst Ontario, CA Job Duties Research market conditions in fertility services industry by designing research format and methodology for data collection, to help identify potential opportunities. Research and track data regarding new clients and...
- ...seeking a talented and driven mid-level Data Scientist to join our evolving and dynamic... ..., Data Science, Mathematics, Statistics, Engineering, or related field Two to five years... ...and emerging AL/ML technologies (e.g., Hadoop, Spark) Experience with DevOps practices...Temporary workFlexible hours
VSolvit
Norco, CA12 days ago - ...Job Description Job Description Job Summary: We are looking for a Senior Data Analyst Specialist to join our team at PM2CM. In this role, you will be responsible for analyzing complex data sets to drive strategic decision-making across various projects within the...Work at office
PM2CM
Pomona, CA20 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - Kafka / PySpark / Hadoop. Be the first to apply!
Related searches
- data science developer Ontario, CA
- data engineer machine learning Ontario, CA
- data center engineer Ontario, CA
- data developer Ontario, CA
- senior data center engineer Ontario, CA
- finance data engineer Ontario, CA
- data engineer Ontario, CA
- data engineer analytics Ontario, CA
- sap master data Ontario, CA
- data coordinator Ontario, CA