Big Data Engineer
Omni Inclusive
Big Data EngineerMandatory skills: Apache Spark/ Hive/ Kafka/ Amazon Glue/ Google Dataflow/ Talend MDM/ Hadoop/ Presto/ Strong experience with MySQL, PostgreSQL, MongoDB, Cassandra.Role: Big Data EngineerJob Overview: We're seeking a highly skilled Data Engineer, Big Data Engineer to build scalable data pipelines, develop ML models, and integrate big data systems. You'll work with structured, semi-structured, and unstructured data, focusing on optimizing data systems, building ETL pipelines, and deploying AI models in cloud environments.Key Responsibilities:Data Ingestion: Build scalable ETL pipelines using Apache Spark, Talend, AWS Glue, Google Dataflow, Apache NiFi. Ingest data from APIs, file systems, and databases.Data TransformationValidation: Use Pandas, Apache Beam, and Dask for data cleaning, transformation, and validation. Automate data quality checks with Pytest, Unittest.Big Data Systems: Process large datasets with Hadoop, Kafka, Apache Flink, Apache Hive. Stream real-time data using Kafka, Google Cloud PubSub.Task Queues: Manage asynchronous processing with Celery, RQ, RabbitMQ, or Kafka. Implement retry mechanisms and track task status.Scalability: Optimize for performance with distributed processing (Spark, Flink), parallelization (joblib), and data partitioning.CloudStorage: Work with AWS, Azure, GCP, Databricks. Store and manage data with S3, BigQuery, Redshift, Synapse Analytics, and HDFS.Required Skills:ETL Data Processing: Expertise in Apache Spark, AWS Glue, Google Dataflow, Talend.Big Data Tools: Proficient with Hadoop, Kafka, Apache Flink, Hive, Presto.Databases: Strong experience with MySQL, PostgreSQL, MongoDB, Cassandra.Machine Learning: Hands-on with TensorFlow, PyTorch, Scikit-learn, XGBoost.Cloud Platforms: Experience with AWS, Azure, GCP, Databricks.Task Management: Familiar with Celery, RQ, RabbitMQ, Kafka.Version Control: Git for source code management.Desirable Skills:Real-time Data Processing: Experience with Apache Pulsar, Google Cloud PubSub.Data Warehousing: Familiarity with Redshift, BigQuery, Synapse Analytics.Scalability Optimization: Knowledge of load balancing (NGINX, HAProxy) and parallel processing.Data Governance: Use of MLflow, DVC, or other tools for model and data versioning.Tools Technologies: ETL: Apache Spark, Talend, AWS Glue, Google Dataflow. Big Data: Hadoop, Kafka, Apache Flink, Presto. Databases: MySQL, PostgreSQL, MongoDB, Cassandra. Cloud: AWS, GCP, Azure, Databricks. Storage: S3, BigQuery, Redshift, Synapse Analytics, HDFS. Version Control: Git.
- ...specializing in the fields of Software Development, Software Consultancy, and Information Technology Enabled Services.Job DescriptionOur Big Data capability team needs hands-on developers who can produce beautiful & functional code to solve complex analytics problems. If you...Big dataPermanent employmentFull timeH1b
Sonsoft
Jersey City, NJ1 day ago - ...Purple DriveWe are seeking a highly skilled Palantir Data Engineer to design, develop, and maintain scalable data solutions using Palantir... ...Required Skills & Experience:Strong expertise in Palantir Foundry and big data frameworks (Spark, Hadoop).Proficiency in Python, PySpark,...Big data
Purple Drive
Jersey City, NJ2 days ago $74 - $84 per hour
...Data Engineer - Jersey City, NJ - Contract - $74-84 Per HRWe are looking for a skilled Data Engineer to join our team remotely. This contract... ...backend development.Proven experience with cloud services and big data processing.Strong understanding of data storage and database...Big dataContract workRemote workMitchell Martin
Jersey City, NJ4 days ago$82.42k - $126.6k
OverviewThe Infosys Data and Analytics (DNA) unit is at the forefront of transforming data into actionable insights, driving business growth... ...with Hadoop ecosystem (Cloudera). •Experience in working with a Big Data implementation in production environment. •Experience with...Big dataFull timeTemporary workRelocationInfosys Technologies
Jersey City, NJ1 day ago- ...JPMorgan Chase & Co. in Jersey City is seeking a Lead Software Engineer for Financial Service Data Engineering with PySpark, Java, and Big Data focus. You will drive AI-first analytics and productized data lifecycles, champion a scalable data mesh, and mentor engineers...Big data
- ...JPMorganChase in Jersey City, NJ, seeks a Lead Software Engineer- Financial Service Data Engineering: PySpark/Java/BigData/Datalake/AI to join the Asset and Wealth Management–Global Prime Brokerage team. You will be a core technical contributor, building trusted data...Big data
- ...JPMorgan Chase & Co. in New Jersey seeks a Big Data Software Engineer to build and enhance a large-scale data platform for Client 360 and Marketing & Events in the Commercial Banking IT group. You will implement ETL processes, collaborate with stakeholders, and apply...Big data
- ...can push the limits of what's possible.As a Lead Software Engineer- Financial Service Data Engineering: Pyspark / Java / BigData / Datalake / AI, at... ...for analytics-ready datasetsDeep hands‑on experience with big data and modern data platforms (e.g., Spark, Databricks, Snowflake...Big data
JP Morgan Chase
Jersey City, NJ2 days ago - ...property and casualty insurance and risk engineering services; affluent and high net worth individuals... ...a highly skilled and experienced lead data engineer to join our dynamic team. The... ...Extensive experience working with Big Data tools and building data solutions for...Big dataWork experience placementLocal area
- ...Azure Data EngineerExpert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service... ...Factory, Azure Databricks, and other Azure services. Fluent with Big Data technologies like Spark, Kafka and Hive. Expert level skills...Big data
InterSources
Weehawken, NJ1 day ago - ...Data EngineerWe are seeking experienced Data Engineers in a contract capacity to join our talented Data Engineering team. In this role, you will contribute to... ...warehousing tools like Snowflake or DatabricksFamiliarity with distributed systems and big data technologies...Big dataContract work
Argyle Infotech
Brooklyn, NY1 day ago - ...client. He/ She will work as a Senior Cloud Engineer within the CDO EDP team and will have the... ...developers to build and enhance the data infrastructure and application patterns that... ...experience with ETL, distributed compute, and big data frameworks such as Spark, Glue,...Big dataFull time
EXL
Newark, NJ7 hours ago - ...Job title: AWS Data Engineer Location: Hybrid 2 days onsite (Newark, NJ) Duration: 24 months Must have at least 12 years of experience... ...Linux, UNIX shell scripts ~ Developer experience in big data or warehouse (Hadoop, DB2 LUW) environment. ~ Experience...Big data
Damco Solutions
Newark, NJ4 days ago - ...Lead AWS Data Engineer Contract Opportunity with a Major Investment Firm Location: Newark, NJ (Hybrid - 3 Days/Week Onsite) Duration... ...frameworks ~ Strong understanding of data modeling and big data technologies ~ Experience mentoring engineers and leading...Big dataLong term contractContract workLocal area3 days per week
JSR Tech Consulting
Newark, NJ4 days ago - ...AWS Lead Data EngineerPosition: AWS Lead Data EngineerLocation: Newark, NJ – HybridInterview... ...looking for an experienced AWS Lead Data Engineer to provide technical leadership and design... ...Warehouses, ETL/ELT, Data Modeling, and Big Data frameworksExperience with DevOps, CI/...Big data
SpikeIT Global Solutions Inc
Newark, NJ4 days ago - ...Data Engineers (AWS)We are seeking a talented AWS Data Engineer to join our dynamic Data Engineering team. The ideal candidate will be responsible... ...pipelines in AWS.Knowledge of ETL/ELT, data modeling, and big data technologies.Familiarity with data warehousing concepts...Big dataHourly pay
JSR Tech Consulting
Newark, NJ13 hours ago - ...AWS Lead Data EngineerWe are seeking an experienced AWS Lead Data Engineer to join our Data Engineering team. As a technical leader, you will be responsible for architecting... ...with ETL or ELT processes, data modeling, and big data frameworks.Demonstrated ability to lead,...Big dataFor subcontractorWork at office
Omni Inclusive
Newark, NJ1 day ago - ...Job Description Looking for data engineers with 9+ years of experience. 1. Experience developing and deploying application code using... ...Linux, UNIX shell scripts. 6. Developer experience in big data or warehouse (Hadoop, DB2 LUW) environment. 7. Experience...Big dataHourly payRelocationWork visa
Cedent
Newark, NJ3 days ago - ...you can push the limits of what's possible.As a Lead Software Engineer at JPMorganChase within Corporate Technology in Financial Planning... ...practices, and security features.Extensive experience with big data technologies, including Apache Spark, for large-scale data processing...Big data
JP Morgan Chase
Jersey City, NJ2 days ago - Position : Sr. Data Engineer Location: Newark, NJ (Hybrid) Duration: 12 Months Contract Job Description Bachelor's degree in Computer Science... ...pipelines in AWS. Knowledge of ETL/ELT, data modeling, and big data technologies. Familiarity with data warehousing concepts and...Big dataContract work
- ...and results-oriented professional to join our dynamic team as a Data Engineer. Details of the same have been highlighted below: Roles and... ...data services. Experience with Hadoop, Spark, Kafka, or other big data technologies is a plus. Strong understanding of relational...Big data
$121.2k - $181.8k
...growth and progress together.Citibank is seeking a Senior Java Big Data Developer to design and build core components of the... ...senior management, combining expert-level Java and J2EE backend engineering with JavaScript-based front-end development. The position sits...Big dataFull timeCitigroup
Jersey City, NJ1 day ago- We are looking for a skilled and enthusiastic Data Engineer to help design and optimize our data infrastructure. JOB REQUIREMENTS: Create... ...of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. Build analytics tools that utilize the data...Big dataVisa sponsorshipWork visa
$85 per hour
...hr (C2C) or $80/hr (W2) We are seeking an experienced AWS Lead Data Engineer to join a high-performing Data Engineering team. This role requires... ...AWS Strong background in ETL/ELT processes, data modeling, and big data frameworks Experience leading and mentoring engineering...Big dataContract work- ...Big Data Developer Location: Strongsville OH / Pittsburgh PA / Dallas TX Duration: Full Time Job Description: Bigdata Hadoop & ecosystem, Scala/Python, PySpark. Oracle PL/SQL, CI/CD, Excellent communication, Data Lake, Informatica, Teradata. Roles...Big dataFull time
JConnect Infotech
Passaic, NJ5 days ago $107k
The Data Engineer is responsible for designing, developing, and optimizing enterprise data pipelines and data platforms. This role focuses on... ...in SQL and Python; familiarity with cloud platforms and big data technologies. Ability to diagnose data issues, optimize performance...Big data- We have a full-time job opportunity of Data Engineer in San Francisco, California (Hybrid). Location: San Francisco, California (Hybrid, 3... ...Responsibilities Design, develop, test, deploy, and maintain scalable Big Data solutions. Manage project priorities, deadlines, and...Big dataFull timeWork at office
- ...manage, optimize, and assist in the design of data pipelines in support of information,... ...guidelines set by architects, senior and lead engineers. Deliver trustworthy data to ensure... ...knowledge of transactional, analytical, big data structures preferred. Understanding...Big dataFull timeWork at office
$148.7k - $199.4k
...ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more - all working to build and... ...spectrum of technologies, including machine learning, big data, microservices, and data visualization. We are...Big data- Our Sportsbook Data Team is looking for a Data Engineer to make a difference in the further development of our Data Warehouse solutions, designing and developing... ...be a plus (.NET, Python, R, Scala) Knowledge around Big Data concepts and data streaming architectures will be a...Big dataLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!
- data engineer machine learning Jersey City, NJ
- aws data engineer Jersey City, NJ
- big data developer Jersey City, NJ
- data engineer hedge fund Jersey City, NJ
- sr data engineer Jersey City, NJ
- big data cloud engineer Jersey City, NJ
- finance data engineer Jersey City, NJ
- junior big data engineer Jersey City, NJ
- sr information security engineer Jersey City, NJ
- data center engineer Jersey City, NJ