Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Big Data Engineer

Innovative Logical Solutions

Be a part of a dynamic team that thrives on innovation and creativity, pushing the boundaries of what is possible in the software industry. At Innovative Logic Solutions, we believe that our people are our greatest asset. We are committed to attracting and retaining top talent in the software industry, and providing our employees with a dynamic and challenging work environment that fosters growth and development. Our team members come from diverse backgrounds and bring a wide range of skills and expertise to the table, but they all share a passion for creating innovative software solutions that solve real-world problems. Coordinate with business stakeholders to ensure solutions requirements and quality is honed within the development cycle. Developing new business initiatives by coordinating with design team and working on websites as per the defined scripting code. Developing full-fledged website with variety of content, ensuring that it is fully optimized with standard navigation tools, HTML scripts and CSS Framework. Monitoring the quality of the front-end code, complying the designs as per suggested Lilly specified guidelines, and ensuring its compliance with the Lilly technical standards. Designing reusability components in vue.js framework. Maintaining the web integration controls by providing proper CSS framework, identifying and resolving the issues related with web usage interface if needed and performing troubleshooting on all scripts related front end loading issues. Implementing effective user integration and documenting all processes involved in web development for preparing component user integrated website. Configuring Mocha/Chai test frameworks for front-end testing. Report status of projects via written and verbal communication as well as through tracking tools (JIRA) to other team members and other group disciplines within the company. Working as a part of development team in an agile environment with new iterations every week, with open communication with Project Managers, DDL’s, Testers and other team members. Coordinating with DDL’s for developing various Lilly Enterprise websites with elaborate features, responsive designs and user-based navigation options in it. Degree Requirement: A minimum of a bachelor’s degree in Computer Science, Computer Information Systems, Information Technology, or a closely related field , or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects. The Software Engineer will contribute to the evolutionary development of advanced analytics and decision support software across all aspects of the development life cycle, working closely with product development engineers and business domain experts. Skills, experience and qualifications required: Good knowledge of CI/CD pipelines for API / SPAs / Mobile Apps Excellent knowledge of tracking tools such as Jira, QC/Quality Center, TeamTrack or similar Knowledge of different testing methodologies - eg V model, WaterfallAgile,approach Desirable experience includes using QTP, Selenium, Test Complete, Unit, JMock, JSUnit, Fitnesse or equivalent tools Good experience in Test Management and bug tracking tools Good knowledge in mobile testing using IOS and Android platform Performance & Automation testing experience (ideally) Knowledge in programming (Java/Microsoft platform) and database skills (SQL, ETL, Business Intelligence) and cloud platforms like AWS and Azure Candidate must have a relevant degree Relevant certifications will be an added advantage Candidate must have extensive experience on relevant skills Gather Requirements from business groups and analyzing design solutions that automate and optimize business process using Python, Django, Hadoop, Hive, MySQL and Spark application Develop web applications in python and Django for Data and provide reports and manage infrastructure. Write test cases in python for unit testing as well as ETL testing. Work on processing large datasets of different forms including structured, semi-structured and unstructured data. Develop rest API’s using python with Django rest framework. Develop necessary documentation for custom modifications, work with development team to achieve new functionality without interrupting normal business cycle. Education Requirement: Bachelor’s Degree in Computer Science, Computer Information Systems, Information technology, or related, or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects. Previous experience as a data engineer or in a similar role Technical expertise with data models, data mining, and segmentation techniques Knowledge of programming languages (e.g. Java and Python) Hands-on experience with SQL database design Great numerical and analytical skills Degree in Computer Science, IT, or similar field; a Master’s is a plus Data engineering certification (e.g IBM Certified Data Engineer) is a plus The developer must have sound knowledge in Apache Spark and Python programming. Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations. Experience in deployment and operationalizing the code is added advantage. We are looking to hire a talented big data engineer to develop and manage our company’s Big Data solutions. In this role, you will be required to design and implement Big Data tools and frameworks, implement ELT processes, collaborate with development teams, build cloud platforms, and maintain the production system. To ensure success as a big data engineer, you should have in-depth knowledge of Hadoop technologies, excellent project management skills, and high-level problem-solving skills. A top-notch Big Data Engineer understands the needs of the company and institutes scalable data solutions for its current and future needs. Meeting with managers to determine the company’s Big Data needs. Developing Hadoop systems. Loading disparate data sets and conducting pre-processing services using Hive or Pig. Finalizing the scope of the system and delivering Big Data solutions. Managing the communications between the internal system and the survey vendor. Collaborating with the software research and development teams. Building cloud platforms for the development of company applications. Maintaining production systems. Bachelor’s degree in computer engineering or computer science. Previous experience as a big data engineer. In-depth knowledge of Hadoop, Spark, and similar frameworks. Knowledge of scripting languages including Java, C++, Linux, Ruby, PHP, Python, and R. Knowledge of NoSQL and RDBMS databases including Redis and MongoDB. Familiarity with Mesos, AWS, and Docker tools. Excellent project management skills. Ability to solve complex networking, data, and software issues. We are searching for a decisive and insightful DevOps engineer to join our reputable company. The DevOps engineer will be involved in various stages of each product's lifespan and should remain abreast of technological advancements to promote efficiency. You should also keep track of customer reviews to enhance marketability. To ensure success as a DevOps engineer, you should ensure that all outputs remain safe for public consumption. Ultimately, a top-notch DevOps engineer will demonstrate excellent leadership skills and the capacity to mentor junior staff. Collaborating with coworkers to conceptualize, develop, and release software. Conducting quality assurance to ensure that the software meets prescribed guidelines. Rolling out fixes and upgrades to software, as needed. Securing software to prevent security breaches and other vulnerabilities. Collecting and reviewing customers' feedback to enhance user experience. Suggesting alterations to workflow in order to improve efficiency and success. Pitching ideas for projects based on gaps in the market and technological advancements. Degree in computer science, software engineering, or an adjacent field. Prior experience at all phases of the software product life cycle, from conception to extinction. Experience using an array of automation tools. Familiarity with software security methods. Demonstrated experience using a wide variety of coding languages. Curious, discerning envelope pusher. Top-notch decision-making and collaborative abilities. Customer-oriented with the ability to receive and use constructive criticism. #J-18808-Ljbffr

Vacancy posted 3 hours ago
Similar jobs that could be interesting for youBased on the Big Data Engineer in Brooklyn, NY vacancy
  •  ...Purple DriveWe are seeking a highly skilled Palantir Data Engineer to design, develop, and maintain scalable data solutions using Palantir...  ...Required Skills & Experience:Strong expertise in Palantir Foundry and big data frameworks (Spark, Hadoop).Proficiency in Python, PySpark,... 
    Big data

    Purple Drive

    Jersey City, NJ
    5 days ago
  •  ...Data EngineerWe are seeking experienced Data Engineers in a contract capacity to join our talented Data Engineering team. In this role, you will contribute to...  ...warehousing tools like Snowflake or DatabricksFamiliarity with distributed systems and big data technologies... 
    Big data
    Contract work

    Argyle Infotech

    Brooklyn, NY
    4 days ago
  •  ...Job Title - Data Analytics Engineer Location - Secaucus, NJ(Remote) Mandatory skills - Eligible to workP, Big Query, Dataflow, Looker, and Python Key Responsibilities: We are seeking a skilled Data Analytics Engineer with strong experience in Google... 
    Big data
    Remote work

    Diverse Lynx

    Secaucus, NJ
    19 hours ago
  •  ...Responsibilities 1. Develop and implement data processing frameworks using Apache spark....  ...the latest trends and advancements in big data technologies. Skill Requirements 1....  ...scalable solutions. 4. Knowledge of data engineering concepts and methodologies. 5. Ability to... 
    Big data

    Omni Inclusive

    Jersey City, NJ
    6 days ago
  • Azure Data Engineer Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required...  ..., Azure Databricks, and other Azure services. Fluent with Big Data technologies like Spark, Kafka and Hive. Expert level... 
    Big data

    InterSources

    Weehawken, NJ
    6 days ago
  • West 4th Strategy is seeking a Data Engineer to support the National Guard Bureau's Minuteman project, designing and maintaining the enterprise data environment on the central analytics platform. You will build end-to-end data pipelines using Databricks, Python, Spark,... 
    Big data
    Work at office
    2 days per week
    1 day per week

    West 4th Strategy

    Brooklyn, NY
    6 days ago
  •  ...and results-oriented professional to join our dynamic team as a Data Engineer. Details of the same have been highlighted below: Roles and...  ...data services. Experience with Hadoop, Spark, Kafka, or other big data technologies is a plus. Strong understanding of relational... 
    Big data

    Ycotek

    Brooklyn, NY
    6 days ago
  • We are looking for a skilled and enthusiastic Data Engineer to help design and optimize our data infrastructure. JOB REQUIREMENTS: Create...  ...of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. Build analytics tools that utilize the data... 
    Big data
    Visa sponsorship
    Work visa

    BeaconFire Inc

    Brooklyn, NY
    6 days ago
  • Sr. Big Data Engineer Sonsoft, Inc. is a USA based corporation duly organized under the laws of the Commonwealth of Georgia. Sonsoft Inc. is growing at a steady pace specializing in the fields of Software Development, Software Consultancy and Information Technology Enabled... 
    Big data
    Permanent employment
    Full time
    H1b

    SonSoft

    Jersey City, NJ
    2 days ago
  • $107k

    The Data Engineer is responsible for designing, developing, and optimizing enterprise data pipelines and data platforms. This role focuses on...  ...in SQL and Python; familiarity with cloud platforms and big data technologies. Ability to diagnose data issues, optimize performance... 
    Big data

    Joint Commission

    Brooklyn, NY
    6 days ago
  • We have a full-time job opportunity of Data Engineer in San Francisco, California (Hybrid). Location: San Francisco, California (Hybrid, 3...  ...Responsibilities Design, develop, test, deploy, and maintain scalable Big Data solutions. Manage project priorities, deadlines, and... 
    Big data
    Full time
    Work at office

    NexDimension

    Brooklyn, NY
    5 days ago
  •  ...manage, optimize, and assist in the design of data pipelines in support of information,...  ...guidelines set by architects, senior and lead engineers. Deliver trustworthy data to ensure...  ...knowledge of transactional, analytical, big data structures preferred. Understanding... 
    Big data
    Full time
    Work at office

    Michigan Farm Bureau

    Brooklyn, NY
    4 days ago
  • Publicis Sapient seeks a Senior Associate Data Engineer in Chicago Hybrid to design, develop, and maintain scalable Big Data solutions. You will work on large datasets, real-time processing frameworks, and cloud-based data platforms to empower data-driven decisions for... 
    Big data

    Publicis Groupe

    Brooklyn, NY
    6 days ago
  • $65 - $70 per hour

    Position: Data Engineer- AWS Location: Jersey City, New Jersey Duration: Contract Job ID: 172543 Job Overview: As a Data Engineer III specializing...  ...formats like JSON, AVRO, or Protobuf. Knowledge of big-data storage formats such as Parquet or Iceberg. Familiarity with... 
    Big data
    Full time
    Contract work

    Pinnacle Group

    Jersey City, NJ
    6 days ago
  • $148.7k - $199.4k

     ...ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more - all working to build and...  ...spectrum of technologies, including machine learning, big data, microservices, and data visualization. We are... 
    Big data

    Disney Cruise Line - The Walt Disney Company

    Brooklyn, NY
    6 days ago
  • Our Sportsbook Data Team is looking for a Data Engineer to make a difference in the further development of our Data Warehouse solutions, designing and developing...  ...be a plus (.NET, Python, R, Scala) Knowledge around Big Data concepts and data streaming architectures will be a... 
    Big data
    Local area

    Betsson Group

    Brooklyn, NY
    2 days ago
  • $155k - $165k

    Data Engineer We need an experienced Data Engineer at the National Guard Bureau (NGB). The NGB is the link between the Army and Air National...  ...maintain the accuracy and integrity of large datasets Leverage big data technologies to efficiently process and analyze large datasets... 
    Big data
    Full time
    Work at office
    2 days per week
    1 day per week

    West 4th Strategy

    Brooklyn, NY
    6 days ago
  •  ...provider of high-quality, AI-powered consumer data, intelligence, and technology, empowering...  ...to join our team! Plenty of data engineering roles are about keeping the pipeline alive...  ...terabytes at a time. You will be on the big data team designing, building, and maintaining... 
    Big data
    Local area
    Remote work
    Work from home
    Flexible hours
    Shift work

    Resonate

    Brooklyn, NY
    3 days ago
  •  ...Virginia-based small business with over 150 years of collective experience, we pride ourselves on providing industry-leading expertise, big data analytics, and all-source research and analysis. Our technical capabilities directly bolster our customers' objectives, from... 
    Big data
    Flexible hours

    AVIAN, Inc.

    Brooklyn, NY
    2 days ago
  • Hi, Position: Lead Data Engineer Experience Required: 12 years+ Location: Jersey City, NJ | Onsite Employment Type: Full-Time NOTE - Must...  ...in delivering enterprise-scale projects. Strong expertise in Big Data concepts , distributed systems, and cloud-native architectures... 
    Big data
    Full time

    Centraprise

    Jersey City, NJ
    6 days ago
  •  ...Solutions (LCS) IT team, help build scalable data pipelines and build out the physical data...  ...latest technologies and trends in data engineering and apply them to improve existing data...  ...security model implementation. Experience with big data technologies such as Spark or Kafka.... 
    Big data
    Part time
    For contractors
    Local area
    Flexible hours

    Exxon Mobil

    Brooklyn, NY
    6 days ago
  • Data Engineer page is loaded## Data Engineerlocations: Houston, TX: Stamford, CTtime type: Full timeposted on: Posted Todayjob requisition id...  ...on project topics such as vendor trends, usage best practices, big data trends, artificial intelligence, vendors, etc.* Help... 
    Big data
    Work experience placement
    Work at office
    Local area

    Castleton Commodities International, LLC

    Brooklyn, NY
    2 days ago
  • About the job Lead Data Engineer Job Title: Lead Data Engineer No of Positions:2 Location: Jersey City/ Bedford, NJ (Hybrid) (3 days from office...  ...Key Skills: Snowflake, SQL, Python, Spark, AWS-Glue, Big Data Concepts Responsibilities: Lead the design, development, and... 
    Big data
    Work experience placement
    Work at office
    3 days per week

    Inizio Partners

    Jersey City, NJ
    6 days ago
  • $128k - $169k

     ...unlocking the world of connected vehicle data to power innovative products and services...  ...ecosystem. Job Overview The Senior Data Engineer reports to the VP of Engineering and is responsible...  ...workloads using Apache Spark and related big-data technologies; improve partitioning,... 
    Big data
    Casual work
    Work at office
    Flexible hours

    Evolving Solution Services

    Brooklyn, NY
    2 days ago
  • $140k - $145k

     ...Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, serving notable financial services and technology firms....  ...: AKV, UAMI. ETL experience is mandatory. Knowledge of Big Data technologies like Spark and Pyspark. Experience applying... 
    Big data
    Full time
    Flexible hours

    Synechron

    Weehawken, NJ
    3 days ago
  •  ...push the limits of what's possible. As a Lead Software Engineer- Financial Service Data Engineering: Pyspark / Java / BigData / Datalake / AI, at...  ...analytics-ready datasets Deep hands‑on experience with big data and modern data platforms (e.g., Spark, Databricks, Snowflake... 
    Big data

    JPMorgan Chase & Co.

    Jersey City, NJ
    16 days ago
  • $61 - $66 per hour

    Job Details Information Technology GCP Data Engineer Dearborn, MI Posted:9/21/2026 Job Description Job ID# 64967 Job Category: Information...  ...operational stores. Job Requirements Skills Required GCP, Big Data, BigQuery, Artificial Intelligence & Expert Systems, API... 
    Big data
    Contract work

    Stefanini, Inc

    Brooklyn, NY
    3 days ago
  •  ...you can push the limits of what's possible. As a Lead Software Engineer Data Platform & Entitlements at JPMorganChase within the Commercial...  ..., Lake house, Delta, IcebergHands-on experience with Spark and big data processing at scaleExperience with Identity and Access Management... 
    Big data

    JP Morgan Chase

    Jersey City, NJ
    1 day ago
  • We are seeking an experienced Senior Data Engineer to support complex data engineering initiatives within our insurance data and analytics practice.Base Compensation Range: 100,000 - 125,000The posted range is the hiring range for this role — a subset of the broader range... 

    EXL Service

    Jersey City, NJ
    2 days ago
  • $125k - $190k

     ...Data Center Engineer We are seeking a dedicated and experienced Data Center Engineer to join our team in Secaucus, NJ. This role requires hands-on work with our physical servers, network infrastructure, and IT components across various colocation facilities. You'll... 
    Permanent employment
    Full time
    Work at office
    Local area
    Remote work
    Monday to Friday
    Flexible hours
    Afternoon shift

    IMC Inc

    Secaucus, NJ
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!