Big Data Engineer
Innovative Logical Solutions
Be a part of a dynamic team that thrives on innovation and creativity, pushing the boundaries of what is possible in the software industry. At Innovative Logic Solutions, we believe that our people are our greatest asset. We are committed to attracting and retaining top talent in the software industry, and providing our employees with a dynamic and challenging work environment that fosters growth and development. Our team members come from diverse backgrounds and bring a wide range of skills and expertise to the table, but they all share a passion for creating innovative software solutions that solve real-world problems. Coordinate with business stakeholders to ensure solutions requirements and quality is honed within the development cycle. Developing new business initiatives by coordinating with design team and working on websites as per the defined scripting code. Developing full-fledged website with variety of content, ensuring that it is fully optimized with standard navigation tools, HTML scripts and CSS Framework. Monitoring the quality of the front-end code, complying the designs as per suggested Lilly specified guidelines, and ensuring its compliance with the Lilly technical standards. Designing reusability components in vue.js framework. Maintaining the web integration controls by providing proper CSS framework, identifying and resolving the issues related with web usage interface if needed and performing troubleshooting on all scripts related front end loading issues. Implementing effective user integration and documenting all processes involved in web development for preparing component user integrated website. Configuring Mocha/Chai test frameworks for front-end testing. Report status of projects via written and verbal communication as well as through tracking tools (JIRA) to other team members and other group disciplines within the company. Working as a part of development team in an agile environment with new iterations every week, with open communication with Project Managers, DDL’s, Testers and other team members. Coordinating with DDL’s for developing various Lilly Enterprise websites with elaborate features, responsive designs and user-based navigation options in it. Degree Requirement: A minimum of a bachelor’s degree in Computer Science, Computer Information Systems, Information Technology, or a closely related field , or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects. The Software Engineer will contribute to the evolutionary development of advanced analytics and decision support software across all aspects of the development life cycle, working closely with product development engineers and business domain experts. Skills, experience and qualifications required: Good knowledge of CI/CD pipelines for API / SPAs / Mobile Apps Excellent knowledge of tracking tools such as Jira, QC/Quality Center, TeamTrack or similar Knowledge of different testing methodologies - eg V model, WaterfallAgile,approach Desirable experience includes using QTP, Selenium, Test Complete, Unit, JMock, JSUnit, Fitnesse or equivalent tools Good experience in Test Management and bug tracking tools Good knowledge in mobile testing using IOS and Android platform Performance & Automation testing experience (ideally) Knowledge in programming (Java/Microsoft platform) and database skills (SQL, ETL, Business Intelligence) and cloud platforms like AWS and Azure Candidate must have a relevant degree Relevant certifications will be an added advantage Candidate must have extensive experience on relevant skills Gather Requirements from business groups and analyzing design solutions that automate and optimize business process using Python, Django, Hadoop, Hive, MySQL and Spark application Develop web applications in python and Django for Data and provide reports and manage infrastructure. Write test cases in python for unit testing as well as ETL testing. Work on processing large datasets of different forms including structured, semi-structured and unstructured data. Develop rest API’s using python with Django rest framework. Develop necessary documentation for custom modifications, work with development team to achieve new functionality without interrupting normal business cycle. Education Requirement: Bachelor’s Degree in Computer Science, Computer Information Systems, Information technology, or related, or a combination of education and experience equating to the U.S. equivalent of a Bachelor’s degree in one of the aforementioned subjects. Previous experience as a data engineer or in a similar role Technical expertise with data models, data mining, and segmentation techniques Knowledge of programming languages (e.g. Java and Python) Hands-on experience with SQL database design Great numerical and analytical skills Degree in Computer Science, IT, or similar field; a Master’s is a plus Data engineering certification (e.g IBM Certified Data Engineer) is a plus The developer must have sound knowledge in Apache Spark and Python programming. Deep experience in developing data processing tasks using pySpark such as reading data from external sources, merge data, perform data enrichment and load in to target data destinations. Experience in deployment and operationalizing the code is added advantage. We are looking to hire a talented big data engineer to develop and manage our company’s Big Data solutions. In this role, you will be required to design and implement Big Data tools and frameworks, implement ELT processes, collaborate with development teams, build cloud platforms, and maintain the production system. To ensure success as a big data engineer, you should have in-depth knowledge of Hadoop technologies, excellent project management skills, and high-level problem-solving skills. A top-notch Big Data Engineer understands the needs of the company and institutes scalable data solutions for its current and future needs. Meeting with managers to determine the company’s Big Data needs. Developing Hadoop systems. Loading disparate data sets and conducting pre-processing services using Hive or Pig. Finalizing the scope of the system and delivering Big Data solutions. Managing the communications between the internal system and the survey vendor. Collaborating with the software research and development teams. Building cloud platforms for the development of company applications. Maintaining production systems. Bachelor’s degree in computer engineering or computer science. Previous experience as a big data engineer. In-depth knowledge of Hadoop, Spark, and similar frameworks. Knowledge of scripting languages including Java, C++, Linux, Ruby, PHP, Python, and R. Knowledge of NoSQL and RDBMS databases including Redis and MongoDB. Familiarity with Mesos, AWS, and Docker tools. Excellent project management skills. Ability to solve complex networking, data, and software issues. We are searching for a decisive and insightful DevOps engineer to join our reputable company. The DevOps engineer will be involved in various stages of each product's lifespan and should remain abreast of technological advancements to promote efficiency. You should also keep track of customer reviews to enhance marketability. To ensure success as a DevOps engineer, you should ensure that all outputs remain safe for public consumption. Ultimately, a top-notch DevOps engineer will demonstrate excellent leadership skills and the capacity to mentor junior staff. Collaborating with coworkers to conceptualize, develop, and release software. Conducting quality assurance to ensure that the software meets prescribed guidelines. Rolling out fixes and upgrades to software, as needed. Securing software to prevent security breaches and other vulnerabilities. Collecting and reviewing customers' feedback to enhance user experience. Suggesting alterations to workflow in order to improve efficiency and success. Pitching ideas for projects based on gaps in the market and technological advancements. Degree in computer science, software engineering, or an adjacent field. Prior experience at all phases of the software product life cycle, from conception to extinction. Experience using an array of automation tools. Familiarity with software security methods. Demonstrated experience using a wide variety of coding languages. Curious, discerning envelope pusher. Top-notch decision-making and collaborative abilities. Customer-oriented with the ability to receive and use constructive criticism. #J-18808-Ljbffr
- ...Purple DriveWe are seeking a highly skilled Palantir Data Engineer to design, develop, and maintain scalable data solutions using Palantir... ...Required Skills & Experience:Strong expertise in Palantir Foundry and big data frameworks (Spark, Hadoop).Proficiency in Python, PySpark,...Big data
- ...Data EngineerWe are seeking experienced Data Engineers in a contract capacity to join our talented Data Engineering team. In this role, you will contribute to... ...warehousing tools like Snowflake or DatabricksFamiliarity with distributed systems and big data technologies...Big dataContract work
- ...Job Title - Data Analytics Engineer Location - Secaucus, NJ(Remote) Mandatory skills - Eligible to workP, Big Query, Dataflow, Looker, and Python Key Responsibilities: We are seeking a skilled Data Analytics Engineer with strong experience in Google...Big dataRemote work
- ...Responsibilities 1. Develop and implement data processing frameworks using Apache spark.... ...the latest trends and advancements in big data technologies. Skill Requirements 1.... ...scalable solutions. 4. Knowledge of data engineering concepts and methodologies. 5. Ability to...Big data
- Azure Data Engineer Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required... ..., Azure Databricks, and other Azure services. Fluent with Big Data technologies like Spark, Kafka and Hive. Expert level...Big data
- West 4th Strategy is seeking a Data Engineer to support the National Guard Bureau's Minuteman project, designing and maintaining the enterprise data environment on the central analytics platform. You will build end-to-end data pipelines using Databricks, Python, Spark,...Big dataWork at office2 days per week1 day per week
- ...and results-oriented professional to join our dynamic team as a Data Engineer. Details of the same have been highlighted below: Roles and... ...data services. Experience with Hadoop, Spark, Kafka, or other big data technologies is a plus. Strong understanding of relational...Big data
- We are looking for a skilled and enthusiastic Data Engineer to help design and optimize our data infrastructure. JOB REQUIREMENTS: Create... ...of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies. Build analytics tools that utilize the data...Big dataVisa sponsorshipWork visa
- Sr. Big Data Engineer Sonsoft, Inc. is a USA based corporation duly organized under the laws of the Commonwealth of Georgia. Sonsoft Inc. is growing at a steady pace specializing in the fields of Software Development, Software Consultancy and Information Technology Enabled...Big dataPermanent employmentFull timeH1b
$107k
The Data Engineer is responsible for designing, developing, and optimizing enterprise data pipelines and data platforms. This role focuses on... ...in SQL and Python; familiarity with cloud platforms and big data technologies. Ability to diagnose data issues, optimize performance...Big data- We have a full-time job opportunity of Data Engineer in San Francisco, California (Hybrid). Location: San Francisco, California (Hybrid, 3... ...Responsibilities Design, develop, test, deploy, and maintain scalable Big Data solutions. Manage project priorities, deadlines, and...Big dataFull timeWork at office
- ...manage, optimize, and assist in the design of data pipelines in support of information,... ...guidelines set by architects, senior and lead engineers. Deliver trustworthy data to ensure... ...knowledge of transactional, analytical, big data structures preferred. Understanding...Big dataFull timeWork at office
- Publicis Sapient seeks a Senior Associate Data Engineer in Chicago Hybrid to design, develop, and maintain scalable Big Data solutions. You will work on large datasets, real-time processing frameworks, and cloud-based data platforms to empower data-driven decisions for...Big data
$65 - $70 per hour
Position: Data Engineer- AWS Location: Jersey City, New Jersey Duration: Contract Job ID: 172543 Job Overview: As a Data Engineer III specializing... ...formats like JSON, AVRO, or Protobuf. Knowledge of big-data storage formats such as Parquet or Iceberg. Familiarity with...Big dataFull timeContract work$148.7k - $199.4k
...ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists, and more - all working to build and... ...spectrum of technologies, including machine learning, big data, microservices, and data visualization. We are...Big data- Our Sportsbook Data Team is looking for a Data Engineer to make a difference in the further development of our Data Warehouse solutions, designing and developing... ...be a plus (.NET, Python, R, Scala) Knowledge around Big Data concepts and data streaming architectures will be a...Big dataLocal area
$155k - $165k
Data Engineer We need an experienced Data Engineer at the National Guard Bureau (NGB). The NGB is the link between the Army and Air National... ...maintain the accuracy and integrity of large datasets Leverage big data technologies to efficiently process and analyze large datasets...Big dataFull timeWork at office2 days per week1 day per week- ...provider of high-quality, AI-powered consumer data, intelligence, and technology, empowering... ...to join our team! Plenty of data engineering roles are about keeping the pipeline alive... ...terabytes at a time. You will be on the big data team designing, building, and maintaining...Big dataLocal areaRemote workWork from homeFlexible hoursShift work
- ...Virginia-based small business with over 150 years of collective experience, we pride ourselves on providing industry-leading expertise, big data analytics, and all-source research and analysis. Our technical capabilities directly bolster our customers' objectives, from...Big dataFlexible hours
- Hi, Position: Lead Data Engineer Experience Required: 12 years+ Location: Jersey City, NJ | Onsite Employment Type: Full-Time NOTE - Must... ...in delivering enterprise-scale projects. Strong expertise in Big Data concepts , distributed systems, and cloud-native architectures...Big dataFull time
- ...Solutions (LCS) IT team, help build scalable data pipelines and build out the physical data... ...latest technologies and trends in data engineering and apply them to improve existing data... ...security model implementation. Experience with big data technologies such as Spark or Kafka....Big dataPart timeFor contractorsLocal areaFlexible hours
- Data Engineer page is loaded## Data Engineerlocations: Houston, TX: Stamford, CTtime type: Full timeposted on: Posted Todayjob requisition id... ...on project topics such as vendor trends, usage best practices, big data trends, artificial intelligence, vendors, etc.* Help...Big dataWork experience placementWork at officeLocal area
- About the job Lead Data Engineer Job Title: Lead Data Engineer No of Positions:2 Location: Jersey City/ Bedford, NJ (Hybrid) (3 days from office... ...Key Skills: Snowflake, SQL, Python, Spark, AWS-Glue, Big Data Concepts Responsibilities: Lead the design, development, and...Big dataWork experience placementWork at office3 days per week
$128k - $169k
...unlocking the world of connected vehicle data to power innovative products and services... ...ecosystem. Job Overview The Senior Data Engineer reports to the VP of Engineering and is responsible... ...workloads using Apache Spark and related big-data technologies; improve partitioning,...Big dataCasual workWork at officeFlexible hours$140k - $145k
...Artificial Intelligence, Consulting, Digital, Cloud & DevOps, Data, and Software Engineering, serving notable financial services and technology firms.... ...: AKV, UAMI. ETL experience is mandatory. Knowledge of Big Data technologies like Spark and Pyspark. Experience applying...Big dataFull timeFlexible hours- ...push the limits of what's possible. As a Lead Software Engineer- Financial Service Data Engineering: Pyspark / Java / BigData / Datalake / AI, at... ...analytics-ready datasets Deep hands‑on experience with big data and modern data platforms (e.g., Spark, Databricks, Snowflake...Big data
$61 - $66 per hour
Job Details Information Technology GCP Data Engineer Dearborn, MI Posted:9/21/2026 Job Description Job ID# 64967 Job Category: Information... ...operational stores. Job Requirements Skills Required GCP, Big Data, BigQuery, Artificial Intelligence & Expert Systems, API...Big dataContract work- ...you can push the limits of what's possible. As a Lead Software Engineer Data Platform & Entitlements at JPMorganChase within the Commercial... ..., Lake house, Delta, IcebergHands-on experience with Spark and big data processing at scaleExperience with Identity and Access Management...Big data
- We are seeking an experienced Senior Data Engineer to support complex data engineering initiatives within our insurance data and analytics practice.Base Compensation Range: 100,000 - 125,000The posted range is the hiring range for this role — a subset of the broader range...
$125k - $190k
...Data Center Engineer We are seeking a dedicated and experienced Data Center Engineer to join our team in Secaucus, NJ. This role requires hands-on work with our physical servers, network infrastructure, and IT components across various colocation facilities. You'll...Permanent employmentFull timeWork at officeLocal areaRemote workMonday to FridayFlexible hoursAfternoon shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Engineer. Be the first to apply!
- senior data center engineer Brooklyn, NY
- senior cloud data engineer Brooklyn, NY
- senior data integration developer Brooklyn, NY
- data developer Brooklyn, NY
- data engineer Brooklyn, NY
- big data cloud engineer Brooklyn, NY
- data platform engineer Brooklyn, NY
- data engineering intern summer Brooklyn, NY
- data engineer analytics Brooklyn, NY
- data engineer manager Brooklyn, NY



