Big Data Developer
eNcloud
Job Duties and Responsibilities
- Build processes that support data transformation, data structures, metadata, dependency and workload management
- Build and Implement data ingestion and curation process developed using Big data tools such as Spark (Scala/python), Hive, Spark, HDFS, Sqoop, Hbase, Kerberos, Sentry and Impala etc.
- Handle ingestion of huge volumes data from various platforms for Analytics needs and writing high-performance, reliable and maintainable ETL code Strong SQL knowledge and data analysis skills for data anomaly detection and data quality assurance.
- Write shell scripts. Complex SQL queries, Hadoop commands and Git.
- Create Database, Schemas, Hive tables (External and Managed) with various file formats (Orc, Parquet, Avro and Text etc.)
- Monitor performance of production jobs and advising any necessary infrastructure changes.
- Work on Code version using Bitbucket and CI/CD pipeline.
- Requirements: Bachelor’s degree in Computer Science, IT, Engg or related with at least 60 months of experience. Travel and relocation to various unanticipated client locations throughout the United States may be required. Equal Opportunity Employer.
Job Category
IT
#J-18808-LjbffrVacancy posted 17 hours ago
Similar jobs that could be interesting for youBased on the Big Data Developer in Boston, MA vacancy
- Req ID:373225NTT DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part... ....Technical Stack (Example)• Languages: Python, SQL, Scala• Big Data: Spark, Hadoop, Kafka• Cloud: AWS / Azure / GCP• AI/ML: MLflow...Big dataWork at officeRemote workFlexible hours
NTT DATA
Boston, MA3 days ago $195.3k - $199.2k
...team with responsibilities that range from developing and deploying our core enterprise... ...will design, develop, and maintain scalable data pipelines and systems using Databricks on... ...engineering, with a strong background in big data technologiesProficiency in Databricks...Big dataApprenticeshipEasy workMcKinsey & Company
Boston, MA2 days ago$120k - $150k
...experience through a powerful combination of passionate people and data-driven technology, strengthening our clients’ relationships with... ...deliver reliable, high-quality, and secure data. Key Outcomes Develop and maintain robust ETL/ELT pipelines focused on our core stack...Big dataFull timeFlexible hours- ...of cutting-edge technologies. R2 specializes in Java, Dot Net, Big Data, Cloud Computing, artificial intelligence (AI), machine... ...Full-time Overview: We are seeking a skilled and experienced Developer to join our team. The ideal candidate will have expertise in programming...Big dataFull timeWork experience placement
- ...Data Engineer The Data Engineer will play a crucial role in developing and fine-tuning data specifically for our LLMs and machine learning models. This individual will... ...in data engineering principles, experience with big data technologies, and a keen understanding of the...Big data
C the Signs
Boston, MA3 days ago - ...Title : Senior Data Engineer Location: Quincy, MA Onsite Type: Contract Responsibilities We are... ...Engineer who will be responsible for designing a solution for a big retail company. The main focus is to support processing of big data...Big dataContract work
Lorven Technologies
Quincy, MA5 days ago - Data Engineer Locations: Quincy, MA - 100% ONSITE Duration: 6 months Pay Range: $45 - $... ...strong experience with Databricks, including developing and optimizing spark jobs, data... ...development work Experience with apache big data implementation with programming proficiency...Big data
- Data Engineer Location: Boston, MA (Onsite role from Day 1) Duration: C2C In-person Interview Job Description: 10+ years of overall IT... ...experience in Java, J2EE and Python Solid tech skillset/experience in Big Data/Data Model, PySpark Strong knowledge on Spark technology -...Big data
- ...Data Engineer Employment Type: Full-Time, Mid-level CGS is seeking a passionate... ...data sets and models. ~ Experience with developing data pipelines from many sources from structured... ..., R, SQL, SAS). ~ Strong knowledge of big data analysis and storage tools and...Big dataFull timeFlexible hours
Contact Government Services LLC
Boston, MA4 days ago - ...would like to share a job opportunity as Data Engineer based in Quincy, MA (Onsite) location... ...Hadoop, Databricks, Scala Senior/Lead developer position who has extensive experience... ...bigdata developer with data engineering/big data developer. 3. The candidate should have...Big dataFull timeWork visa
- Senior Data Engineer Location: Boston, MA (Day one Onsite) Job Description: A robust background in AWS services such as Lambda, Glue... ..., prompt engineering, and MCP. Experience in designing & developing a big data solution using Spark, Scala, AWS Glue, Lambda, SNS/SQS, Cloudwatch...Big data
- ...Full Stack Data Engineer As a Full Stack Data Engineer within the Healthcare Graph team... ...products use advanced Cloud, Graph and Big Data technologies in analytical space to provide... ...and CI/CD pipelines Design and develop reusable frameworks, libraries, and components...Big data
Software Technology Inc
Boston, MA4 days ago - Data Engineer Database Developer Boston, MA Banking CRM JO-1905-1234 Are you interested in having a hand in updating and transforming the technical environment... ...), star schemas, exposure to NoSQL database technologies, Big Data Platforms like Hadoop or Cloud services like...Big data
- ...GCP Data Engineer Job Location: Woonsocket, RI / Dallas, TX (Onsite) Duration: Full... ...of data sources using SQL, and GCP 'big data' technologies. Work with data and... ...support their data infrastructure needs. Developing, supporting and monitoring data pipelines...Big dataFull timeWork visa
Syntricate Technologies
Boston, MA17 hours ago - As a Senior Data Engineer, CASM Platform, you will: Data Integration, API Development: Integrate... ...Engineering Optimization: Design, develop, and optimize large-scale ETL/ELT pipelines... ..., hands-on experience with Databricks and big data frameworks. Expertise in integrating...Big dataContract work
- Purple Drive Data Engineer (Local Candidates Only) The client's Security Architecture, Analytics... .... As a Data/Analytics Engineer, you will develop the data flows, analytics pipelines, and... ...core is highly desirable. Experience in big data technologies like Presto/Trino, Spark...Big dataWork experience placementLocal area
- ...We are looking for a skilled and motivated Data Engineer to join our team. In this role,... ...primary responsibilities will include: Design, develop, and maintain scalable ETL/ELT pipelines... ...professional experience Experience with big data technologies such as Spark, Kafka, or...Big dataWorldwideShift work
$135k - $165k
...About The Role We are seeking a Senior Data Engineer with deep expertise in data engineering... ...Support Managing Directors in developing and scoping client proposals where data engineering... ..., Apache Airflow) and familiarity with Big Data tools such as Spark or Scala Hands‑...Big dataPermanent employmentH1bLocal areaWorldwideHome office$131k - $220k
...collects, stores, processes and builds applications within our big data platform. Responsible for integrating these applications with the... ...and operational support aspects of team's responsibilities Develops deep understanding of data sources, granularity, availability,...Big dataFull timeLocal areaRelocation package- Job Title- Lead Data Engineer Location: Remote Duration: 6 Months Contract to Hire (Quarterly... ...analytics. We specialize in cutting-edge Big Data, Machine Learning, and Custom... ...an engineering mindset and an interest in developing enterprise scale solutions using industry...Big dataContract workRemote work
$125.5k - $261.6k
...markets and economies all over the world. We develop outstanding leaders who are teamed to... ...future. Our elite team of product leaders, data scientists, designers, and software engineers... ...pipelines, streaming architectures, and big data ecosystems. Cloud Computing Architecture...Big dataFull timeSummer holidayFlexible hours- Google Cloud Data Engineer Are you ready to step up to the New and take your technology expertise to the next level? Join and help transform... ...data pipelines from ingestion to consumption within a hybrid big data architecture, using Java, Python, Scala etc. Minimum 1 year...Big dataWork experience placement
$175.5k - $180k
...sitting on enormous clinical and operational data assets, but turning those assets into... ...will write optimized, production-ready code; develop reusable ML pipelines and model assets... ...and SQL; experience with PySpark or another big-data framework is a plusExperience with NLP...Big dataApprenticeshipEasy workMcKinsey & Company
Boston, MA2 days ago$181.2k - $229.5k
...world while gaining meaningful experience across a variety of disciplines.About the teamOur Data Science team is a high-impact research team actively shaping the future of TV, using Big Data to build and enhance the user experience on the Roku streaming platform. Our...Big dataWork at officeLocal areaRemote workMonday to ThursdayFlexible hoursRoku
Boston, MA1 hour ago- ...Overview We are seeking a Senior or Principal Data Scientist / Machine Learning Scientist to... ...decision making and model development. Develop and deploy machine learning models for... ...or matching algorithms. Proficiency in big data technologies (e.g., Spark, Dask,...Big dataWork at office
Datalign Advisory, LLC.
Cambridge, MA1 day ago - Data Scientist We are a new kind of travel company that uses big data to empower travelers. Our app for iOS and Android analyzes trillions of past and current trips to predict when you should fly and buy. The app notifies you when fares drop so you can book flights at...Big data
- Machine Learning Scientist BigR.io is a technology consulting firm empowering data to drive innovation and advanced analytics. We specialize in cutting-edge Big Data, Machine Learning, and Custom Software strategy, analysis, architecture, and implementation solutions....Big data
- ...security needs. The Group’s efforts include developing software architectures and systems,... ...support harmonization of medical and health data, web-based app development, modeling and... ...settingsDesired RequirementsExperience with Big Data cloud platforms like Databricks a...Big data
Lincoln Laboratory
Lexington, MA1 day ago $80 - $130 per hour
...technical expertise and culture fit. Responsibilities: Develop innovative approaches to solve challenging data and analytics problems and uncover actionable... ...optimization techniques. Experience with data warehouses, big data platforms, and cloud-based data services (AWS S...Big dataWork experience placementLocal area$100k - $130k
Overview Rue Gilt Groupe is looking for a Data Scientist to join our industry-leading Data... ...shopping experiences of our 50M members Develop innovative solutions to business problems... ...frameworks Strong knowledge of Spark or similar big data platforms and technologies...Big dataFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Big Data Developer. Be the first to apply!
Related searches
- senior data center engineer Boston, MA
- senior cloud data engineer Boston, MA
- senior data integration developer Boston, MA
- data developer Boston, MA
- entry level data engineer Boston, MA
- data engineer Boston, MA
- big data cloud engineer Boston, MA
- data platform engineer Boston, MA
- data engineer intern Boston, MA
- data engineering intern summer Boston, MA