AWS Data Engineer - Open Data Platform

$85 per hour

Swift Hire LLC

Location: NYC, NY | Remote | Contract | Payrate: $85

AWS Data Engineer Open Data Platform

We are seeking an experienced AWS Data Engineer to support the design and implementation of an AWS-based open data platform. The role will focus on building scalable data pipelines and integrating Amazon S3, AWS Glue, Apache Iceberg, and Apache Polaris with Snowflake and other analytics platforms.

Key Responsibilities

Design and develop data ingestion, transformation, and processing pipelines using AWS Glue, Apache Spark, and Amazon S3.
Build and manage Apache Iceberg tables supporting open, interoperable access across multiple compute engines.
Configure and integrate Apache Polaris to Iceberg REST Catalog with AWS and Snowflake.
Support platform security, access controls, networking, performance optimization, and operational monitoring.
Participate in workload benchmarking and evaluate the cost and performance of AWS Glue versus Snowflake.
Required Skills

Strong hands-on experience with Amazon S3, AWS Glue, Apache Spark, Iceberg, IAM, and AWS networking.
Experience designing and implementing cloud-based data pipelines and large-scale data processing solutions.
Good understanding of Apache Iceberg architecture, table maintenance, schema evolution, partitioning, and performance optimization.
Should have Install/configuration experiance in Apache Polaris and Iceberg over Amazon S3 files.

Apply

Vacancy posted 3 days ago

Similar jobs that could be interesting for youBased on the AWS Data Engineer - Open Data Platform in New York, NY vacancy

Data Engineer
$165k - $175k
...Data EngineerBoston, Massachusetts, United States Job Openings Data EngineerJob Title: Data Engineer - AI PlatformLocation: Boston, Chicago, New York City... ...sophisticated AI-powered platform that supports advanced, data... ...multi-cloud environments (AWS primarily; exposure to...
Platform
Amazon Web Service
RedBeard Solutions
New York, NY
5 days ago
Senior Data Engineer I - Amex Ads, Offers, and Dining Technology
$123k - $215.25k
...benefitsJob Function: Engineering & ArchitectureSchedule:... ...engineering community through open source, our mission is... ..., grounded in our data technology model that prioritizes... ..., offers, and dining platforms to elevate the... ...ingesting Rooam POS data from AWS S3 into Snowflake and...
Platform
Amazon Web Service
Worldwide
Visa sponsorship
American Express
New York, NY
6 days ago
Data Engineer
$100k - $115k
...000 + Expenses Paid (Travel on site) Data Engineer (Multiple Openings) US Citizens or Green Card Holders Only... ...shape a scalable enterprise data platform, and play a key role in delivering business... ...Snowflake experience Experience with AWS and/or Azure Advanced SQL skills...
Platform
Amazon Web Service
Remote work
Flexible hours
Areti Group
New York, NY
3 days ago
Senior Data Engineer
$137.4k - $161.7k
...West Monroe is seeking a Senior Data Engineer to join our Data & AI... ...and optimize modern cloud data platforms that enable advanced analytics... ...modern cloud data solutions using AWS, Azure, and GCP, integrating... ...inclusion and diversity, and openness to new ideas and perspectives...
Platform
Amazon Web Service
Local area
Immediate start
Flexible hours
West Monroe Partners
New York, NY
4 days ago
Sr. Data Engineer
...Sr. Data EngineerLocation: New York, NYDuration: C... ...US Citizen, GC, H1Pay: Open rateInterview: 1 - 2 rounds... ...Seeking a Senior Data Engineer to expand data... ...managing cloud-based data platforms. The ideal candidate will... ...roleExtensive experience with AWS data services: Redshift...
Platform
Amazon Web Service
Cardinal Integrated
New York, NY
4 days ago
Data Scientist / Engineer
$105k - $135k
...growing: The Impact Project is hiring a data engineer to join our small New York City-... ...with federal or other government open data sources (e.g. EPA AQS or AirNow... ...Census/ACS) Familiarity with cloud platforms and data hosting (e.g. AWS, Google Cloud) Data visualization...
Platform
Amazon Web Service
Full time
Local area
datadotorg
New York, NY
4 days ago
AWS St Data Engineer
...in New York is currently searching for a AWS Data Engineer. They are one of the largest Retail businesses... ...the build-out of a brand new data platform on AWS Data Lake. They are looking to hire 5 people for this team and are open to hiring both Contract and Full Time Employees...
Platform
Amazon Web Service
Full time
Contract work
InterEx Group
New York, NY
3 days ago
Sr Data Engineer
...hands-on experience building modern data pipelines and platforms using Python, SQL, and AWS services such as Glue, Lambda, S... ...-native analytics, leveraging open-source tools and machine... ...discipline, such as Mathematics, Engineering, Computer Science, Economics, or...
Platform
Amazon Web Service
Full time
TechTriad Inc
New York, NY
1 day ago
Data engineer
...areas of Computer Science, Engineering, Information Systems, Business... ...experience in working with data solutions. 3+ years of experience... ...data pipelines in AWS Cloud Platform (preferred), Azure, or Snowflake... ...Cassandra, MongoDB or other open source big data tools. Knowledge...
Platform
Amazon Web Service
TechDigital Group
New York, NY
1 day ago
Senior Data Engineer
...We're looking for a Senior Data Engineer that is innovative, curious, and... ...to help evolve the data platform that powers our sell-side media... ...with an emphasis on scalable, open data architectures. You’ll work... ...platforms, such as GCP & AWS Experience designing and building...
Platform
Amazon Web Service
Work experience placement
VALID8 Financial
New York, NY
4 days ago
Data Engineer
$60k - $80k
...–4 years) Eligibility: Open to U.S. Citizens and Green... ...that places elite engineering talent into high-impact... ...technologies across cloud, data, and software... ...Warehousing solutions using platforms like Snowflake or BigQuery... ...with cloud platforms like AWS , GCP , or Azure Nice...
Platform
Amazon Web Service
Full time
Contract work
Remote work
Aaratech
New York, NY
4 days ago
Senior Architect, Data Engineering
$215.2k - $244.7k
...clients developing Big Data strategy and roadmaps,... ...Responsibilities:Provide guidance and platform selection advice for... ...Models (LLMs), prompt engineering, embeddings, vector... ...platforms such as AWS, Azure, GCP etc.... ...inclusion and diversity, and openness to new ideas and perspectivesProven...
Platform
Amazon Web Service
Local area
Immediate start
Flexible hours
West Monroe Partners
New York, NY
6 days ago
Senior, Data Engineer
$96k - $140k
...Senior, Data EngineerNY, US, CA, US, IL, US - RemoteWhy Dept... ...approach to Product Managers, Engineers, Marketing, Sales, Finance... ..., SPSSExperience with cloud platforms such as AWS, GCP, Azure to use/deploy analytics... ...?We are a collaborative, open-door, best-idea-wins...
Platform
Amazon Web Service
Remote work
Flexible hours
Dept
New York, NY
1 day ago
Senior Manager, Data Engineering
$240k - $310k
.... We solve HR issues. We’re data-driven and never stop iterating... ..., data science, analytics engineering, platform data/ML engineering and AI... ...desire to know and help others.Openness - Your default is to be open... ...rigorStrong expertise with AWS services (S3, Lambda,...
Platform
Amazon Web Service
Casual work
Work at office
Local area
Justworks
New York, NY
8 days ago
Senior Data Engineer, Fleet Monitoring & Analysis
$153k - $204k
...pioneers, CoreWeave delivers a platform of technology, tools, and... ...operations.About the role:As a Senior Data Engineer you will own and evolve the... ...of cloud platforms (e.g., AWS, GCP, Azure) and related data... ...and enjoy working with open-source data tooling.You’re an...
Platform
Amazon Web Service
Permanent employment
Full time
Temporary work
Casual work
Work at office
Flexible hours
CoreWeave
New York, NY
5 days ago
Data & Analytics Engineer
...science to patients. Our platform automates the complexity... ...About the role As a Data & Analytics Engineer at Forus, you will build the... ...dbt, Fivetran, Airflow, GCP/AWS/Snowflake, Metabase (or equivalents... ...from the ground up. Stay open to changing your mind. Resourceful...
Platform
Amazon Web Service
Full time
Work at office
Forus Inc
New York, NY
4 days ago
Staff Data Engineer
...Staff Data EngineerWe're hiring a Staff Data Engineer to own the data platform layer at Verse. This is a high-impact role where you'll... ...someone succeed in this role, we are open to being convinced by your... ...and observability best practices.AWS infrastructure fluency,...
Platform
Amazon Web Service
Work at office
Verse Medical
New York, NY
4 days ago
Remote Data Engineer (Remote / Contract)
$30 - $50 per hour
...supply technologies fusing data, media, and technology... ...’s top demand-side platforms. Learn more at verve.com... ...are looking for a Data Engineer to join our engineering... ...should be willing and open to collaborate with our... ...Familiarity with Google Cloud or AWS big data products We’re...
Platform
Amazon Web Service
Hourly pay
Contract work
Work experience placement
Local area
Remote work
GrabJobs
New York, NY
1 day ago
Data Engineer
$165k - $180k
...skilled and motivated Sr. AI Data Engineer with a proven track record in building scalable data platforms and incorporating Generative AI... ...seamlessly integrating a variety of open-source tools to enhance data... ...: 3+ years of experience with AWS data services (e.g., S3, Glue,...
Platform
Amazon Web Service
Work experience placement
H1b
Work at office
Local area
Karsun Solutions
New York, NY
4 days ago
Data Engineer II - Digital and Technology Partners - Hybrid/Remote
$90k - $135.29k
...42nd Street, New York, NYThe Data Engineer II focuses on data collection... ...sets of data using multiple platforms to allow for Research and Data... ...technologies and languages with an open mind and without technology... ...implementation on Azure or AWS, is a big plus. Experience...
Platform
Amazon Web Service
Traineeship
Local area
Remote work
Flexible hours
Mount Sinai Health System
New York, NY
4 days ago
Lead Data Engineer
...Commerce Orchestration Engine helping enterprise retailers... ...We are hiring a Lead Data Engineer to drive delivery... ...our analytics and data platform initiatives. This role... ...with sources in Azure, AWS, GCP, and others Why Logicbroker... ...in your growth, open to your ideas, and committed...
Platform
Amazon Web Service
For contractors
Work at office
Remote work
Logicbroker
New York, NY
4 days ago
Lead Data Engineer
...Lead Data EngineerLocations: Richmond - VA / McLean... ...deployment of scalable data engineering solutions that deliver... ...technologies like AWS.Develop and maintain efficient... ...workflows.Knowledge of open table formats such as... ...us to grow as a global platform and they are causally...
Platform
Amazon Web Service
Work at office
InterSources
New York, NY
5 days ago
Data Engineer
...Data Engineer Opportunity At Independence Pet HoldingsEstablished in 20... ...building and operating the data platform, governance framework, and... ..., and other cloud platforms (AWS, Snowflake).Apply best practices... ...CCPA).Prior contributions to open-source projects, internal...
Platform
Amazon Web Service
Temporary work
Work at office
Remote work
Flexible hours
Independence Pet Group
New York, NY
3 days ago
Data Engineer
$64.89k - $173.04k
...We are hiring a fully remote Data Engineer with strong SQL and Python experience... ...of data pipelines and platform infrastructure that power a... ...This is a fully remote position open to candidates anywhere in the... ...platforms (RabbitMQ, AWS SNS/SQS). Familiarity with containerized...
Platform
Amazon Web Service
Hourly pay
Full time
Temporary work
Work experience placement
Local area
Remote work
CVS Health
New York, NY
1 day ago
Senior Data Engineer
$126k - $180k
...a global crypto and Web3 platform founded by Cameron and Tyler... ...in a way that is more open, fair, and secure. As a publicly... ...impact. The Department: Data At Gemini, our Data Team is the engine that powers insight,... ...optimizing Databricks on AWS It Pays to Work Here The compensation...
Platform
Amazon Web Service
Contract work
Work at office
Remote work
Flexible hours
Gemini Personnel Pte Ltd
New York, NY
4 days ago
Data Engineer - Senior Manager
$124k - $280k
...Data And Analytics Engineering Senior ManagerAt PwC, our people in data and analytics engineering focus... ...focus on staying relevant.Initiate open and honest coaching conversations... ...Sets You Apart:Certification in Cloud Platforms [e.g., AWS Solutions Architect, AWS Data...
Platform
Amazon Web Service
PwC (US)
New York, NY
5 days ago
Staff Data Engineer
...building and operating the data integrations that feed... ...'s mission-critical platform. Twenty builds software... ...problems. This is a hands-on engineering role with real... ...solution to ambiguous, open-ended problems. New data... ...applications, primarily as AWS Glue jobs written in...
Platform
Amazon Web Service
Full time
Work at office
Remote work
Twenty Inc.
New York, NY
4 days ago
Snowflake Lead - Data Engineering
...Snowflake Lead – Data Engineering San Antonio, TX / New York, NY (Onsite) Local to San... ...-on expertise in Talend ETL / Talend Open Studio * Strong Python programming... ...AI / AI-enabled analytics * Strong AWS Cloud data platform experience * Expertise in Data Engineering...
Platform
Amazon Web Service
Temporary work
Local area
Delan Associates, Inc
New York, NY
a month ago
Director, Data Engineering & AI
$200k - $250k
...opportunity for a strategic Director, Data Engineering & AI to join our Technology... ..., including data platform transformation, data quality... ...within Databricks, Snowflake, AWS, Azure, and/or GCP environments... ...and diversity, and openness to new ideas and perspectivesAbility...
Platform
Amazon Web Service
Work at office
Local area
Immediate start
Flexible hours
2 days per week
1 day per week
West Monroe Partners
New York, NY
7 days ago
Staff Data Engineer
$190k - $250k
...Data ArchitectOwn and drive the technical architecture... ...and distributed data platforms in a cloud-native environment (Azure, AWS, or GCP).Architect and lead... ..., and BI consumers.Set engineering standards and best practices... ...speaking, writing, or open-source contributions...
Platform
Amazon Web Service
Full time
Temporary work
Flexible hours
Newmark ltd
New York, NY
4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AWS Data Engineer - Open Data Platform. Be the first to apply!