Data Engineer - AI-Assisted Data Ingestion Platform
Full-time
Twenty20 Systems
Role Description
We are looking for a Data Engineer to join our team and help build an AI-powered data ingestion platform. The platform collects data from multiple sources and loads it into analytical databases that power our B2B analytics product, BevGenie, where users can interact with their data using natural language.
In this role, you will work closely with a senior data engineer and use AI coding assistants (Claude Code) to build, improve, and maintain production-grade data pipelines. You will be responsible for extracting, cleaning, transforming, and loading data from both structured and unstructured sources.
- Design, build, and maintain scalable data ingestion pipelines
- Extract data from websites, APIs, documents (PDFs), emails, and databases
- Parse, clean, and transform structured and unstructured data
- Implement data validation, quality checks, and error handling
- Handle missing, duplicate, and malformed data
- Ensure pipelines are reliable and idempotent
- Orchestrate workflows using Dagster or similar tools
- Load data into PostgreSQL, Supabase, Snowflake, and MongoDB
- Collaborate with senior engineers and cross-functional teams
Qualifications
- 3–5 years of experience in Python-based development or data engineering
- Strong Python fundamentals (functions, classes, logging, error handling)
- Experience working with APIs and common data formats (JSON, CSV, XML)
- Strong SQL skills including joins, aggregations, CTEs, and window functions
- Understanding of relational database concepts, schema design, and indexing
- Basic knowledge of web scraping and data extraction techniques
- Familiarity with handling structured and unstructured data
- Understanding of data quality, validation, and pipeline reliability
- Good problem-solving and analytical skills
- Strong communication and collaboration skills
- Willingness to learn new tools and technologies
Requirements
- Experience with Dagster, Airflow, or Prefect
- Experience with web scraping tools (BeautifulSoup, Scrapy, Playwright)
- Experience with PDF extraction tools
- Exposure to AWS services (S3, Lambda, Glue)
- Knowledge of dimensional modeling or analytics data modeling
- Any experience using AI/LLMs for data extraction
Benefits
- Work closely with and learn from a senior data engineer
- Hands-on experience building real production data pipelines
- Exposure to modern AI-assisted development workflows
- Opportunity to work with diverse data sources and technologies
- Remote-first and flexible work environment
- Competitive compensation
- High-impact role in a growing product-focused team
Vacancy posted 6 days ago
Similar jobs that could be interesting for youBased on the Data Engineer - AI-Assisted Data Ingestion Platform in Remote vacancy
$191k - $287k
...Description As a Senior Staff Data Engineer at Afresh, you'll be one of... ...end to end: ~From raw data ingestion through transformation and delivery... ...customer data. ~Champion AI-forward engineering: evaluate... ..., Airflow, and modern data platforms (Databricks, Snowflake, or...PlatformFull timeHome officeFlexible hoursAfresh
Remote1 day ago$216k
...Description I'm looking for a Staff Data Engineer who can co-define the technical direction of our Data Platform, establish standards for other... ...for analytics, ML, and AI agents. ~Author and maintain... ...and have used a lakehouse to ingest multi-source performance data...PlatformFull timeTeamworks
Remote7 days ago$175k - $200k
...process of modernizing its data stack to build the foundation... ...we are hiring a Staff Data Engineer to build and own the platform backbone for governed reporting and trusted AI workflows. This is a hands-on... ...strategic architecture calls—from ingestion patterns and Snowflake...PlatformFull timeRemote workFlexible hoursFora Financial
Remote5 days ago- ...seeking an experienced Staff Data Engineer to architect, scale, and operationalize... ...’s creative intelligence platform. Vidmob turns creative into a... ...products driven by ML and AI, including LLMs, VLMs,... ...Own the Data Lifecycle: Drive ingestion, transformation, modeling, validation...PlatformPermanent employmentFull timeRemote work
Vidmob
Remote5 days ago $166.4k - $244.8k
Role Description The Data Platform team builds and operates... ...a Starburst query engine, Databricks Feature Store... ...data landscape and power AI at BILL. To achieve... ...data lifecycle, spanning ingest, store, enrich, query,... .... ~Employee Assistance Program (EAP). ~11+...PlatformFull timeTemporary workWork experience placementFlexible hoursBILL
Remote4 days ago- ...Artisan We're building AI employees. Not chatbots... ...employee can do. The engineering problems are hard and... ...You'll be the first Data Engineer on the Artisan... ...unstructured data Manage ingestion from third-party APIs,... ...Experience with cloud platforms (AWS, GCP, or Azure)...PlatformRemote work
Artisan
New York, NY11 hours ago $138.5k - $221.6k
...In this role, you'll join the Data & AI Foundations team to design... ...scale ~Own and evolve core platform assets, AE tooling, reusable... ...Define and deploy scalable data ingestion, replication, and transfer... ...consumption ~Proficiency with AI-assisted development tools such as...PlatformFull timeFlexible hoursHubSpot
Remote11 hours ago$145k - $175k
...Staff Data Engineer Forterra is seeking an experienced... ...components of the data platform. In this role, you will... ...pipelines for ingestion and transformation of... ...metrics, and KPIs and assist in the development of... ...with implementation of AI/ML technologies....PlatformFull timeTemporary workWork experience placementLocal areaRemote workSoftbank Investment Advisers
United States4 days ago$210k - $255k
...role In the role of Staff Data Engineer for Concepts Lab , you will... ...Own the reliability of AI systems in the lab, including... ...Spark, Databricks SQL), spanning ingestion from AWS and GCP sources through... ...between central Data and Platform teams and upstream and downstream...PlatformFlexible hoursCrunchyroll
Los Angeles, CA2 days ago- ...metabolic health clinic. We’re an AI‑native digital health system... ...are hiring a Senior or Staff Data Engineer to become the first dedicated... ...team. This is a high‑impact platform role where you’ll build the... ...RudderStack and Fivetran for ingestion, Snowflake as our warehouse,...PlatformFull timeWork at officeRemote work2 days per week3 days per week
$200k - $230k
...leading technology platform that helps... ...combines software, data, and service into... ...seamlessly integrated, AI‑driven platform that... ...need dedicated data engineering leadership to... ...reliable pipelines that ingest, transform, and... ...laws. If you need assistance during the application...PlatformPermanent employmentRemote workVisa sponsorshipFlexible hours$210k - $240k
Role Description As our Staff Data Platform Engineer, you'll be the technical lead: a senior individual... .... ~Able to effectively leverage AI-powered development tools (e.g., Claude... ...architecture of the Databricks Data Warehouse — ingestion, storage, compute, orchestration,...PlatformFull timeWork at officeRemote workWork from homeFlexible hoursTilt Finance
Remote6 days ago$268k - $368.5k
...a technology wholesale platform built on the belief that... ...the power of tech, data, and machine learning to... ...About this role Our Engineering organization owns the software... ...platform for data ingestion, storage, compute,... ...Artificial Intelligence (AI) to screen and select applicants...PlatformWork experience placementWork at officeLocal areaRemote workMonday to FridayFlexible hours3 days per weekFaire Inc
San Francisco, CA5 hours ago$225k - $290k
...define and drive the strategy for data reliability, quality, and... ...continuous improvement ~Influencing platform and architectural decisions to... ...~Defining best-in-class data engineering practices ~Leading complex,... ...~Experience leveraging AI tools and methodologies to design...PlatformFull timeCircle
Remote6 days ago- ...following client to help find a Software Engineer, Staff II for their Data Ingestion team. The Data Ingestion Team... ...for critical parts of our ingestion platform, lead large-scale initiatives, and... ...tolerance, scalability) ~Maintains an AI-forward approach to software...PlatformFull timeWork from homeHome office
Plus10 Recruitment
Remote11 hours ago - Role Description We are looking for a Staff Data Engineer with significant experience to join our data team. Our data platform powers analytics for both internal business... ...existing data pipelines ~Ensure reliable data ingestion, transformation, and reporting across...PlatformFull timeRemote work
Rezdy
Remote11 hours ago - ...Description We're hiring a Senior to Staff-level Data Engineer to build and evolve the data layer that... ...and serve data for analytics and platform features ~Ensure data quality,... ...testing, code review, CI/CD ~Fluency with AI-assisted development tools and workflows ~...PlatformFull timeWork at officeRemote work
GRIDSIGHT
Remote5 days ago $180.2k - $252.3k
...Description Reddit’s Corporate Engineering Team is looking for a highly experienced Staff Data Engineer to advance the data architecture... ...into a scalable, well-modeled platform that serves Finance... ...Hands-on experience architecting ingestion patterns that land high-volume...PlatformFull timeFlexible hoursReddit
Remote3 days ago$145k - $170k
Role Description We are seeking a Staff Data Engineer to serve as the core developer and owner of our data pipelines and platform tools. In this role you will focus on execution... ...of end-to-end data pipelines from ingestion to analytics application delivery. You will...PlatformFull timeNABIS
Remote3 days ago- Role Description As Staff Data Engineer, you will provide senior onshore... ...centralized Databricks data platform with full accountability for... ...building pipelines that serve AI/ML workloads (preparing data... ...insurance, will preparation, and an employee assistance program....PlatformFull time
Exactera
Remote1 day ago - Role Description This is the founding Data Engineer for the Data Engineering team at Payabli.... ...data for years to come: ~Architect the platform: Set our warehouse/lakehouse direction and... ...of places it doesn't belong. ~Enable AI/ML and analytics: Build the feature...PlatformFull timeFlexible hours
Payabli
Remote1 day ago - Role Description The Staff Data Engineer is responsible for designing, building... ...Pub/Sub, or similar messaging platforms ~Strong experience with star... ...~Working knowledge of AI-based development tools and workflows (e.g., AI-assisted coding, LLM integration, prompt...PlatformWork experience placement
WellSky
Remote4 days ago - Role Description Awetomaton is seeking a Staff Data Engineer to support a new customer by building and maintaining the ingestion, curation, and metadata-enrichment pipelines... ...Familiarity with data management/governance platforms ~Cloud & platform certifications (AWS...PlatformFull timeWork at officeLocal areaFlexible hours
Awetomaton
Remote5 days ago $160k - $207.4k
Role Description We're hiring a Staff Data Engineer to architect and lead the data foundation... ...products. In this role, you'll own our data platform, visualization, advanced machine... ...quality, consistency, and performance. ~AI Innovation & Actionable Insights: Translate...PlatformFull timeRemote workWork from homeFlexible hoursBoulevard
Remote6 days ago$123.76k - $185.64k
Role Description The Senior Staff Data Engineer is responsible for the design and implementation... ...and deliver data for analytics, AI, business intelligence (BI) systems and... ...architecture based on business requirements and platform capabilities which may include defining...PlatformFull timeBayer
Remote7 days ago- ...Staff Data Engineer We are looking for a Staff Data Engineer to design... ...internal systems, vendor platforms, and the data warehouse Improve... ..., and thoughtful use of AI Experiment pragmatically with... ...wellness programs, and an employee assistance program. Foley is an Equal...PlatformContract workRemote work
Foley
United States3 days ago $159k - $187k
...technical anchor for an analytics engineering team—owning ontology design,... ...team's slice of our shared data model correct, durable, and trusted... ...on a shared, self-serve platform — and adopts an ontology-driven... ...ontology and semantic layer to be AI-consumable so tools like...PlatformFull timeTemporary workFlexible hoursKin Insurance
Remote3 days ago$196k - $230k
...Staff Data Engineer Denver, Colorado, United States; San Francisco, California, United States... ...Checkr Checkr is building the data platform to power safe and fair decisions. Over 1... ...and millions of people rely on Checkr for AI verification in the moments that matter...PlatformWork at officeLocal areaRemote workRelocationFlexible hours3 days per weekCheckr
San Francisco, CA11 hours ago$160k - $200k
...brighter way forward. About JLL and Data Engineering JLL is a leading professional... ...think big, and deliver production-grade platforms that transform how JLL operates - combining... ...deep hands-on engineering across data and AI agents, and the interpersonal skills to...PlatformDaily paidLocal areaShift workJLL
Chicago, IL1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer - AI-Assisted Data Ingestion Platform. Be the first to apply!
Related searches