Data Engineer (ADF, Orchestration, Data Flows, Snowflake) | REMOTE
Kaav Inc.
- 10+Years of IT experience
- Background candidates should ideally have come from SSIS /SQL transitioning to Azure Data Factory (Data Integration) Snowflake (Source and Target End Point)
- Azure has a lot of components to it such as Fabric, DataBricks, ADF, Pyspark
- They need candidates that can use ADF as both orchestration and integration
- many candidates claim to have ADF but for orchestration really just use Pyspark to trigger the jobs through ADF
- They need to be advanced at Building the Integration through data flows logic as well
Project:
- As a member of the Data Management team, the Data Engineer supports the Alabama EDS by developing and maintaining workflows, identifying, and resolving data quality issues, and optimizing processes to improve performance.
- The Data Engineer will also support intrastate agencies by monitoring automated data extracts and working directly with state partners to create new extracts based on business specifications.
- Develop and manage effective working relationships with other departments, groups, and personnel with whom work must be coordinated or interfaced
- Efficiently communicate with ETL architect while understanding the requirements and business process knowledge in order to transform the data in a way that's geared towards the needs of end users
- Assist in the overall architecture of the ETL Design, and proactively provide inputs in designing, implementing, and automating the ETL flows
- Investigate and mine data to identify potential issues within ETL pipelines, notify end-users and propose adequate solutions
- Developing ETL pipelines and data flows in and out of the data warehouse using a combination of Azure Data Factory and Snowflake toolsets
- Developing idempotent ETL process design so that interrupted, incomplete, or failed processes can be rerun without errors using ADF dataflows and Pipelines
- Ability to work in Snowflake Virtual Warehouses as needed in Snowflake and automate data pipelines using Snowpipe for tedious ETL problems
- Capturing changes in data dimensions and maintaining versions of them using Stream sets in snowflake and scheduling them using Tasks
- Optimize every step of the data movement not only limited to source and during travel but also when it's at rest in the database for accelerated responses
- Must have the ability to build a highly efficient orchestrator that can schedule jobs, execute workflows, perform Data quality checks, and coordinate dependencies among tasks
- Responsible for testing of ETL system code, data design, and pipelines and data flows. Root cause analysis on all processes and resolving production issues are also a part of the process and routine tests on databases and data flow and pipeline testing
- Responsible for documenting the implementations, and test cases as well as responsible for building deployment documents needed for CI/CD
- 5+ years of Data engineering experience with a focus on Data Warehousing
- 2+ years of experience creating pipelines in Azure Data Factory (ADF)
- 3+ years of experience creating stored procedures with Oracle PL/SQL, SQL Server T-SQL, or Snowflake SQL
- 5+ years of Data engineering experience with a focus on Data Warehousing
- 2+ years of experience creating pipelines in Azure Data Factory (ADF)
- 5+ years developing ETL using Informatica PowerCenter, SSIS, Azure Data Factory, or similar tools.
- 5+ years of experience with Relational Databases, such as Oracle, Snowflake, SQL Server, etc.
- 3+ years of experience creating stored procedures with Oracle PL/SQL, SQL Server T-SQL, or Snowflake SQL
- 2+ years of experience with GitHub, SVN, or similar source control systems
- 2+ years of experience processing structured and un-structured data.
- Experience with HL7 and FHIR standards, and processing files in these formats.
- 3+ years analyzing project requirements and developing detailed specifications for ETL requirements.
- Excellent problem-solving and analytical skills, with the ability to troubleshoot and optimize data pipelines.
- Ability to adapt to evolving technologies and changing business requirements.
- Bachelors or Advanced Degree in a related field such as Information Technology/Computer Science, Mathematics/Statistics, Analytics, Business
- 2+ years of batch or PowerShell scripting
- 2+ years of experience with Python scripting.
- 3+ years of data modeling experience in a data warehouse environment
- Experience or familiarity with Informatica Intelligent Cloud Services (specifically Data Integration)
- Experience designing and building APIs in Snowflake and ADF (e.g. REST, RPC)
- Experience with State Medicaid / Medicare / Healthcare applications
- Azure certifications related to data engineering or data analytics.
This is a high PRIORITY requisition. This is a PROACTIVE requisition Background Check : No
Drug Screen : No
Vacancy posted 16 hours ago
Similar jobs that could be interesting for youBased on the Data Engineer (ADF, Orchestration, Data Flows, Snowflake) | REMOTE in United States vacancy
- ...Sr. Data Analyst 100% Telecommute Summary: As a member of the Optum Data Management team, the Data Engineer supports the CT EDW by developing and maintaining workflows, identifying, and resolving data quality issues, and optimizing processes to improve performance...Remote work
Software Technology Inc
United States3 days ago - ...Position: Data Engineer (ADF & Snowflake) Location: Remote Duration: Long term Required Skills & Qualifications Strong hands-on experience with Azure Data Factory (ADF). Strong experience working with Snowflake (data modeling, performance...Remote workLocal area
E-Solutions
San Jose, CA2 days ago - ...Onsite/ Hybrid/ Remote: Hybrid (4 days onsite... ...Have: Snowflake Snowpark... ...Advanced SQL Azure Data Factory ETL /... ...Analyze existing ADF pipeline logic and... ...assignments from engineering leadership Ensure... ...jobs, Tasks, and orchestration workflows inside Snowflake...Remote workH1b
aKube, Inc.
Burbank, CA16 hours ago - ...Job Title: HVR Data Engineer Location: Remote but must be located in one of these states... ...We are seeking a skilled ADF Data Engineer to join our... ...will ensure seamless data flow, enhance data quality, and... ...Azure Cloud Experience # Snowflake # Healthcare experience (...Remote workLocal areaFlexible hours
My3Tech Inc
Oak Brook, IL16 hours ago - ...Developers, and 7 Data Analyst. Project... ...for creating the ADF pipeline to move data... ...SQL Server to Snowflake and from Snowflake... ...and consistent data flow across platforms.... ...cloud-based data engineering and data warehousing... ...SnowSQL, with orchestration via ADF. Proficient...Remote work
Samprasoft
United States3 days ago - ...will be moving all ETL to Azure Data Factory/Snowflake as a database Team Size & Breakdown: ~100% Remote, just need to be in the US ~... ...Tools): • 5 + years of data engineering experience • Experience creating pipelines in ADF • Understanding of other ETL...Remote workLocal area
My3Tech Inc
Eden Prairie, MN16 hours ago $50 - $70 per hour
...Job Title: Snowflake Data Engineer Location: Stamford, CT ( Remote ) Position : ( C2C Role ) Duration : ( 6+... ...and integrity. Document data flows, transformations, and technical... ...SQL/PLSQL Job scheduling and orchestration using industry standard tools...Remote workWork experience placementDiverse Lynx
Stamford, CT16 hours ago- ...Location: Remote (Occasional Travel Required) Contract... ...develop, and optimize Snowflake data warehouses for... ...Collaborate with cloud engineers to integrate Snowflake... ...Informatica, Talend, Airflow, ADF, Glue). Experience... ...containerization and orchestration (Docker, Kubernetes)....Remote workFull timeContract work
Tek Ninjas
New York, NY3 days ago - ...Lead Snowflake Data Engineer Location: Rosemont, IL (Remote ok) Position overview: We are looking for a hands-on Lead Data Engineer to join our team! In... ...specific documentation like architecture diagrams, flow charts Analyze new data sources to understand quality...Remote work
United IT
United States2 days ago - We have 3 positions open for Data Engineers. The roles are 100% remote It's very important that along with expert level SQL, Snowflake, Experience with connecting large... ...Familiarity with Python and data orchestration tools (Airflow, ADF, dbt a plus). Collaborative...Remote work
ZEEKTEK
United States4 days ago - ...Role: Senior Data Engineer - Investments Data Platform (Snowflake) Location: New York, NY (Hybrid). No remote Note: Expertise on data engineering... ..., transactions, cash flows, exposure analytics, and... ...such as dbt and orchestration tools like Airflow ....Remote work
Diverse Lynx
New York, NY3 days ago - ...Applications - preferably on Snowflake, Oracle, or other SQL-based platform... ...using ETL tools like ADF, Talend etc. • Hands-on experience... ...- primarily Azure Data Factory; Azure Data Bricks, Azure... ...Additional Skills : Azure Engineer,Data EngineerThis is a high PRIORITY...Remote work
Kaav Inc.
United States16 hours ago - 6+ years of ADF data flows/Snowflake. if you just worked on ADF pipelines, this is not for you.... ...Data Factory. This role is key in data engineering and ETL (Extract, Transform, Load) processes... ...Factory Azure Synapse Analytics, Azure SQL This is a remote position....Remote work
CLOUDSCOUTS SOFTWARE SOLUTIONS LLC
Frisco, TX4 days ago - ...Cloud Data Engineer- Azure/Databricks/Snowflake Location - NYC, NY or Iselin, NJ 12+ months – hybrid 3 days in office / 2 days remote Key Requirements Must have pipeline development, platform... ...-on experience with Databricks, ADF, and Snowflake platform setup and...Remote workWork at office
Saxon Global
Iselin, NJ2 days ago - ...Sr. Data Engineer (Azure, Snowflake, Big Data) Position: Contract Summary of the project/initiative... ...contractor may start to do some work on the ADF side of things where they would do... ...period of time ~ Ability to work remotely and be focused on the job at hand...Remote workContract workFor contractorsWork experience placement
Samprasoft
United States3 days ago - A consulting firm is seeking a Senior Data Engineer with extensive experience in building and optimizing data pipelines using Snowpark and Snowflake. The role is 100% remote but requires candidates to reside in the Eastern or Central Time Zone. Successful candidates will...Remote work
The Brixton Group, Inc.
Florham Park, NJ3 days ago - A leading technology services provider is seeking a Data Ingestion Engineer for a remote contract role. The ideal candidate will have over 5 years of experience with Azure Data Factory (ADF), Databricks, and SQL. Responsibilities include converting VQL from Denodo into...Remote workContract work
Insight Global
Cincinnati, OH2 days ago $55 - $65 per hour
...tech staffing firm is seeking a Senior Data Engineer with a finance data focus to handle robust... ...to extract and load ERP data into Snowflake. Ideal candidates have 5+ years of experience... ...and finance teams. This is a fully remote, contract-to-hire position with a pay rate...Remote workContract workInteractive Resources LLC
New York, NY3 days ago- ...Role: Snowflake Data Engineer Location - San Jose, CA - Later it will be onsite- Initial remote (Please take consent letter) Duration: 6+ Months Job Description: ~ Strong Snowflake, Unix Shell Scripting, Python, Experience in any NoSQL...Remote work
Zortech Solutions
San Jose, CA3 days ago - ...leading technology firm is seeking a seasoned Data Migration Engineer to migrate enterprise data platforms from Snowflake to AWS. The successful candidate will design migration... ..., and data handling methodologies. This is a remote position open to USC and GC, offering a...Remote work
Snowrelic Inc
New York, NY3 days ago - ...Mutual of Omaha is looking for a highly skilled Data Engineer to develop and maintain data-centric solutions for group insurance... ...pipelines, especially with technologies like AWS and Snowflake. The role is remote, accessible to applicants in the United States or Puerto...Remote work
Mutual of Omaha
New York, NY3 days ago $100k - $130k
...Celigo is hiring a Senior Data Engineer to architect and maintain a Snowflake-based data ecosystem. The ideal candidate will design scalable ELT pipelines... ...significant Snowflake experience. The role promotes a remote-first culture with competitive benefits, including vacation...Remote workCeligo
New York, NY3 days ago- ...We are seeking an experienced Data Migration Engineer to support a Federal client in migrating enterprise data platforms from Snowflake to AWS Cloud services. The ideal candidate will... ...standards. Role : Data Engineer. Location : Remote. Note : Only USC and GC. Key...Remote workWork experience placement
Snowrelic Inc
New York, NY3 days ago - A technology firm is seeking a Data Engineer to design and maintain data pipelines within the healthcare sector in Chicago. The ideal candidate... ...ETL processes using modern tools such as Python and Snowflake SQL. This full-time role focuses on optimizing the integration...Remote workFull time
Medix Technology
Chicago, IL1 day ago - ...Technical Recruiter at STI Org. I am reaching out to you on an exciting job opportunity with one of our clients. Position: Snowflake Data Engineer Contract: W Requirements: Candidates must have experience with AWS implementation Data pipelines, snowpipes...Remote workContract work
Software Technology Inc
United States2 days ago - ...SpringbokIT is seeking a Senior Data Engineer in Dallas, Texas, to own and modernize their Snowflake data warehouse. In this senior, hands-on role, you will oversee architecture, build new data pipelines, and partner with the existing team to migrate SQL Server workloads...Remote work
SpringbokIT
Dallas, TX10 days ago - ...A staffing agency is seeking a Senior Data Engineer for a pivotal role in cloud data modernization. This fully remote position involves leading Snowflake and AWS initiatives, optimizing data platforms, and enhancing data processing capabilities. Ideal candidates will have...Remote work
Pro Search
New York, NY3 days ago - ...Data Engineer – Data Catalog (Alation) We are seeking an... ...contract. This is a fully remote position, offering an... ...connectivity and data flow. Develop custom solutions... ...Google Cloud, AWS, and Snowflake. ~ Understanding of... ...such as Mulesoft, ADF, Talend, SSIS. ~ Knowledge...Remote workHourly payContract work
Georgia IT Inc
United States2 days ago - ...A remote technology services company is seeking a Senior Data Engineer for a contract role focused on developing and optimizing data pipelines. The ideal candidate... ...experience in data engineering, proficiency with Snowflake, and strong SQL skills. Responsibilities...Remote workContract work
Knowledge Services
New York, NY3 days ago - A leading financial institution is seeking a Senior Data Engineer for a remote role. The successful candidate will design and maintain the bank... ...years in data engineering, strong skills in Informatica and Snowflake, and the ability to manage complex data initiatives. This...Remote workFull timeFlexible hours
WesBanco Bank Inc.
Columbus, OH16 hours ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer (ADF, Orchestration, Data Flows, Snowflake) | REMOTE. Be the first to apply!
Related searches
- junior data developer United States
- director data engineering United States
- junior big data engineer United States
- data engineer graduate United States
- senior data engineer United States
- data platform engineer United States
- sr information security engineer United States
- senior data integration developer United States
- data developer United States
- data engineer United States