Lead Data Engineer
Openkyber
Job Title: Sr.Data Engineer AWS & Streaming
Location: Fort Mill SC or New York, NY (2-3 Days Hybrid)
Experience level 10- 15 Years
Role Summary: We are seeking a Mid Senior Data Engineer with strong expertise in AWS-based data engineering, real-time streaming technologies, and enterprise-grade data quality frameworks. The ideal candidate will design, build, and optimize scalable batch and streaming data pipelines, implement robust data validation and monitoring processes, and support mission-critical analytics platforms.
Key Responsibilities:
- Develop and maintain scalable ETL/ELT pipelines using AWS Glue, PySpark, and Python
- Build event-driven workflows using AWS Lambda
- Design and manage real-time streaming solutions using Kafka, KSQL, and Apache Flink
- Implement and enforce comprehensive data quality frameworks, including validation, profiling, monitoring, and reconciliation
- Optimize data processing performance, scalability, reliability, and cost in cloud environments
- Collaborate with cross-functional teams to deliver reliable, production-grade data platforms and ensure data integrity across the pipeline
Required Skills: Strong hands-on experience with Python and PySpark Proven expertise in AWS Glue, Lambda, and other cloud-native data services Solid experience with the Kafka ecosystem (topics, partitions, consumer groups, streaming patterns) Demonstrated experience building and supporting data quality frameworks (validation rules, reconciliation checks, profiling, anomaly detection) Strong understanding of distributed data processing and scalable architecture patterns
Good-to-Have Skills Experience with Apache Flink for real-time stream processing and stateful computations Knowledge of KSQL or other streaming SQL engines Exposure to CI/CD pipelines, IaC (Terraform/CloudFormation), and DevOps practices Familiarity with data lake/lakehouse architectures and table formats such as Iceberg, Delta, or Hudi Experience working in enterprise or financial data environments
For applications and inquiries, contact:View email address on us.fitly.work
- ...Job Role: Databricks Data Engineer / Data Platform Consultant Location: Remote Duration: Long Term Project No C2C only W2 Mandatory Requirement: Databricks Partner Certification (Required Client will not consider resumes without this certification...SuggestedRemote work
Openkyber
Montana, MT1 day ago $77.58 - $82.58 per hour
...Job Title: Senior Data Engineer / Data Architect Location: Plano ,TX Duration: Contract - 8 months Pay Range: $77.58 - $82.58 About OpenKyber OpenKyber is a leading global IT consulting and workforce solutions firm providing services and...SuggestedContract workOpenkyber
Montana, MT1 day ago$74.1k - $148.3k
...Job Description Team Overview: The Data Center Construction organization at Oracle... ...of OCI’s data centers. As a Principal Engineer, you will serve as a technical authority... ...resolving execution issues quickly. • Lead the change order process, validating scope...SuggestedTemporary workLive inLocal areaWorldwideRelocationRelocation packageFlexible hoursOracle
Helena, MT1 day ago$165.56k - $188.09k
...Sr. Data Analytics Engineer Sr. Data Analytics Engineer Values & Innovation At Under Armour, we are committed to empowering those who... ...visualization and reporting requirements. Leverage industry-leading integration and analytics tools to build a modern data...SuggestedLocal areaRemote workRelocationUnder Armour
Helena, MT2 days ago$64.89k - $173.04k
...of something bigger - helping to simplify health care one person, one family and one community at a time. Position Summary The Data Engineer will be responsible for delivering high quality modern data solutions through collaboration with our engineering, analysts, data...SuggestedHourly payFull timeTemporary workLocal area$15k
Summary This Senior Data Engineer will be a member of the Agile team responsible for D.A. Davidson’s flagship Client Access web and mobile applications. As a member of the team, responsibilities will include troubleshooting, expanding and optimizing the existing data integrations...Remote jobWork experience placementLocal area$91.4k - $187k
...Technician - Mechanical, you will provide engineering ownership for the reliability, efficiency... ...and HVAC systems supporting hyperscale data centers. This role sets technical direction... ...operability and maintainability. Lead or support vendor technical reviews, commissioning...Temporary workFlexible hoursOracle
Helena, MT4 days ago- ...Company Description We are Olsson. We engineer and design solutions that improve the world... ...'s most innovative companies. Our U.S. data center projects rank among the largest and... ...Project Manager to join our industry-leading Data Center Project Management team. In this...Full timeContract workFor contractorsFlexible hours
Olsson
Helena, MT3 days ago $146.3k - $306.4k
...leadership role that requires strong technical depth, sound engineering judgment, and the ability to lead teams through growth, complexity, and fast-paced... ...develop a team of electrical engineers supporting OCI’s data center design initiatives. Provide technical and...Temporary workFlexible hoursOracle
Helena, MT1 day ago$86.4k
...: Highmark Health Job Description : JOB SUMMARY This job promotes the understanding, integration, and use of healthcare data and information technology in support of scholarly research and addressing healthcare business challenges. Leveraging experience in...For contractorsWork at officeLocal areaHighmark Health
Helena, MT1 day ago$35.99 - $56.68 per hour
...intermediate informatics principles, to support and periodically lead moderately complex projects with limited supervision, while continuing... ...to ensure fairness. We protect your privacy and follow strict data protection rules. Your information is safe and used only for...Hourly payRemote workFlexible hoursIntermountain Health
Helena, MT3 days ago- ...approach to where we work. About the Role: As a Machine Learning Engineer, you will have the opportunity to collaborate closely with... ...problems executing on a high throughput system with dynamic data. About the Job: Design, develop, and deploy machine learning...Permanent employmentFull timeWork at officeRemote workWork from homeFlexible hours
Instacart
Helena, MT2 days ago $107.66k - $161.7k
...working on our Poe product. About the Team and Role: Our small engineering team works on challenging problems every day. We have a culture... ...end ownership of Machine Learning systems -- from prototyping, data pipelines and training, to realtime LLM application at scale...Remote jobFull timeWork experience placementInternshipQuora
Helena, MT2 days ago$102.3k - $209.5k
...supporting our Mechanical Facilities, you will lead the safe, reliable, and efficient... ...critical mechanical systems supporting OCI data centers. This role owns mechanical maintenance... ...action closure. Partner with engineering, commissioning, and deployment teams on retrofits...Temporary workFor contractorsFlexible hoursOracle
Helena, MT4 days ago- ...and fulfillment. We use machine learning and Internet-scale data to elevate customer experience, improve efficiency, and reduce... ...and Discovery ML team at Instacart works alongside world-class engineers, data scientists, and product managers to shape the future of search...Remote jobPermanent employmentWork experience placementInternshipWork at officeWork from homeFlexible hours
Instacart
Helena, MT17 hours ago $60 per hour
...A leading AI development company seeks proficient programmers to contribute to cutting-edge AI systems. This fully remote position allows for a flexible schedule and hourly pay up to $60. Qualifications include fluency in English and proficiency in programming languages...Hourly payRemote workFlexible hoursDataAnnotation
Helena, MT4 days ago- ...Senior Life Sciences Knowledge Engineer Company: Norstella Location: Remote, United... ...Why Norstella? Norstella unites market-leading companies that all have a shared goal of... ...wider breadth of expertise, cutting-edge data solutions and expert advisory services alongside...Full timeTemporary workWork at officeLocal areaRemote workFlexible hoursShift work
Norstella
Helena, MT17 hours ago $40 per hour
A cybersecurity company is seeking experienced professionals to evaluate AI-generated security content and contribute to building reliable AI tools. This remote role offers flexibility to choose projects and work hours, with pay starting at $40+ per hour. Ideal candidates...Hourly payRemote work$15k
...latest annual report.We are seeking a Mid-Level AI and Automation Engineer to develop next-generation AI capabilities within our Microsoft-... ...Experience building or consuming RESTful APIs and working with data integration.• Cloud & AI Knowledge: Working knowledge of Azure...Work experience placementLocal areaD.A. Davidson
Helena, MT2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer. Be the first to apply!
