AWS/Databricks Cloud Data Engineer

About MoserFor more than 25 years we have formed partnerships and grown through open and honest collaboration with our clients, partners, and employees. We are best known for taking great care of our clients, our dedication to creating a work environment where employees do their best work, and our deep commitment to continuous improvement. Our consultants work in a collaborative and fast-paced environment, are self-motivated, and are passionate about evolving technology. It is no accident that we are recognized as one of the Best Places to Work in Indiana for 10 consecutive years.Internally, we believe in building strong teams from the top down with a focus on values in our Model-Coach-Care philosophy. Our leadership are encouraged and trained to model good practices, mentor other employees and each other, and show empathy and caring in all interactions. This is the base of our core values: Accountability, Balance, Collaboration, Focus, Integrity, Social Responsibility, Support and Transparency.Moser Consulting believes in equal opportunity for all people and is committed to enabling a diverse, equitable, and inclusive culture. We foster a spirit of unity that respects the remarkable individuality of everyone's culture, history, and service.DescriptionWe are seeking a Senior Cloud Data Engineer to join our consulting team. In this role, you will design, build, and maintain scalable data pipelines and acquisition workflows across cloud and on-premises environments, contributing to the full spectrum of data engineering – from ingestion through transformation and delivery. You will collaborate with cross-functional teams to translate business requirements into robust data solutions, while upholding best practices in architecture, cloud infrastructure, documentation, and continuous improvement. This position sits within our genomics organization, providing senior-level, strategic technical leadership alongside two existing engineers to build out AWS-based data pipelines that support genomic data science models — hands-on, production AWS pipeline-building experience is the top priority for this role.Role ResponsibilitiesParticipate in team stand-ups and other agile ceremonies to support project deliveryAssist in building and maintaining data acquisition pipelines from diverse sources including databases, flat-files, and REST APIsLeverage development design patterns and best practices across workloads.Support data modeling efforts including dimensional models and semantic layers for analytics under senior guidanceProactively troubleshoot and resolve pipeline failures, query performance issues, data quality concerns, notebook timeouts, data model inconsistencies, semantic model refresh errors, and capacity throttling using monitoring tools and query optimization techniquesParticipate in requirement gathering meetings and translate business needs into technical specificationsEngage in ongoing learning through online trainings, certifications, and mentorship opportunitiesCreate and maintain technical documentation, operational documentation, data flow diagrams, and data mapping documentsCollaborate effectively with cross-functional teams and communicate progress to technical and non-technical stakeholdersBuild and maintain AWS-native pipelines (Lambda, ECS, EKS, S3) that ingest and store genomic data retrieved via API callsGenerate and manage sequencing and post-sequencing data processing workflows in support of genomics data science modelsRequirementsTop priority: proven, hands-on experience designing and building AWS-based data pipelines end-to-end (API ingestion through storage and delivery)Must-have: hands-on experience working with High Performance Computing (HPC) clustersExperience with dimensional modeling, SCDs, and medallion architecturePrevious cloud experience required; AWS preferred with hands-on exposure to Lambda, S3, and DockerExperience implementing and managing role-based access control (RBAC), including group/permissions management for secure system accessStrong Python experience across pipeline development, ETL, and data engineering tasks, with exposure to complex datasets from disparate batch and streaming sources and an emphasis on low latency, high throughput, and cost-efficiency; pandas experience a plusLeveraging SQL and python data tooling develop scalable, optimized ETL/ELT pipelines to ingest, transform, and load large, complex datasets from disparate sources (batch and streaming). Streamlining for low latency, high throughput and cost-efficiency.Experience with API development in both consumer and producer capacities (REST APIs)CI/CD experience with modern DevOps tooling and deployment pipelinesExperience with API-based access patterns for retrieving and integrating data from external systemsLambda functions must be developed and deployed via code/infrastructure-as-code (e.g., CloudFormation, Terraform, CDK, SAM) — GUI/console-based Lambda creation is not acceptableHands-on experience with Amazon ECS and/or EKS for container orchestrationDemonstrated ability to learn new technologies quickly and apply them to solve problemsStrong attention to detail and commitment to producing high-quality workExcellent oral and written communication skillsProfessionalism and ethics including trust, positive attitude, commitment, honesty, collaboration, and approachabilityProven ability to adapt in a fast-paced consulting environment, including managing time effectively and handling multiple tasks simultaneouslyPreferred Skills Experience with version control systems (Git) and agile development frameworksDesign and document end-to-end testing plans for Python-based applicationsPySpark experience for large-scale distributed data processingExperience with Datadog or similar observability and monitoring platformsFamiliarity with Dagster or other modern data orchestration frameworksTerraform or infrastructure-as-code (IaC) experience for cloud resource provisioningDatabase experience with databases such as AWS RDS, MySQL, or PostgreSQLFamiliarity with D3.js for building custom data visualizationsWhere You'll WorkMoser Consulting has 2 local, Indianapolis, IN offices. You will be able to work a hybrid/onsite schedule, flexing your time between either office.SalaryAt Moser Consulting, we believe in pay transparency and fairness. The $85k-$125k salary range for this role is not just a number—it’s a reflection of the value we place on the skills and experience our employees bring to our team. We are committed to offering a competitive salary that aligns with the industry standards and the unique competencies you bring to our community.BenefitsFor over a quarter of a century, Moser Consulting has been a beacon for top-tier IT talent who excel in self-management. Our people are our greatest asset. We don’t just hire the best—we welcome them into our family, connect them with opportunities, and empower them to create innovative solutions to technology challenges.Our unique culture is our competitive edge. It fosters happiness, health, and low stress, even in an industry known for its demands. This is why we’re consistently recognized as one of the Best Places to Work in Indiana. We provide our employees with an inspiring workspace, a fun and collaborative atmosphere, and a generous compensation package. But that’s not all.We also offer a suite of benefits designed to support and enrich our employees’ lives. These include:Training Opportunities: We believe in lifelong learning and provide numerous avenues for skill enhancement.Fully Invested 401K Plan: We help secure your future with a fully invested 401K plan.PPO and HDHP Medical Plans: Choose the health insurance program that best fits your needs.Employer-Paid Dental and Vision Plans: We cover dental and vision plans, ensuring our employees have access to comprehensive health care.Onsite Fitness Center: Stay fit and healthy with our state-of-the-art fitness center.Wellness Program: We promote a healthy lifestyle with our wellness program.Catered Lunches: Enjoy delicious catered lunches regularly.At Moser Consulting, we don’t just offer jobs—we offer careers, growth, and a chance to join a thriving community. Come, be a part of our family.Moser Consulting is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a professional.Full-Time