Software Engineer III - Data Engineer

Responsibilities

• Design, develop, and support scalable Databricks feature engineering solutions for risk analytics, fraud detection, machine learning, and enterprise data products.
• Build and maintain reusable batch and real-time feature pipelines, including onboarding, versioning, testing, monitoring, and lifecycle management.
• Implement data engineering solutions using Databricks, Apache Spark, PySpark, Delta Lake, Lakeflow, and declarative pipeline patterns.
• Modernize the platform by migrating legacy Spark and EMR workloads to Databricks-native and cloud-native architectures.
• Apply software engineering practices including CI/CD, automated testing, code reviews, observability, release management, and production support.
• Use enterprise-approved AI-assisted engineering tools to accelerate development, improve code quality, automate SDLC activities, and identify innovation opportunities.
• Implement data quality, governance, lineage, metadata, retention, security, PCI, and audit controls.
• Develop and support AWS cloud-native solutions using services such as S3, Glue, Lambda, ECS/EKS, Aurora/RDS, and Terraform.
• Participate in architecture reviews and technical decisions concerning performance, stability, cost, resiliency, and scalability.
• Collaborate with data scientists, model developers, business stakeholders, and engineering teams; mentor junior engineers.

Requirements

• Formal training or certification in software engineering, computer science, data engineering, or a related discipline with 3+ years of applied industry experience.
• Hands-on experience with Databricks, Apache Spark, PySpark, Delta Lake, and modern Lakehouse architectures.
• Experience building and supporting large-scale batch and streaming data pipelines.
• Proficiency in Python and SQL with an understanding of distributed computing principles.
• Hands-on experience using enterprise-authorized AI-assisted software development tools and critically evaluating, validating, and refining generated outputs for correctness, performance, and security.
• Understanding of responsible AI use in engineering workflows, including data sensitivity, secure input and output handling, resiliency, and security expectations.
• Working knowledge of Databricks Unity Catalog, Delta Live Tables, and modern declarative pipeline frameworks.
• Experience with AWS services including S3, Glue, EMR, Lambda, ECS/EKS, and related technologies.
• Experience implementing data-quality, data-governance, and lineage solutions.
• Familiarity with data security controls, encryption, and PCI data handling requirements.
• Preferred certifications include Databricks Data Engineer Associate or Professional and AWS Solutions Architect or Developer certifications.
• Preferred experience includes Databricks Feature Engineering, Feature Store, Feature Views, real-time and event-driven data solutions, Kafka, streaming frameworks, distributed services architectures, platform modernization, Terraform, Snowflake, Databricks SQL, Unity Catalog, and Infrastructure-as-Code practices.

Benefits

• Competitive total rewards package with base salary determined by role, experience, skill set, and location.
• Eligible roles may receive commission-based pay or discretionary incentive compensation in cash or forfeitable equity.
• Comprehensive health care coverage, on-site health and wellness centers, retirement savings plan, backup childcare, tuition reimbursement, mental health support, and financial coaching are available based on eligibility.