Senior Data Engineer

Role Description

As a Sr. Data Engineer at Oportun, you will be a key member of our team, responsible for designing, developing, and maintaining sophisticated software/data platforms in achieving the charter of the engineering group. Your mastery of a technical domain enables you to take up business problems and solve them with a technical solution. With your depth of expertise and leadership abilities, you will actively contribute to architectural decisions, mentor junior engineers, and collaborate closely with cross-functional teams to deliver high-quality, scalable software solutions that advance our impact in the market. This is a role where you will have the opportunity to take up responsibility in leading the technology effort – from technical requirements gathering to final successful delivery of the product - for large initiatives (cross-functional and multi-month-long projects).

What You'll Do

• Data Architecture and Design:
  • Lead the design and implementation of scalable, efficient, and robust data architectures to meet business needs and analytical requirements.
  • Collaborate with stakeholders to understand data requirements, build subject matter expertise, and define optimal data models and structures.
• Data Pipeline Development and Optimization:
  • Design and develop data pipelines, ETL processes, and data integration solutions for ingesting, processing, and transforming large volumes of structured and unstructured data.
  • Optimize data pipelines for performance, reliability, and scalability.
• Database Management and Optimization:
  • Oversee the management and maintenance of databases, data warehouses, and data lakes to ensure high performance, data integrity, and security.
  • Implement and manage ETL processes for efficient data loading and retrieval.
• Data Quality and Governance:
  • Establish and enforce data quality standards, validation rules, and data governance practices to ensure data accuracy, consistency, and compliance with regulations.
  • Drive initiatives to improve data quality and documentation of data assets.
• Mentorship and Leadership:
  • Provide technical leadership and mentorship to junior team members, assisting in their skill development and growth.
  • Lead and participate in code reviews, ensuring best practices and high-quality code.
• Collaboration and Stakeholder Management:
  • Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand their data needs and deliver solutions that meet those needs.
  • Communicate effectively with non-technical stakeholders to translate technical concepts into actionable insights and business value.
• Performance Monitoring and Optimization:
  • Implement monitoring systems and practices to track data pipeline performance, identify bottlenecks, and optimize for improved efficiency and scalability.
  • Provide technical leadership and expertise, guiding the team in making sound architectural decisions and solving challenging technical problems.
  • Conduct code reviews and provide constructive feedback to ensure code quality, performance, and maintainability.
  • Mentor and coach junior engineers, fostering a culture of continuous learning, growth, and technical excellence within the team.
  • Take ownership of (customer) issues, including initial troubleshooting, identification of root cause and issue escalation or resolution, while maintaining the overall reliability and performance of our systems.
  • Independently drive and lead multiple features, contribute to large projects and lead smaller projects.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field.
• 5+ years of experience in data engineering, with a focus on data architecture, ETL, and database management.
• Proficiency in programming languages like Python/PySpark and Java or Scala.
• Expertise in big data technologies such as Hadoop, Spark, Kafka, etc.
• In-depth knowledge of SQL and experience with various database technologies (e.g., PostgreSQL, MariaDB, NoSQL databases).
• Experience and expertise in building complex end-to-end data pipelines.
• Experience with orchestration and designing job schedules using CICD tools like Jenkins, Airflow, or Databricks.
• Ability to work in an Agile environment (Scrum, Lean, Kanban, etc).
• Ability to mentor junior team members.
• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services (e.g., AWS Redshift, S3, Azure SQL Data Warehouse).
• Strong leadership, problem-solving, and decision-making skills.
• Excellent communication and collaboration abilities.
• Familiarity or certification in Databricks is a plus.

Company Description