Data Engineer

We have a full-time job opportunity of Data Engineer in San Francisco, California (Hybrid).

Location: San Francisco, California (Hybrid, 3 times a week at client’s office in the Bay Area)

Responsibilities

• Design, develop, test, deploy, and maintain scalable Big Data solutions.
• Manage project priorities, deadlines, and deliverables across multiple initiatives.
• Build and optimize data pipelines using Spark and other distributed processing frameworks.
• Develop software using modern programming languages such as Java or Scala.
• Work with both relational (e.g., MySQL, Postgres) and NoSQL databases.
• Integrate and manage data workflows using tools like Hive, Impala, OOZIE, Airflow, NIFI, and Kafka.
• Write shell scripts and manage deployments in Linux/Unix environments.
• Collaborate with cross-functional teams in agile ceremonies including daily stand-ups, story pointing, and retrospectives.
• Mentor junior engineers and foster a culture of continuous improvement and technical excellence.

Skills/Experience

• Extensive experience as a Big Data engineer, with expertise in cloud or on-premise data platforms.
• Proficient in at least one modern programming language (Java, Scala, or Python).
• Strong foundation in computer science principles including object-oriented design, data structures, algorithms, and complexity analysis.
• Deep experience with Spark and distributed data processing.
• Hands-on experience with both SQL and NoSQL databases.
• Familiarity with data orchestration and streaming tools (Kafka, Airflow, NIFI, etc.).
• Comfortable working in Linux/Unix environments with basic shell scripting skills.
• Proven ability to design and develop software systems at scale.
• Strong understanding of Agile methodologies, with the ability to drive iterative delivery and cross-team collaboration.
• Strong communicator with the ability to explain complex concepts to both technical and non-technical audiences, and to influence stakeholders across product, engineering, and acquisition teams.

Bachelor’s degree in Computer Science, Data Analytics, Mathematics, Software Engineering, or a related field or equivalent practical experience

Ready to architect a data platform built to last?

“Zbis is here… Wow!!! It’s better than what we expected. This is transformational to our company and how we drive our business forward. It’s customer‑centric, fact‑driven, and captures margins and market‑share across all dimensions. Zbis is truly a game changer!”
– Frederick Howard