Data Engineer
Compunnel
The Data Engineer will be responsible for collecting, parsing, managing, analyzing, and visualizing large datasets to transform information into actionable insights. The role involves building scalable, repeatable, and secure data pipelines across various platforms. The engineer will ensure that data workflows support multiple users while maintaining high performance, security, and reliability.
Key Responsibilities
- Design, develop, and maintain robust and efficient data pipelines to ingest, transform, catalog, and deliver curated, trusted, and quality data into the Common Data Platform.
- Actively participate in Agile ceremonies and follow Scaled Agile processes defined by the CDP Program team.
- Deliver high-quality data products and services following SAFe Agile practices.
- Identify and resolve issues related to data pipelines and analytical data stores.
- Implement monitoring and alerting for pipelines and data stores, enabling auto-remediation to ensure uptime and reliability.
- Apply a security-first approach with strong testing and automation practices.
- Collaborate with product managers, data scientists, analysts, and business stakeholders to understand data needs and provide tools and infrastructure.
- Stay updated with modern data engineering trends and evaluate new tools, frameworks, and technologies to improve efficiency.
Required Skills
- Bachelor’s degree in computer science, Information Systems, or related field, or equivalent experience.
- 2+ years’ experience with Databricks, Collibra, and Starburst.
- 3+ years’ experience with Python and PySpark.
- Experience using Jupyter notebooks for coding and unit testing.
- Strong recent experience with relational and NoSQL data stores, STAR and Dimensional Modeling methods.
- 2+ years’ experience with a modern data stack (S3, Spark, Airflow, Lakehouse architecture, real‑time databases).
- Experience with cloud data warehouses such as RedShift or Snowflake.
- Overall data engineering experience across traditional ETL and Big Data (on‑prem or cloud).
- Experience building end‑to‑end pipelines for unstructured and semi‑structured data using Spark.
- Ability to work with confidential supervisory information (Must meet Protected Individual requirements).
Education
Bachelor’s degree in computer science, Information Systems, or related field (or equivalent experience).
#J-18808-LjbffrVacancy posted 20 hours ago
Similar jobs that could be interesting for youBased on the Data Engineer in San Francisco, CA vacancy
- We are seeking a hands-on Lead Data Engineer to join the Data & AI Supply Chain organization and drive the design, development, and delivery of enterprise-scale data products on Google Cloud Platform (GCP). This role will support Supply Chain initiatives across Sourcing...SuggestedHourly payFreelance
- We are looking for a Contract Data Engineer to support financial systems and data operations in San Francisco, California. This is a remote on-going contract. This role will work across Finance, Systems, and Engineering to improve data reliability, strengthen integrations...SuggestedContract workRemote work
$100.79k - $160.26k
...DLA Piper is a place you can engage in meaningful work and grow your career. Let’s see what we can achieve. Together.SummaryThe Data Engineer, Solutions & Data role designs, builds, and operates data pipelines and data integration processes that translate raw data into...SuggestedFull timeWork at officeRemote workVisa sponsorship$235k - $385k
About the teamThe Applied team works across research, engineering, product, and design to bring OpenAI’s technology to consumers and businesses... ...to us than unfettered growth.About the role:We're seeking a Data Engineer to take the lead in building our data pipelines and...SuggestedWork at officeLocal areaRelocation packageFlexible hours$170k - $200k
About the RoleWe're looking for a Data Engineer who wants to operate like a founder. Not someone who wants to spend their time maintaining dashboards, moving tickets across a board, or optimizing pipelines in a mature environment. Someone who sees infrastructure as a product...SuggestedWork at officeRemote workFlexible hoursShift work- ...strong team culture.Preferred locations for team members for our Data Platform roles are our San Francisco or Seattle hubs.About... ...superhuman.com and about our values here.The OpportunityAs a Data Engineer on the RTM Growth team, you’ll own the pipelines, models, and datasets...WorldwideHome officeFlexible hours
$153k - $207k
...years. But that only happens when clean, structured scientific data and AI are built into how science gets done.Benchling is the AI... ...modern software to modern science.Benchling is building AI & Data Engineering (AIDE), a small, autonomous team within our Security & IT...Work at officeLocal areaFlexible hours3 days per week$140k - $180k
Data Engineer, Data Platform About the RoleWe are building out our Data Platform team at Sigma, with a relentless focus on developing data models that fuel trusted insights across the company. As a Data Platform Engineer, you will be responsible for the underlying data...Full timeWork at office- THE GLOBAL LEADER IN DATA & ANALYTICS RECRUITMENTHarnham Search and Selection Company Number: 05723485Harnham Search and Selection is a registered company in England and Wales. Reg no. 05723485Harnham Europe Limited Company Number: 09956940Harnham GmbH HRB: 196954Harnham...Work at office
$76.73k - $127.88k
...Are you an experienced, passionate pioneer in technology who wants to work in a collaborative environment? As an experienced Data Engineer, you will have the ability to share new ideas and collaborate on projects as a consultant without the extensive demands of travel...Local areaVisa sponsorship$90.2k - $112.7k
...environmental, and humanitarian sectors. We are both a space company and data company all rolled into one.Customers and users across the... ...hardware design, manufacturing, data processing, and software engineering, our office is a truly inspiring mix of experts from a variety...Full timeTemporary workWork at officeLocal areaRemote workHome office3 days per week$120.8k - $151k
...dream. We make this a reality by empowering you with the tools, resources, and support you need to grow your career.Data at BrexOur Scientists and Engineers work together to make data — and insights derived from data — a core asset across Brex. But it's more than just...Work at officeRemote workWork from home- ...Lead Data Engineer with MarTech Location: SFO, CA (Hybrid 2 days a week) Key Responsibilities Lead end-to-end MarTech engineering initiatives across orchestration, data processing, and activation pipelines. Architect scalable, event-driven systems that...Remote work2 days per week
Staffing the Universe
San Francisco, CA20 hours ago $112.5k - $149.97k
...the needs of the organization and its stakeholders. Job Overview The Office of Information Technology is seeking a Lead Data Engineer to join our dynamic team at the State Bar of California. In this role, you will define and drive the modernization of our data infrastructure...Work at officeRemote workCalifornia Correctional Health Care Services
San Francisco, CA20 hours ago$170k - $260k
...customers, growing our community, or shaping our story, you’ll help define what comes next.About the Role:We're looking for a Senior Data Engineer to join our Data team and own the reliability of the data our GTM stakeholders depend on. This is a full-stack role: you'll...Work from homeWorldwideFlexible hours$160k - $260k
...teammates and grow your career as we build the future of global banking. If that sounds like you, let’s build what’s next.Senior Data Engineer Hiring Location: San FranciscoPart 1. Data ModelingDesign and implement robust and scalable data models that support business...Temporary workWork at officeLocal areaWorldwide$141.9k - $190.3k
Job Posting Title:Sr Data EngineerReq ID:10153288Job Description:Technology is at the heart of Disney’s past, present, and future.... ...Entertainment and ESPN Product & Technology is a global organization of engineers, product developers, designers, technologists, data scientists,...Full timeWork experience placement$190k - $220k
...fluency as the technology evolves. AI experience requirements vary by role and will be assessed during the interview process.The Data Engineering team builds tools and systems that make Gusto's data consistent, user-friendly, and helpful. This helps our teams make...Full timeWork at officeLocal area2 days per week3 days per week$170k - $220k
San Francisco or DenverEngineering /Full-Time /HybridWindfall is seeking a Sr. Data Engineer to join our data team. As a Sr. Data Engineer on our data team, you will be building out the core data asset that everything else at Windfall is built on top of. Communication and...Full time$172.5k - $260.1k
...the right place! Agentforce is the future of AI, and you are the future of Salesforce.About the TeamAt Slack, data isn't just infrastructure — it's the engine behind every great enterprise decision. Our Data Engineering team builds the products and pipelines that power...Permanent employmentFull time$147k - $181k
...our team as we help shape a brighter way forward. About JLL and Data PlatformJLL is a leading professional services firm that specializes... ...pipelines of their own.What this job involvesAs a Senior Data Engineer on the Data Consumption team, you will own the design and...Full timeContract workLocal areaRemote work$128k - $249k
...search for talented visionaries and your search for important and impactful work lead to the same place.We are seeking a Senior Data Engineer to join the firm. The Senior Data Engineer contributes to the Enterprise Applications, Data, and AI Platforms group, designing scalable...Full timeTemporary workLocal areaRemote workFlexible hoursAfternoon shift- ...Open role: Data Engineer (Full-time) Location: San Francisco, CA (On-site) As a Data Engineer, you'll design and manage the data pipelines and storage solutions that handle our platform's information. This role ensures that everything from transaction records to audit...Full time
Methodic
San Francisco, CA4 days ago - ...Data EngineerStuut is transforming accounts receivable for B2B companies—making collections smarter and faster for companies that... ...s intelligence layer. You'll work closely with our product and engineering teams to transform raw financial data into actionable insights...Full time
Stuut
San Francisco, CA2 days ago - ...Data EngineerPrimary: Bay Area (San Francisco/Peninsula)About UsBeast Industries is a multifaceted media and entertainment company... ...legacy that inspires future generations.We're doing an AI-first engineering rebuild for a company that already has an audience of 100M+ people...Work at office3 days per week
MrBeast
San Francisco, CA3 days ago - ...Need help in your climate job search? Dive deep into climate with Terra.do’s 12-week climate bootcamp course. As a Junior Data Engineer you will work directly with the the Director of Geospatial and own data engineering for the soul of the system. You will have the opportunity...
Blumen
San Francisco, CA3 days ago - ...Data Engineer Job Description Client Location: Onsite in San Francisco, CA Potential exposure to graphic and/or objectionable content to any extent TRAVEL - Potentially a few times a year REQUIRED: Bachelor's required 8+ years of experience in Data Engineering (will...Internship
Rose International
San Francisco, CA2 days ago $150k - $185k
...takes two minutes to set up. We're building the layer between powerful AI models and real people's lives. Position As Lindy\'s Data Engineer, you\'ll work closely with senior leadership (Lindy\'s CEO, GTM leads, and Head of Engineering) to uncover critical business insights...Work at officeRelocationVisa sponsorshipRelocation packageFree visaLindy
San Francisco, CA20 hours ago- ...Data EngineerLocation: San Francisco, CARequired Clearance: SecretSalary: CompetitiveJob DescriptionWe are looking for a skilled Data Engineer with a strong focus on AI and machine learning to join our dynamic team. The ideal candidate will play a critical role in designing...
Fullscope
San Francisco, CA20 hours ago $110k - $169k
...real work: We offer generous compensation packages that recognize hard work and excellence. Job Description The Pricing Data Engineer builds and maintains the data infrastructure and tools that enable consistent, data-driven pricing across the firm while modernizing...Full timeWork at officeMcDermott Will & Schulte
San Francisco, CA4 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Engineer. Be the first to apply!
Related searches
- data platform engineer San Francisco, CA
- aws data engineer San Francisco, CA
- data engineer intern San Francisco, CA
- data science developer San Francisco, CA
- sr information security engineer San Francisco, CA
- data engineer graduate San Francisco, CA
- senior data engineer San Francisco, CA
- data engineer machine learning San Francisco, CA
- principal data engineer San Francisco, CA
- junior data engineer remote San Francisco, CA