Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior/Staff Big Data Storage and Computing Engineer, Recommendation Data Ecosystem

TikTok

Our team plays a crucial role in the data ecosystem of the TikTok Recommendation System, focusing on creating offline and real-time data storage solutions for large-scale recommendation, search, and advertising businesses, serving over 1 billion users. The core goals of the team are to ensure high system reliability, uninterrupted service, and smooth data processing. We are committed to building a storage and computing infrastructure that can adapt to various data sources and meet diverse storage requirements, ultimately providing efficient, cost-effective, and user-friendly data storage and management tools for the business.Responsibilities1. Architecture Design and Implementation: Design and implement offline and real-time data architectures for large-scale recommendation, search, and advertising systems based on Paimon and Flink. Ensure efficient data processing and storage to meet the strict requirements of the business for data timeliness and accuracy.2. System Construction and Optimization: Design and implement flexible, scalable, stable, and high-performance storage systems and computing models. Use Paimon as the storage foundation and combine it with the powerful computing capabilities of Flink. Continuously optimize system performance to cope with the challenges brought by business growth.3. Troubleshooting and Stability Assurance: Be responsible for troubleshooting production systems. For problems that occur in the Paimon-Flink architecture during operation, design and implement necessary mechanisms and tools, such as data consistency assurance and exception recovery, to ensure the overall stability of the production system.4. Distributed System Construction: Build industry-leading distributed systems, including offline and online storage based on Paimon and batch and stream processing frameworks based on Flink, providing solid and reliable infrastructure support for massive data and large-scale business systems.Minimum Qualifications:- A bachelor's degree or above in computer science, software engineering, or related fields, with experience in building scalable systems.- Thorough understanding of Paimon and Flink, and be able to understand and use them at the source-code level.- In-depth understanding of at least one data lake technology (such as Paimon), with practical implementation and customization experience, which should be highlighted in the resume.- Proficiency in programming languages such as Java, C++, and Scala, with strong coding and problem-solving abilities.- Experience in data warehouse modeling and be able to design efficient data models that meet complex business scenarios.Preferred Qualifications:- Experience in using other big-data systems/frameworks (such as Hive, HBase, Kudu, etc.) and handling large-scale data (PB - level and above) is preferred.- Have the courage to take on complex problems and be willing to explore problems without clear solutions.- Passions in learning new technologies and be able to quickly master and apply them to practical work.- Familiar with the principles of HDFS, and knowledge of columnar storage formats such as Parquet and ORC is preferred.Req ID: A187812

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Senior/Staff Big Data Storage and Computing Engineer, Recommendation Data Ecosystem in Seattle, WA vacancy
  • A leading social media platform is seeking a Senior/Staff Big Data Storage and Computing Engineer to join their Recommendation Data Ecosystem team in Seattle. This role is pivotal in enhancing the data ecosystem and contributing to innovative data-driven improvements.... 
    Senior
    Big data

    Tik Tok

    Seattle, WA
    6 days ago
  • $241.68k

     ...optimize the advanced recommendation systems that...  ...global e-commerce ecosystem.Balance user...  ...real-world user data modeling in statistical...  ...from a ML engineering viewpoint.Mentor...  ...equivalent degree in Computer Science, Engineering...  ...learning; andUtilizing Big data framework... 
    Senior
    Big data
    Full time
    Work experience placement

    TikTok

    Seattle, WA
    1 day ago
  • $155k - $190k

    Senior Software Engineer - Big Data/ML OpsTruveta is the world’s first health provider led data platform with...  ...job and are motivated by solving computing problems that can have a direct...  ...concepts across the Truveta and partner ecosystems.Bring your curiosity,... 
    Senior
    Big data
    For contractors
    Visa sponsorship
    Work visa
    Flexible hours

    Truveta

    Seattle, WA
    2 days ago
  • $142k - $220.5k

    Job DescriptionNordstrom Data Platform is a real-time, event-...  ...-time.We are looking for an engineer to help our team design and...  ...processing pipelines in the Big Data ecosystem in the Cloud, apply data definitions...  ...’s or Master's degree in Computer Science, Engineering, or... 
    Senior
    Big data
    Full time
    Work at office

    Nordstrom

    Seattle, WA
    1 day ago
  • $218.8k - $295.9k

     ...The AWS Analytics Engineering (AAE) is at the forefront...  ...from their data. We are a cross functional...  ...-oriented Senior Applied Science Manager...  ...system, develop big data engineering...  ..., and strategic recommendations to technical and non...  ...in statistics, computer science, engineering... 
    Senior
    Big data
    Flexible hours

    Amazon

    Seattle, WA
    2 days ago
  • Senior Machine Learning Engineer, Data InfrastructureUnity Vector builds an Data platform that powers insight,...  ...that supports both batch and stream big data processing using technologies such...  ...utilization across distributed compute systems used for data processingPartner... 
    Senior
    Big data
    Full time
    Work at office
    Worldwide
    Relocation package

    Unity Technologies

    Bellevue, WA
    1 day ago
  • $185k - $235k

     ...by advanced AI, recommendation systems, and adtech...  ...to dream big, innovate fast,...  ...events, contextual data, engagement history...  ...an experienced Senior Machine Learning Engineer to design and build...  ...'s degree in Computer Science, Machine...  ...of the big-data ecosystem (Hadoop, Spark,... 
    Senior
    Big data
    Full time
    Local area
    Work from home

    News Break

    Bellevue, WA
    1 day ago
  •  ...Summary The Apple Services Engineering team combines art and technology...  ...s degree in Machine Learning, Computer Science, Computer Engineering,...  ...computer science fundamentals like data structures, algorithms, and...  ...JAX). Distributed Systems & Big Data: Proven experience with... 
    Senior
    Big data

    Apple

    Seattle, WA
    1 day ago
  •  ...expenses, benefits, and computers. For the first time...  ...com addresses.About the Data Cloud TeamRippling Data...  ...first "context-aware, senior analyst" for HR and Finance...  ...- Owns the query engine (RQL, Rippling Query Language...  ...CDC, and KafkaLeverage big data technologies like... 
    Senior
    Big data
    Work at office
    3 days per week

    Rippling

    Seattle, WA
    5 days ago
  • $157.7k - $213.8k

     ...passionate about enabling data teams to solve the...  ...their business. Founded by engineers — and customer obsessed...  ...distributed data storage and processing systems...  ...standard framework for big data.Data Plane Storage...  ...for:BS (or higher) in Computer Science, related technical... 
    Senior
    Big data
    Local area
    Worldwide

    DataBricks

    Bellevue, WA
    7 hours ago
  •  ...create hyper-scale data-center networking...  ...passionate development engineer who combines deep...  ...intelligent ecosystem with predictive and...  ...autonomously generate, recommend, or even execute...  ...Fundamentals in Computer Science and Networking...  ...desirable:- Big Data Processing: Familiarity... 
    Big data

    TikTok

    Seattle, WA
    2 days ago
  • $61.31 - $104.39 per hour

     ...management to analyze clinical data, and to identify and...  ...leverage cloud computing, big data, data science, and...  ...less experiences Data Engineers and participate in...  ...years of experience as a Senior Data Engineer Preferred...  ...NoSQL (Spark, Hadoop ecosystem, Kafka, Flink, NoSQL... 
    Senior
    Big data
    Minimum wage
    Full time
    Local area
    Remote work
    Shift work

    Providence Health & Services

    Renton, WA
    5 days ago
  • $170k - $240k

     ...work on applications and data at scale, serving...  ...and employers and need senior data engineers to build fast, scalable...  ...a focus on Python and big data technologies ~...  ...technologies ~ Comprehensive computer science fundamentals...  ...Familiarity with data storage technologies such as... 
    Senior
    Big data
    Full time
    Remote work
    Flexible hours

    ZipRecruiter

    Seattle, WA
    9 days ago
  • $136k - $184.1k

     ...Now deliveries to big-screen TV’s, increasingly...  ...costs from the ecosystem. Our mission is to...  ...technology and engineering muscle as our...  ...technologies in big data, machine learning,...  ...communicating insights and recommendations to audiences of...  ...in influencing senior leadership through... 
    Senior
    Big data
    Worldwide
    Flexible hours

    Amazon

    Bellevue, WA
    7 days ago
  •  ...Databricks, we empower data teams to solve the...  ...focus on innovation.Our engineering teams push the boundaries...  ...developing a growing ecosystem of applications and...  ...impactful solutions.As a Senior Staff Software Engineer, you...  ...NoSQL, OLAP/OLTP) and big data processing (... 
    Senior
    Big data
    Worldwide

    DataBricks

    Seattle, WA
    7 hours ago
  • $175k - $215k

     ...We are seeking a Senior Machine Learning Engineer to play a key role to join...  ...deploying state-of-the-art computer vision and recommendation models that power our...  ...technical expertise in data and machine learning, innovative...  ...engineers, building an ecosystem of technologies at the... 
    Senior
    Temporary work
    Work experience placement
    Work at office
    Local area

    Metropolis Corp

    Seattle, WA
    4 days ago
  •  ...Responsibilities: Advanced Data Modeling: Build and...  ...opportunities for improvement, and recommend enhancements to optimize...  ...: Bachelor’s degree in Computer Science, Engineering, Information Systems, or a...  ...services for data solutions. Big Data Expertise: Extensive... 
    Senior
    Big data
    For contractors

    Samprasoft

    Seattle, WA
    5 days ago
  • $154.6k - $209.1k

    The AWS Analytics Engineering (AAE) organization...  ...build and operate the data platform that...  ...We are seeking a Senior Data Engineer to join...  ..., able to think big while paying careful...  ...'s degree in computer science, engineering...  ...pertains to data storage and computingAmazon... 
    Senior
    Big data
    Temporary work
    Flexible hours

    Amazon

    Seattle, WA
    2 days ago
  • $202.3k - $273.7k

     ...Product Managers, Technical Program Managers, Data Engineers, and Business Intelligence Engineers...  ...term view on architecting advanced data ecosystems and the ability to integrate data...  ...processes, leveraging AWS technologies and big data tools- Gather business and functional... 
    Senior
    Big data
    Worldwide
    Flexible hours

    Amazon

    Seattle, WA
    7 hours ago
  •  ...process here.We’re looking for a senior staff software engineer to lead the next generation of data infrastructure at Pinterest which powers mission critical big data and AI applications. You’ll...  ...petabytes-scale datasets, including compute engines, job management, resource... 
    Senior
    Big data
    Work at office
    Local area
    Remote work
    Relocation
    Relocation package

    Pinterest

    Seattle, WA
    1 day ago
  •  ...project, we are working with bleeding-edge big data technologies to develop a high-...  ...enthusiastic and technology-proficient Senior Data Engineer, who is eager to participate in design...  ...environment BS/BA or equivalent degree in computer science or similar (preferred) What we... 
    Senior
    Big data
    Flexible hours

    Grid Dynamics Holdings, Inc.

    Seattle, WA
    5 days ago
  •  ...easily accessible.The Data-E-commerce team...  ...-tier product and engineering teams to tackle...  ...Global E-commerce ecosystem has accumulated massive...  ...and personalized recommendation.Key Challenges1....  ...models, data, and compute to achieve scaling...  ...Familiarity with big data frameworks such... 
    Big data

    TikTok

    Seattle, WA
    2 days ago
  • $147.3k - $193.3k

     .... About this team The Data & AI Security team is...  ...across a complex, global ecosystem. Our work is highly...  ...with Data & Analytics, Engineering, Legal, Privacy, and GRC...  ...friction. The Senior Cybersecurity Engineer...  ...Bachelor’s degree in Computer Science, Cybersecurity... 
    Senior
    Permanent employment
    Full time
    Part time
    Work at office
    Work visa

    Lululemon Athletica

    Seattle, WA
    1 day ago
  • $152.4k - $268.2k

    Manage data collection and streamlining processes, creating frameworks...  ...planning for partner Data Engineering team, including gathering...  ...to back up assumptions, make recommendations and drive actions. Career examples...  ...data formats and tools for big data systems (e.g. Parquet,... 
    Senior
    Big data
    Relocation

    Apple Inc.

    Seattle, WA
    5 days ago
  •  ...forefront of developing the search recommendation algorithm for TikTok's...  ..., we're shaping a pioneering engine within the industry. Our objective...  ...or advanced degree in computer science or a related technical...  ...coding skills. Solid knowledge of data structure and algorithms.-... 
    Senior
    Work experience placement
    Worldwide

    TikTok

    Seattle, WA
    7 days ago
  • $175k - $200k

    Senior Machine Learning Engineer Truveta is the world’s first health provider led data platform with a vision of Saving Lives with...  ..., software and big data experience who...  ...across the healthcare ecosystem.If you’re seeking a...  ...GCP). B.S. or M.S. in Computer Science, Artificial... 
    Senior
    Big data
    For contractors
    Visa sponsorship
    Work visa
    Flexible hours

    Truveta

    Seattle, WA
    6 days ago
  • $161.9k - $219k

     ...using our technology and engineering muscle as our biggest...  ...top technologies in big data and optimization techniques...  ...and Marketing Ops to recommend and implement products...  ...into the broader ecosystem- Define user stories and...  ...Bachelor's degree in Computer Science, Engineering,... 
    Big data
    Remote work
    Worldwide
    Flexible hours

    Amazon

    Bellevue, WA
    7 hours ago
  • $108k - $216k

     ...massive scale. As a Data Scientist, you’ll turn...  ...closely with product, engineering, and business teams...  ...and communicate clear recommendations that drive measurable...  ...learning algorithms and computational algorithms based on...  ...goals Utilize big data analytics and advanced... 
    Senior
    Big data
    Full time
    Temporary work
    Part time
    Work at office
    Flexible hours

    Walmart

    Bellevue, WA
    5 days ago
  • $153k - $204k

     ...accelerate breakthroughs and turn compute into capability. Founded in...  .... About the role: As a Senior Data Engineer you will own and evolve the...  ...(e.g., Apache Airflow) and big data technologies (e.g.,...  ...data services (e.g., object storage, managed databases, analytics... 
    Senior
    Big data
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    Bellevue, WA
    15 days ago
  •  ...Beyond is seeking a hands-on AI / Data Engineer & Analyst to serve as our AI...  ...'s, Master's, or Ph.D. in Computer Science, Engineering, or a...  ...(Type 1/2). Experience with big data processing frameworks (Apache...  ...Spark, Flink) and cloud data ecosystems (AWS, Azure, GCP). Hands-on... 
    Big data

    Concepts Beyond

    Seattle, WA
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior/Staff Big Data Storage and Computing Engineer, Recommendation Data Ecosystem. Be the first to apply!