Machine Learning Research Engineer

Machine Learning Research EngineerCupertino, CAEtched is building AI chips that are hard-coded for individual model architectures. Our first product (Sohu) only supports transformers, but has an order of magnitude more throughput and lower latency than a B200. With Etched ASICs, you can build products that would be impossible with GPUs, like real-time video generation models and extremely deep & parallel chain-of-thought reasoning agents. Etched Labs is the organization within Etched whose mission is to democratize generative AI, pushing the boundaries of what will be possible in a post-Sohu world.Key ResponsibilitiesPropose and conduct novel research to achieve results on Sohu that are unviable on GPUsTranslate core mathematical operations from the most popular Transformer-based models into maximally performant instruction sequences for SohuDevelop deep architectural knowledge informing best-in-the-world software performance on Sohu HW, collaborating with HW architects and designers.Co-design and finetune emerging model architectures for highest efficiency on SohuGuide and contribute to the Sohu software stack, performance characterization tools, and runtime abstractions by implementing frontier models using Python and Rust.Representative ProjectsPropose and implement a novel test time compute algorithm that leverages Sohu's unique capabilities to unlock a product could never be achieved on a typical GPUImplement diffusion models on Sohu to achieve GPU-impossible latencies that allow for real-time image generationOptimize model instructions and scheduling algorithms to optimize for utilization, latency, throughput, and/or a mix of these metrics.Implement model-specific inference-time acceleration techniques such as speculative decoding, tree search, KV cache sharing, priority scheduling, etc by interacting with the rest of the inference serving stack.You May Be A Good Fit If You HaveAn ML Research background with interests in HW co-designExperience with Python, Pytorch, and / or JAXFamiliarity with transformer model architectures and/or inference serving stacks (vLLM, SGLang, etc.) and/or experience working in distributed inference/training environmentsExperience working cross-functionally in diverse software and hardware organizationsStrong Candidates May Also HaveML Systems Research and HW Co-design backgroundsPublished inference-time compute research and/or efficient ML researchExperience with RustFamiliarity with GPU kernels, the CUDA compilation stack and related tools, or other hardware acceleratorsBenefitsFull medical, dental, and vision packages, with 100% of premium coveredHousing subsidy of $2,000/month for those living within walking distance of the officeDaily lunch and dinner in our officeRelocation support for those moving to CupertinoEtched believes in the Bitter Lesson. We think most of the progress in the AI field has come from using more FLOPs to train and run models, and the best way to get more FLOPs is to build model-specific hardware. Larger and larger training runs encourage companies to consolidate around fewer model architectures, which creates a market for single-model ASICs.We are a fully in-person team in Cupertino, and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both as needed.