Live data from Hacker News

In-browser LLM inference engine with WebGPU and OpenAI API

Post reply on HN

Print view View original

1–5 of 5 posts

In-browser LLM inference engine with WebGPU and OpenAI API

#1

Post by CharlieRuan » Thu, Jun 13, 2024, 7:27 PM UTC

In-browser LLM inference engine with WebGPU and OpenAI API
blog.mlc.ai

Re: In-browser LLM inference engine with WebGPU and OpenAI API

#2

Post by crowwork » Thu, Jun 13, 2024, 7:30 PM UTC

Comes with ability to do full structured generation with json schema

also a in-browser demo https://chat.webllm.ai/

Re: In-browser LLM inference engine with WebGPU and OpenAI API

#3

Post by ruihangl » Thu, Jun 13, 2024, 7:44 PM UTC

It's good to see that it contains the standard OpenAI API interface in JavaScript which looks very convenient.

Re: In-browser LLM inference engine with WebGPU and OpenAI API

#4

Post by cyx6 » Thu, Jun 13, 2024, 10:25 PM UTC

AMAZING!

Re: In-browser LLM inference engine with WebGPU and OpenAI API

#5

Post by leshenj15 » Thu, Jun 13, 2024, 10:30 PM UTC

So cool. No more privacy concerns!

Post reply on HN