Funny, i did this myself some months ago - its actually not that tricky to run a llm in your browser. Tho, the problem is that the models fitting in there are well relativly "tiny" in my opinion.
The Web Platform is great for agents, especially given the browser already is the user's agent itself. Give Prompt API (or other providers, Transformers.js, LiteRT-LM) a try. You can have agents communicate over WebRTC as well: https://xt-ml.github.io/shadow-claw/
HashAgent started with a simple question: can you run a useful AI agent with zero hosting costs? The answer I landed on: the user's own device is the server.
How it works:
- The entire agent definition (name, system prompt, greeting, generation config) is encoded as compressed base64url JSON in the URL hash fragment. The hash never hits a server — sharing a link is sharing the agent.
- Inference runs in-browser via WebGPU. Two runtimes: WebLLM (MLC) for Llama 3.2 / Phi / Mistral / Qwen, and Transformers.js for newer ONNX models like Gemma 4 E2B and LFM2 1.2B. First load downloads the weights, then they're cached and work offline.
- Tools are split into two tiers. Local tier (calculator, time, Wikipedia, weather via direct CORS calls) never touches my infrastructure. Gateway tier (web search, page reading) goes through open-source Cloudflare Pages Functions because search engines don't allow browser CORS — this can be switched off in settings, at which point nothing touches any server I run.
- Each turn runs a temperature-zero planning pass that can only emit a JSON tool call matching an allowlisted schema, separate from the answer pass — so 1B-class models don't accidentally "hallucinate" tool invocations mid-prose.
Honest limitations:
- Model ceiling is ~8B quantized on 16GB+ desktops. Phones are realistically 1B-class (LFM2 1.2B / Llama 3.2 1B). These are single-task agents, not GPT-4 replacements.
- iOS is the hard wall: WebKit caps a single tab at roughly 1.5GB regardless of how much RAM the iPhone has, so iPhones are locked to the smallest models. Mid-range Android often outruns the newest iPhone here.
- First load is a 700MB–2GB download depending on the model. After that it's cached.
- Optional short links (/s/<id>) store the agent payload in KV for social previews — the full hash URL remains self-contained and works even if this deployment disappears.
Funny, i did this myself some months ago - its actually not that tricky to run a llm in your browser. Tho, the problem is that the models fitting in there are well relativly "tiny" in my opinion.
Cool idea tho to make it as accessible (:
This is excellent!
The Web Platform is great for agents, especially given the browser already is the user's agent itself. Give Prompt API (or other providers, Transformers.js, LiteRT-LM) a try. You can have agents communicate over WebRTC as well: https://xt-ml.github.io/shadow-claw/
so thats the library it uses under hood https://npmx.dev/search?q=%40mlc
One click agent run on browser, What can go wrong :)
Yeah. What possibly can go wrong? I literally can't think of a single risk for this.
What models have you been able to run with this?
Hi HN, author here.
HashAgent started with a simple question: can you run a useful AI agent with zero hosting costs? The answer I landed on: the user's own device is the server.
How it works:
- The entire agent definition (name, system prompt, greeting, generation config) is encoded as compressed base64url JSON in the URL hash fragment. The hash never hits a server — sharing a link is sharing the agent.
- Inference runs in-browser via WebGPU. Two runtimes: WebLLM (MLC) for Llama 3.2 / Phi / Mistral / Qwen, and Transformers.js for newer ONNX models like Gemma 4 E2B and LFM2 1.2B. First load downloads the weights, then they're cached and work offline.
- Tools are split into two tiers. Local tier (calculator, time, Wikipedia, weather via direct CORS calls) never touches my infrastructure. Gateway tier (web search, page reading) goes through open-source Cloudflare Pages Functions because search engines don't allow browser CORS — this can be switched off in settings, at which point nothing touches any server I run.
- Each turn runs a temperature-zero planning pass that can only emit a JSON tool call matching an allowlisted schema, separate from the answer pass — so 1B-class models don't accidentally "hallucinate" tool invocations mid-prose.
Honest limitations:
- Model ceiling is ~8B quantized on 16GB+ desktops. Phones are realistically 1B-class (LFM2 1.2B / Llama 3.2 1B). These are single-task agents, not GPT-4 replacements.
- iOS is the hard wall: WebKit caps a single tab at roughly 1.5GB regardless of how much RAM the iPhone has, so iPhones are locked to the smallest models. Mid-range Android often outruns the newest iPhone here.
- First load is a 700MB–2GB download depending on the model. After that it's cached.
- Optional short links (/s/<id>) store the agent payload in KV for social previews — the full hash URL remains self-contained and works even if this deployment disappears.
MIT licensed: https://github.com/mason131928/hashagent
Happy to answer anything about squeezing LLMs into browser memory limits — the iOS jetsam debugging alone was a journey.