I built FreeWeights because I wanted a simpler way to try local AI.

The usual local setup can still feel like a project. You install a desktop app, learn a model manager, pick from a pile of model files, then hope your machine can handle the one you chose. That is fine if you already know what you are doing. It is a lot if you just want to ask an AI model a question without paying for another subscription.

FreeWeights takes a different route. You open a website, load a model, and chat. The model runs on your own device inside the browser.

That sounds a little strange at first. We are used to web apps sending hard work to servers. Most AI chat apps work that way too. You type a message, the message goes to a company server, a model runs somewhere else, and the answer comes back.

FreeWeights is not built that way. After the model downloads, the browser does the work locally.

The browser tech behind it

The important browser technology here is WebGPU.

WebGPU lets a web app use the graphics hardware in your computer or phone. That matters because language models need a lot of parallel math. A CPU can run some models, but it is usually slow. A GPU is much better suited to the job.

FreeWeights uses WebLLM, an open-source runtime that makes browser-based model loading and inference possible. In the current app, the model runtime runs inside a web worker. That keeps the heavy work away from the main page thread so the interface can stay responsive while the model loads or answers.

The worker imports the WebLLM runtime from the app's vendored files. When you choose a model, FreeWeights creates a WebLLM engine in that worker and asks it to load the selected model. The model files are fetched by the browser, cached by the browser, and then used locally for generation.

There is no server-side model API hiding behind the chat box.

What happens during a model load

The first model load is the part that surprises people.

A language model is not a tiny script. Even a small one can be a gigabyte-scale download. Bigger models can need several gigabytes of storage and enough GPU memory to fit the model while it runs.

So the first load is closer to installing game data than opening a normal web page. The browser downloads the model files, WebLLM prepares the runtime, and the model gets loaded into GPU memory. Once that finishes, the app can send your messages to the local model and stream replies back into the chat.

FreeWeights tries to make that less mysterious. The app shows model size, estimated memory needs, cache guidance, WebGPU status, and model load progress. If a model is too large for the device, the app tries to explain what happened in normal language instead of dumping a raw browser error on the screen.

That part took more work than I expected. Browser-based AI fails in very browser-shaped ways. A model can hit cache limits. A GPU device can disappear. A worker can stall. A phone can simply decide it has had enough. FreeWeights includes diagnostics for those cases, but diagnostic reports intentionally leave out chat transcripts and prompt text.

What stays local

The key privacy difference is where inference happens.

With most web-based AI chat products, the model runs somewhere else. The service has to receive your message to answer it. That can be fine for many use cases, but it is not the only possible shape for AI chat.

With FreeWeights, the model answer is generated on your device after the model has loaded. Your prompts and replies stay in your browser. The app does not need a paid API key because it is not paying a cloud model every time you send a message.

There are still network requests. The website has to load. The model files have to download the first time you choose a model. Analytics can record page events. But the chat itself is not powered by sending every prompt to a remote model server.

That distinction is the whole point of the project.

Why not just use a desktop local AI app?

You absolutely can.

Tools like Ollama and LM Studio are great if you want a native local AI setup. They are usually better for power users, heavier models, and more serious offline workflows. I am not trying to replace them.

FreeWeights is for the moment before that. It is for someone who has heard local AI is possible, but does not want to install anything yet. It is also useful when you want a quick browser-based option on a machine where installing software is annoying or impossible.

The tradeoff is performance and control. A browser runtime has limits. Native apps can do more. FreeWeights wins on simplicity.

The model choice problem

One thing I underestimated was how confusing model choice is for normal users.

A technical user sees a model name and can guess quite a bit from it: family, parameter count, quantization, context length, release era. A normal person sees a string that looks like a failed password reset token.

FreeWeights tries to translate that mess into a friendlier catalog. It parses model metadata, estimates storage and memory needs, groups models by task, and gives users a better starting point for their hardware.

The app still exposes technical details for people who want them. It just does not require that knowledge up front.

What this is good for today

Browser-local AI is already useful, but it is not magic.

Smaller models are good for quick questions, drafts, simple explanations, and experimenting with local chat. Newer machines can try larger models. Phones and older laptops should start small.

The best way to think about FreeWeights is not "this replaces every frontier model." It does not. The better question is simpler: do you really need the latest paid model for this particular task?

A lot of the time, you do not.

Open-source language models have improved quickly. Models that run locally today can feel surprisingly capable compared with the best public chat models from just a few years ago. For casual use, private notes, learning, and lightweight writing help, that is often enough.

Where FreeWeights is headed

The goal is to make local AI feel less like a hobbyist setup and more like a normal website.

Open the page. Pick a model. Chat privately. Learn more only if you want to.

There is still plenty to improve. Model recommendations can get smarter. Browser compatibility can be clearer. Error handling can always be better. The app should keep getting easier for people who do not know what WebGPU is and do not want to learn before asking their first question.

That is the direction: local AI without the ceremony.