The AI headlines are dominated by ever-larger models with trillions of parameters. But a quieter revolution is happening at the opposite end of the scale: models so small — some just a few megabytes — that they run entirely inside your web browser, with no server and no cloud.
The shift to small models
Not every task needs a giant model. Turning text into search-ready embeddings, removing an image background, transcribing a short clip, classifying a message — these can be handled by compact, specialized models that fit in the memory of a phone. Techniques like quantization and distillation shrink them further without gutting quality.
How it runs in the browser
Modern browsers can execute these models using WebAssembly and WebGPU, tapping your device’s own CPU and graphics for inference. The model downloads once, caches, and then runs locally — often in a fraction of a second.
Why it matters
Three big wins. Privacy: your data never leaves the device. Cost: there are no per-request server bills, so tools can be free and scale infinitely. Resilience: it works offline and can’t be rate-limited. We built our own background-remover on exactly this principle — the AI runs in your browser and your images are never uploaded.
The takeaway
Bigger isn’t always better. For a huge range of real tasks, tiny on-device AI is faster, cheaper, and more private — and it’s only getting better.
Comments
Post a Comment