Run AI Models on Your Own Computer — LLM 0.32 Update

Today, two open-source releases make AI more personal: a major update to the llm command-line tool lets you control which models you use from your terminal, and Mistral released a small, open safety model. For beginners, this means more privacy, more control, and fewer reasons to be locked into one AI website.

Can you run AI models on your own computer?

Yes — you absolutely can, and the free, open-source llm tool just made it easier. Simon Willison’s release post calls version 0.32 “the most significant new version of LLM since the initial launch of the project,” and it connects to both hosted models (like GPT-5.6) and local ones (like Ollama).

What’s new in LLM 0.32

The llm tool is a command-line program that lets you talk to AI models from your terminal instead of a website. It’s free, open-source (Apache 2.0), and works with a wide range of providers — OpenAI, Anthropic, Gemini, Mistral, OpenRouter — plus local models running in LM Studio or Ollama.

Version 0.32, released yesterday, adds several features that matter even if you’ve never touched a terminal:

• Reasoning traces displayed: The tool now shows the model’s “thinking” steps, so you can see how it reached an answer. You can hide them with a flag if you don’t want the noise.
• New default model: GPT-5.6 Luna is the new default — described as inexpensive and capable in the changelog.
• Server-side tools: OpenAI’s CodeInterpreter and WebSearch are now supported, as are Anthropic’s WebSearch, WebFetch, CodeExecution, and MCP servers via the Anthropic plugin.
• One-line endpoint command: The new llm openai endpoint command lets you run a prompt against any OpenAI-compatible API — including a local model in LM Studio — without configuration. The example in the release post uses google/gemma-4-12b on localhost.
• Better Python API: Structured messages and streaming events (reasoning, text, tool calls) are now supported, per the Python API docs.
• Content-addressable logging: Your prompt logs are deduplicated like Git, so you don’t store the same thing twice.
• Local API server: The new llm-chat-completions-server plugin turns your machine into an OpenAI-compatible server, per the plugin repo.

Why this matters for beginners

If you’ve only used ChatGPT or Claude in a browser, this tool is a different way to work: you type a prompt, get a response, and move on — all from your own computer. You don’t need a subscription for local models, and your conversations stay private because they never leave your machine.

The “self-host vs hosted” choice is now a slider, not a binary. You can start with a free local model (like Gemma) and switch to a paid hosted one (like GPT-5.6) with the same tool. That flexibility is rare.

Context: Previously, running AI locally meant installing separate tools for each model provider, or wrestling with configuration files. The llm tool consolidates that into one interface. It replaces the need to juggle multiple web apps or custom scripts.

How it compares to alternatives

If you’re choosing between this and a web app like ChatGPT or Claude, the trade-off is simplicity vs control. Web apps are zero-setup; llm gives you more options and privacy. If you’re comparing it to local-only tools like Ollama or LM Studio, llm adds hosted-model support on top of local ones. If you’re choosing between this and X, our Comparison Database at /comparisons/ breaks down how they score across Ease, Features, Performance, Docs, and Support.

What’s on the roadmap

The “Local LLM” category in our Comparison Database tracks tools like Ollama and LM Studio — see our roadmap for what’s being evaluated next. This tool sits alongside those, and we’re watching how it evolves.

Verdict

Yes, this is worth knowing about — even for beginners. The llm tool is free, open-source, and removes the friction of choosing between local and hosted models. If you’re curious about running AI on your own hardware, or you want a single interface for multiple providers, this is a strong starting point. You don’t need to be a developer to benefit; the command-line basics are simple to learn.

FAQ

Is the llm tool free?

Yes. The llm tool is open-source under the Apache 2.0 license, and it works with free local models. If you connect to paid hosted models like GPT-5.6, you pay the provider directly — the tool itself has no cost. See the changelog for details.

Do I need to know code to use it?

Basic comfort with a terminal helps, but you don’t need to be a programmer. The core commands are simple: install, type a prompt, get an answer. The new llm openai endpoint command even removes the need for configuration when using local models, per Simon Willison’s post.

What’s the difference between llm and Ollama?

Ollama is focused on running local models only; llm does that and connects to hosted providers. If you only care about local models, Ollama is simpler — our Ollama review scores it 8.6/10. If you want one tool for both worlds, llm is worth trying.

How do AI apps filter harmful content?

AI apps use safety classifiers to decide what content is allowed, and Mistral’s new open-source model, Shieldstral, makes this process smaller and more customizable. Mistral’s announcement describes it as a 3-billion-parameter multimodal safety classifier under the Apache 2.0 license, small enough to run on a single 16GB NVIDIA GPU.

What Shieldstral does differently

Most safety filters have a fixed list of harm categories baked into their weights. Shieldstral flips that: you write the moderation policy as a plain-language question at inference time — e.g., “Does this content promote violence?” — and it returns a calibrated yes/no score. No retraining needed to change policy. It also handles text and images in one model, so you can check prompts, responses, and images with a single interface.

The model is available on Hugging Face, with a technical report. Mistral claims it matches or outperforms open guard models up to 7x its size on text safety, refusal detection, and multimodal benchmarks — but this briefing is based on official documentation and the vendor’s announcement; we did not run Shieldstral hands-on.

Why this matters for beginners

Every AI app you use — chatbots, image generators, social features — has a safety filter deciding what’s allowed. Shieldstral shows these filters are becoming open, small, and self-hostable. That matters for privacy: your content can be judged on your own hardware, not sent to a third party. For app makers, it means more control over moderation policies without retraining.

How it compares

If you’re choosing between Shieldstral and a closed safety API (like OpenAI’s moderation endpoint), the trade-off is convenience vs control. Closed APIs are plug-and-play; Shieldstral gives you policy flexibility and privacy. Our Comparison Database at /comparisons/ tracks how such tools score across Ease, Features, Performance, Docs, and Support.

Verdict

Shieldstral matters mostly to app makers, but it signals a shift: safety is becoming open and self-hostable. For a beginner, the practical takeaway is that AI apps you use may soon offer more privacy and better-tailored moderation. It’s not a tool you need to install today, but it’s worth understanding.

FAQ

Do I need to be an AI expert to use Shieldstral?

No, but you’d need to be building an AI app to use it directly. The model runs on a single GPU and takes a plain-language policy question — no retraining needed. If you’re just an app user, you’ll benefit indirectly through better moderation, per Mistral’s announcement.

Is Shieldstral really open-source?

Yes, it’s released under the Apache 2.0 license, which means you can use, modify, and distribute it freely. You can download it from Hugging Face and run it on your own hardware.

What this means for your next AI tool choice

If you’re choosing between running AI on your own computer and sticking with hosted apps, this week’s releases say: you don’t have to pick. The llm tool gives you one interface for both worlds, and Shieldstral shows that even safety filtering can live on your own hardware. For a beginner, the practical step is to try a local model first — see our Ollama review or LM Studio review for hands-on comparisons. If privacy or cost is a concern, local is now a real option. If you just want things to work, hosted apps are still fine. The choice is yours — and that’s the point. Compare your options in the Comparison Database, and check our roadmap for what’s being evaluated next. This story was produced by our automated pipeline — track what’s coming next at /cron-pipeline/.

References

[1] Simon Willison’s release post [2] LLM 0.32 changelog [3] llm-anthropic plugin [4] LLM Python API docs [5] llm-chat-completions-server [6] Mistral Shieldstral announcement [7] Shieldstral on Hugging Face [8] Shieldstral technical report