Run an LLM on your own machine.

There is a specific moment that pushes security people toward local models: you want to paste a log snippet, a config, or an incident timeline into a model to help you think, and you realise that snippet should absolutely not leave your network. A local LLM solves that. It runs on your own hardware, works offline, and nothing you type is sent to anyone. For sensitive drafting, note-taking and documentation, that property alone is worth the setup.

Why local, specifically

• Privacy by construction. The data never leaves the machine. There is no API provider to trust, no retention policy to read, no cross-border transfer to justify to a compliance team.
• Offline and reproducible. The model works on a plane or in an air-gapped lab, and the same model version gives you the same behaviour tomorrow, no silent server-side updates.
• No per-token cost. Once it runs, you can throw as much text at it as your hardware allows without watching a meter.

The trade-off is honest: a model you can run on a laptop is smaller and less capable than the largest hosted models. For summarising notes, reformatting, drafting documentation and rubber-ducking a problem, that gap rarely matters. For frontier reasoning, it does.

Installing Ollama

Ollama is the least-friction way to run open-weight models locally on macOS, Linux and Windows. After installing it from the official site, pulling and running a model is two commands:

# download a small, capable open-weight model
ollama pull llama3.2

# start chatting with it, fully offline
ollama run llama3.2

That is the whole setup. The first command downloads the weights once; after that everything runs locally. If you have a GPU, Ollama uses it automatically; on CPU-only machines a smaller model (1–3 billion parameters) keeps responses quick. You can list what you have pulled with ollama list and remove a model with ollama rm.

Prompts that earn their keep

A local model is a strong writing and thinking aid for security work. A few patterns that consistently help:

• Summarise an incident timeline. Paste rough, timestamped notes and ask for a clean chronological summary with an "open questions" section. Good for handoffs.
• Explain a config or log line. "Explain what this iptables rule does, line by line, and flag anything that looks overly permissive." You still verify, but it accelerates the read.
• Draft documentation. Turn a bulleted list of what you did into a first-draft runbook, then edit. The model handles the boilerplate; you supply the judgement.
• Generate study material. Feed it a topic and ask for practice questions – pairs well with our prompts for self-study.

What never goes into any model – local or not

Running locally removes the network-exposure risk, but it does not suspend good judgement. Keep these off the prompt regardless:

• Live credentials and secrets. API keys, passwords, private keys. A local model will not exfiltrate them, but they end up in your shell history, scrollback and any saved transcript. Redact first.
• Real personal data you have no need to process. Minimising what you handle is a habit worth keeping even when the tool is offline.
• Anything you would then paste into a shell or a query unread. Treat model output as a draft by a fast, confident junior: useful, occasionally wrong, never trusted blindly. This is the "insecure output handling" risk from the OWASP LLM list, applied to your own workflow.

Picking a model size for your hardware

The most common early frustration is pulling a model too big for the machine and watching it crawl. A rough guide to match model to hardware:

Start small, confirm the workflow is useful, then size up only if the quality genuinely falls short. A fast 3B model you actually use beats a sluggish 8B one you avoid. You can keep several pulled at once and switch per task: a small one for quick reformatting, a larger one when you need it to reason.

A local LLM will not replace your best hosted model for hard reasoning. But as a private, offline, zero-cost assistant for the daily grind of security writing – summaries, explanations, drafts, study – it is one of the highest-value tools you can set up in ten minutes, and every byte stays on your machine.

Frequently asked questions