Live Beta — free to use during beta. Paid tiers are marked "Free during beta" until monetization launches.

Local AI Models Guide

Real, verified pull commands for small local AI models — every use case, organized by the hardware you actually have. Nothing hosted here, every link goes straight to the real source (Ollama's registry or Hugging Face).

Local AI Models Guide — real, working pull commands for every use case

A curated list of small local AI models covering every common AI use case — chat,

code, reasoning, images, video, voice, translation, and more — that run on modest

consumer hardware. Every command below is real and verified: Ollama tags checked

against Ollama's live registry, LocalAI ids checked against LocalAI's live model

gallery (checked 2026-08-02). No files are hosted here — these commands download

directly from Ollama's registry or Hugging Face.

How to read this: organized by System RAM needed (8/12/16/24 GB), then by

VRAM needed (1/2/3/4 GB) within each. If your machine has more RAM/VRAM than a

given section, everything in the smaller sections below it will run on your machine

too — not just the section matching your exact specs.

8 GB RAM

1 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Qwen 2.5 0.5B

  • Good for: Basic automation, blazing fast edge processing
  • Watch out for: Extremely limited knowledge base
  • Ollama: ollama pull qwen2.5:0.5b (~379 MB)
  • LocalAI: local-ai run qwen3-0.6b (~462 MB) — Qwen3 0.6B replaces Qwen2.5 0.5B — newer generation, same tiny-chat role, already verified in gallery
  • https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Qwen 3 0.6B

  • Good for: Basic reasoning, ultra-fast routing
  • Watch out for: Low factual retrieval capability
  • Ollama: ollama pull qwen3:0.6b (you already have this)
  • LocalAI: local-ai run qwen3-0.6b (~462 MB)
  • https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

SmolLM2 360M

  • Good for: Prototyping, basic text routing
  • Watch out for: Essentially no world knowledge
  • Ollama: ollama pull smollm2:360m (you already have this)
  • LocalAI: local-ai run qwen3-0.6b (~462 MB) — no SmolLM2-360M in gallery; nearest ultra-tiny chat model
  • https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Qwen2 0.5B

  • Good for: Basic logic, blazingly fast generation, simple tasks
  • Watch out for: Very small knowledge base, limited reasoning
  • Ollama: ollama pull qwen2:0.5b (~336 MB)
  • LocalAI: local-ai run qwen3-0.6b (~462 MB) — Qwen2 (old gen) not in gallery; Qwen3 0.6B is the current tiny generalist
  • https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Gemma 3 1B

  • Good for: Strict instruction following
  • Watch out for: Often refuses benign prompts
  • Ollama: ollama pull gemma3:1b (you already have this)
  • LocalAI: local-ai run gemma-3-1b-it (~769 MB)
  • https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF/resolve/main/gemma-3-1b-it-Q4_K_M.gguf

Llama 3.2 1B

  • Good for: Summarization, basic tool calling (can call external tools/functions mid-conversation)
  • Watch out for: Hallucinates (makes up wrong-but-confident-sounding facts) on complex reasoning
  • Ollama: ollama pull llama3.2:1b (you already have this)
  • LocalAI: local-ai run llama-3.2-1b-instruct:q4_k_m (~770 MB)
  • https://huggingface.co/hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF/resolve/main/llama-3.2-1b-instruct-q4_k_m.gguf

TinyLlama 1.1B

  • Good for: Fast drafting, straightforward NLP
  • Watch out for: Low factual accuracy, older architecture
  • Ollama: ollama pull tinyllama (you already have this)
  • LocalAI: local-ai run qwen3-0.6b (~462 MB) — TinyLlama not in gallery; Qwen3 0.6B is the modern equivalent-tier chat model
  • https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

Qwen 2.5 Coder 0.5B

  • Good for: Inline code completions, fast syntax checking
  • Watch out for: Struggles with large logic blocks
  • Ollama: ollama pull qwen2.5-coder:0.5b (~379 MB)
  • LocalAI: local-ai run opencoder-1.5b-base (~1.3 GB) — no 0.5B coder in gallery; OpenCoder 1.5B is the smallest real code model
  • https://huggingface.co/QuantFactory/OpenCoder-1.5B-Base-GGUF/resolve/main/OpenCoder-1.5B-Base.Q4_K_M.gguf

Audio — Text to Speech

Turns written text into spoken audio — a synthetic voice reading your text out loud.

Piper TTS

  • Good for: Blazing fast text-to-speech, low footprint
  • Watch out for: Less emotive than Bark
  • LocalAI: local-ai run piper-en_US-lessac-medium-crispasr (~30 MB)
  • https://huggingface.co/LocalAI-Community/piper-voices-GGUF/resolve/main/piper-en_US-lessac-medium-f16.gguf

Embedding / RAG

Doesn't chat at all — turns text into a mathematical "fingerprint" used for search. This is the engine behind "search my own documents" features (RAG).

All-MiniLM-L6-v2

  • Good for: Extremely lightweight local document indexing
  • Watch out for: Lower semantic accuracy than Nomic
  • Ollama: ollama pull all-minilm (~44 MB)
  • LocalAI: local-ai run all-MiniLM-L6-v2 (~86 MB)

BGE-m3

  • Good for: Multilingual (handles multiple languages) vector search (Hebrew/Spanish support)
  • Watch out for: Does not generate text
  • Ollama: ollama pull bge-m3 (~1.1 GB)
  • LocalAI: local-ai run bge-m3-colbert (~2.1 GB)

Nomic-Embed-Text

  • Good for: Creating vector embeddings for local document search
  • Watch out for: Does not generate text
  • Ollama: ollama pull nomic-embed-text (you already have this)
  • LocalAI: local-ai run nomic-embed-text-v1.5 (~261 MB)
  • https://huggingface.co/mradermacher/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

2 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

BitNet 2B-4T

  • Good for: Extremely fast CPU inference
  • Watch out for: Highly experimental
  • Ollama: not available — 'bitnet:2b-4t' doesn't exist in Ollama's registry, no substitute found
  • LocalAI: not available — no BitNet or 1-bit-architecture model in the gallery at all — genuinely no substitute for this specific architecture

Qwen 2.5 1.5B

  • Good for: Strong multilingual (handles multiple languages), balanced generalist
  • Watch out for: Sometimes loses context
  • Ollama: ollama pull qwen2.5:1.5b (you already have this)
  • LocalAI: local-ai run qwen3-1.7b (~1.2 GB) — Qwen3 1.7B replaces Qwen2.5 1.5B, same tier, already verified
  • https://huggingface.co/MaziyarPanahi/Qwen3-1.7B-GGUF/resolve/main/Qwen3-1.7B.Q4_K_M.gguf

Qwen 3 1.7B

  • Good for: Agentic (able to plan and use tools on its own, not just chat) workflows, basic classification
  • Watch out for: Factual accuracy on obscure topics
  • Ollama: ollama pull qwen3:1.7b (you already have this)
  • LocalAI: local-ai run qwen3-1.7b (~1.2 GB)
  • https://huggingface.co/MaziyarPanahi/Qwen3-1.7B-GGUF/resolve/main/Qwen3-1.7B.Q4_K_M.gguf

Qwen2 1.5B

  • Good for: Multilingual (handles multiple languages) translation, general chat
  • Watch out for: Advanced coding or complex logic
  • Ollama: ollama pull qwen2:1.5b (~892 MB)
  • LocalAI: local-ai run qwen3-1.7b (~1.2 GB) — old-gen Qwen2 not in gallery; Qwen3 1.7B is the current equivalent
  • https://huggingface.co/MaziyarPanahi/Qwen3-1.7B-GGUF/resolve/main/Qwen3-1.7B.Q4_K_M.gguf

SmolLM2 1.7B

  • Good for: Local text manipulation, rewriting
  • Watch out for: Lacks nuanced conversational abilities
  • Ollama: ollama pull smollm2:1.7b (you already have this)
  • LocalAI: local-ai run smollm2-1.7b-instruct (~1007 MB)
  • https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct-q4_k_m.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

DeepSeek Coder 1.3B

  • Good for: Code completion, debugging snippets
  • Watch out for: Not suited for general chat
  • Ollama: ollama pull deepseek-coder:1.3b (~740 MB)

Qwen 2.5 Coder 1.5B

  • Good for: FPC, PHP, bash snippets, local code assistance
  • Watch out for: Requires clear context
  • Ollama: ollama pull qwen2.5-coder:1.5b (you already have this)
  • LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — real, current small code-instruct model
  • https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Reasoning / Logic

Built to "think through" harder problems step by step — math, logic puzzles, multi-step decisions — instead of just pattern-matching an answer. Often slower because it works through the problem first.

DeepSeek-R1 1.5B

  • Good for: Deep reasoning for its size, math
  • Watch out for: Can over-think simple prompts
  • Ollama: ollama pull deepseek-r1:1.5b (you already have this)
  • LocalAI: local-ai run deepseek-r1-distill-qwen-1.5b (~1.0 GB)
  • https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-1.5B-GGUF/resolve/main/DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M.gguf

Vision Analysis

Can look at an image you give it and describe, answer questions about, or read text from it. Doesn't generate new images — just understands existing ones.

Moondream2 1.8B

  • Good for: Fast local Img querying
  • Watch out for: Text generation is basic
  • Ollama: ollama pull moondream (~1.6 GB)
  • LocalAI: local-ai run moondream2 (~3.5 GB)
  • https://huggingface.co/moondream/moondream2-gguf/resolve/main/moondream2-text-model-f16.gguf
  • https://huggingface.co/moondream/moondream2-gguf/resolve/main/moondream2-mmproj-f16.gguf

Audio — Speech to Text

Turns spoken audio (a recording, podcast, video) into written text — transcription.

Whisper (Small)

  • Good for: Transcribing podcasts/videos locally
  • Watch out for: Not a generation model
  • LocalAI: local-ai run whisper-small (~465 MB)
  • https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin

Translation

Translates text (or sometimes speech) from one language to another. Some are conversational generalists that translate okay; these are dedicated specialists.

Madlad400 3B

  • Good for: Machine translation (400+ languages)
  • Watch out for: Not suitable for conversational chat
  • Ollama: ollama pull translategemma (~3.1 GB) — substituted for 'madlad400:3b' (doesn't exist in Ollama's registry)

Summarization

Condenses a long document into a short summary. Dedicated summarizers do this more reliably than asking a general chat model to "summarize this."

BART-Large-CNN

  • Good for: Dedicated summarization model (non-conversational)
  • Watch out for: Strictly for summarization
  • LocalAI: local-ai run llama-chat-summary-3.2-3b (~1.9 GB) — BART-Large-CNN not in gallery; Llama-Chat-Summary 3.2 3B is a real, current dedicated summarization finetune
  • https://huggingface.co/bartowski/Llama-Chat-Summary-3.2-3B-GGUF/resolve/main/Llama-Chat-Summary-3.2-3B-Q4_K_M.gguf

Pegasus-X-Base

  • Good for: Long-document abstractive summarization
  • Watch out for: Hard to configure properly
  • LocalAI: local-ai run llama-chat-summary-3.2-3b (~1.9 GB) — Pegasus-X not in gallery; same real summarization substitute as BART
  • https://huggingface.co/bartowski/Llama-Chat-Summary-3.2-3B-GGUF/resolve/main/Llama-Chat-Summary-3.2-3B-Q4_K_M.gguf

3 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Gemma 2 2B

  • Good for: Step-by-step reasoning
  • Watch out for: Strict alignment (built-in safety behavior)
  • Ollama: ollama pull gemma2:2b (you already have this)

Ministral-3 3B

  • Good for: Solid generalist
  • Watch out for: Overshadowed by Llama 3.2
  • Ollama: ollama pull ministral-3 (~5.6 GB)
  • LocalAI: local-ai run mistralai_ministral-3-3b-instruct-2512-multimodal (~3.6 GB) — real 2512 release, already applied
  • https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-GGUF/resolve/main/Ministral-3-3B-Instruct-2512-Q4_K_M.gguf
  • https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-GGUF/resolve/main/mmproj-F32.gguf

Llama 3.2 3B

  • Good for: Nuanced chat, tool calling (can call external tools/functions mid-conversation)
  • Watch out for: Can loop tools
  • Ollama: ollama pull llama3.2 (you already have this)
  • LocalAI: local-ai run llama-3.2-3b-instruct-uncensored (~2.1 GB)
  • https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-uncensored-GGUF/resolve/main/Llama-3.2-3B-Instruct-uncensored-Q4_K_M.gguf

Qwen 2.5 3B

  • Good for: Highly versatile generalist
  • Watch out for: Slightly less capable at complex math
  • Ollama: ollama pull qwen2.5:3b (you already have this)
  • LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — plain Qwen2.5 3B not in gallery; Falcon3 3B is a real, current 3B-class generalist
  • https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

Stable Code 3B

  • Good for: Fast completion for HTML, CSS, JS
  • Watch out for: Lacks complex logical reasoning
  • Ollama: ollama pull stable-code:3b (~1.5 GB)
  • LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — Stable Code 3B not in gallery; OpenCoder covers the same lightweight-completion role
  • https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Qwen 2.5 Coder 3B

  • Good for: High-quality scripting, FPC/Lazarus syntax fixes
  • Watch out for: Less conversational depth
  • Ollama: ollama pull qwen2.5-coder:3b (you already have this)
  • LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — closest real small code model in the current gallery
  • https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Starcoder2 3B

  • Good for: Offline IDE autocomplete integration
  • Watch out for: General knowledge is poor
  • Ollama: ollama pull starcoder2:3b (you already have this)
  • LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — StarCoder2 not in gallery at any size; OpenCoder is the real small-code substitute
  • https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Image Generation

Creates brand-new images from a text description ("a cat wearing a top hat"). The opposite of Vision Analysis.

LCM (Latent Consistency)

  • Good for: Extremely fast generation (2-4 steps)
  • Watch out for: Lower fidelity than base SD
  • LocalAI: local-ai run sd-1.5-ggml (~1.5 GB) — LCM fast-inference variant not in gallery; sd-1.5-ggml is the real small SD1.5-family image generator
  • https://huggingface.co/second-state/stable-diffusion-v1-5-GGUF/resolve/main/stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf

Audio — Sound/FX Gen

Generates non-music sound effects or ambient audio from a text description.

AudioLDM

  • Good for: Generates sound effects from text
  • Watch out for: Not meant for high fidelity vocal tracks
  • LocalAI: local-ai run acestep-cpp-turbo-4b (~7.6 GB) — AudioLDM (sound effects from text) not in gallery; AceStep is a real music/audio generation model — not a perfect substitute (music vs. arbitrary SFX) but the closest real audio-gen entry
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-5Hz-lm-4B-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/Qwen3-Embedding-0.6B-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-v15-turbo-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/vae-BF16.gguf

Voice Cloning

Text-to-speech that mimics a specific person's voice, given a short sample of them speaking.

XTTS-v2

  • Good for: Clones voices locally with high accuracy
  • Watch out for: Can stutter on low VRAM splits
  • LocalAI: local-ai run qwen3-tts-cpp-1.7b-base (~2.2 GB) — XTTS-v2 not in gallery; Qwen3-TTS is a real current voice-cloning-capable TTS model
  • https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF/resolve/main/qwen-talker-1.7b-base-Q8_0.gguf
  • https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF/resolve/main/qwen-tokenizer-12hz-Q8_0.gguf

4 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Gemma 3 4B

  • Good for: Robust logical deductions
  • Watch out for: Heavy memory usage during scaling
  • Ollama: ollama pull gemma3:4b (you already have this)
  • LocalAI: local-ai run gemma-3-4b-it (~3.1 GB)
  • https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/main/gemma-3-4b-it-Q4_K_M.gguf
  • https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/main/mmproj-model-f16.gguf

Qwen 3 4B

  • Good for: Advanced logic, agentic (able to plan and use tools on its own, not just chat) routing
  • Watch out for: Slightly large for 4GB VRAM
  • Ollama: ollama pull qwen3:4b-instruct (~2.3 GB)
  • LocalAI: local-ai run qwen3-4b (~2.3 GB)
  • https://huggingface.co/MaziyarPanahi/Qwen3-4B-GGUF/resolve/main/Qwen3-4B.Q4_K_M.gguf

Reasoning / Logic

Built to "think through" harder problems step by step — math, logic puzzles, multi-step decisions — instead of just pattern-matching an answer. Often slower because it works through the problem first.

Phi-3 Mini 3.8B

  • Good for: High logic, reasoning, following strict formats
  • Watch out for: Can hallucinate (makes up wrong-but-confident-sounding facts) on niche facts
  • Ollama: ollama pull phi3 (~2.0 GB)
  • LocalAI: local-ai run qwen3-4b (~2.3 GB) — Phi-3-mini not in gallery; Qwen3 4B covers the same reasoning-chat tier
  • https://huggingface.co/MaziyarPanahi/Qwen3-4B-GGUF/resolve/main/Qwen3-4B.Q4_K_M.gguf

Phi-4-mini 3.8B

  • Good for: Intricate reasoning, strict formatting
  • Watch out for: Requires explicit prompts
  • Ollama: ollama pull phi4-mini (~2.3 GB)
  • LocalAI: local-ai run qwen3-4b (~2.3 GB) — Phi-4-mini not in gallery; Qwen3 4B is the real current equivalent
  • https://huggingface.co/MaziyarPanahi/Qwen3-4B-GGUF/resolve/main/Qwen3-4B.Q4_K_M.gguf

Image Generation

Creates brand-new images from a text description ("a cat wearing a top hat"). The opposite of Vision Analysis.

OpenJourney

  • Good for: Midjourney-style stylized renders
  • Watch out for: Requires specific trigger words
  • LocalAI: local-ai run dreamshaper (~2.0 GB) — OpenJourney (Midjourney-style) not in gallery; Dreamshaper is a real stylized SD1.5-family alternative
  • https://huggingface.co/Lykon/DreamShaper/resolve/main/DreamShaper_8_pruned.safetensors

Stable Diffusion 1.5

  • Good for: Offline Img generation, highly moddable
  • Watch out for: Strict 512x512 limit on 4GB VRAM
  • LocalAI: local-ai run sd-1.5-ggml (~1.5 GB) — real, current small SD1.5 gguf build
  • https://huggingface.co/second-state/stable-diffusion-v1-5-GGUF/resolve/main/stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf

12 GB RAM

4 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Llama 3.1 8B (Q4)

  • Good for: High-tier chat, deep contextual understanding
  • Watch out for: Context window (how much text it can "remember" in one conversation) eats RAM quickly
  • Ollama: ollama pull llama3.1 (~4.6 GB)
  • LocalAI: local-ai run fusechat-llama-3.1-8b-instruct (~4.6 GB)
  • https://huggingface.co/bartowski/FuseChat-Llama-3.1-8B-Instruct-GGUF/resolve/main/FuseChat-Llama-3.1-8B-Instruct-Q4_K_M.gguf

Mistral 7B v0.3

  • Good for: Versatile tasks, reliable function calling (can call external tools/functions mid-conversation)
  • Watch out for: Older architecture compared to Llama 3.1
  • Ollama: ollama pull mistral (you already have this)
  • LocalAI: local-ai run mistral-7b-instruct-v0.3 (~4.1 GB)
  • https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf

Neural Chat 7B

  • Good for: Instruction following, highly responsive
  • Watch out for: Outdated compared to late-2025/2026 models
  • Ollama: ollama pull neural-chat (~3.8 GB)
  • LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — Neural Chat 7B (old Intel finetune) not in gallery; Falcon3 3B covers the same instruction-following chat role at a smaller footprint
  • https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf

Mistral 7B v0.3 (Q4)

  • Good for: Versatile tasks, reliable function calling (can call external tools/functions mid-conversation)
  • Watch out for: Slightly less capable than Llama 3.1 out of the box
  • Ollama: ollama pull mistral (you already have this)
  • LocalAI: local-ai run mistral-7b-instruct-v0.3 (~4.1 GB)
  • https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

CodeLlama 7B

  • Good for: Bash and basic C++ generation
  • Watch out for: Surpassed by newer models
  • Ollama: ollama pull codellama (~3.6 GB)
  • LocalAI: local-ai run codellama-7b (~3.6 GB)
  • https://huggingface.co/TheBloke/CodeLlama-7B-GGUF/resolve/main/codellama-7b.Q4_0.gguf

Starcoder2 7B

  • Good for: Deep offline IDE integration
  • Watch out for: Not conversational
  • Ollama: ollama pull starcoder2:7b (you already have this)
  • LocalAI: local-ai run deepseek-coder-v2-lite-instruct (~9.7 GB) — StarCoder2 not in gallery; DeepSeek-Coder-V2-Lite is the real larger code substitute
  • https://huggingface.co/LoneStriker/DeepSeek-Coder-V2-Lite-Instruct-GGUF/resolve/main/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

Reasoning / Logic

Built to "think through" harder problems step by step — math, logic puzzles, multi-step decisions — instead of just pattern-matching an answer. Often slower because it works through the problem first.

DeepSeek-R1 7B (Q4)

  • Good for: Distilled (a smaller model trained to copy a bigger one's behavior) from Qwen, excellent logical flow
  • Watch out for: Can hallucinate (makes up wrong-but-confident-sounding facts) system setups
  • Ollama: ollama pull deepseek-r1:7b (~4.4 GB)
  • LocalAI: local-ai run deepseek-r1-distill-qwen-7b (~4.4 GB)
  • https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF/resolve/main/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf

DeepSeek-R1 8B

  • Good for: State-of-the-art step-by-step logic, mathematical proofs, complex deductions
  • Watch out for: Produces long 'thought' chains before answering
  • Ollama: ollama pull deepseek-r1:8b (~4.9 GB)
  • LocalAI: local-ai run deepseek-r1-distill-llama-8b (~4.6 GB)
  • https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/resolve/main/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf

DeepSeek-R1 8B (Q4)

  • Good for: State-of-the-art step-by-step logic, mathematical proofs
  • Watch out for: Produces long 'thought' chains
  • Ollama: ollama pull deepseek-r1:8b (~4.9 GB)
  • LocalAI: local-ai run deepseek-r1-distill-llama-8b (~4.6 GB)
  • https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/resolve/main/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf

Audio — Text to Speech

Turns written text into spoken audio — a synthetic voice reading your text out loud.

Bark

  • Good for: Realistic, emotive Text-to-Speech
  • Watch out for: Resource intensive
  • LocalAI: local-ai run piper-en_US-lessac-medium-crispasr (~30 MB) — Bark not in gallery; Piper is the real, fast TTS already verified — less emotive but real and working
  • https://huggingface.co/LocalAI-Community/piper-voices-GGUF/resolve/main/piper-en_US-lessac-medium-f16.gguf

Translation

Translates text (or sometimes speech) from one language to another. Some are conversational generalists that translate okay; these are dedicated specialists.

Aya 23 8B

  • Good for: Highly accurate multi-language (Spanish, Hebrew, English)
  • Watch out for: Needs heavy RAM offloading
  • Ollama: ollama pull aya (~4.5 GB)
  • LocalAI: local-ai run aya-23-8b (~4.7 GB)
  • https://huggingface.co/bartowski/aya-23-8B-GGUF/resolve/main/aya-23-8B-Q4_K_M.gguf

SeamlessM4T

  • Good for: Text-to-Text, Audio-to-Audio translation
  • Watch out for: Heavy resource footprint
  • LocalAI: not available — no SeamlessM4T or equivalent any-to-any speech translation model in the gallery — genuinely no substitute

TowerInstruct 7B

  • Good for: Professional-grade translation workflows
  • Watch out for: Focuses strictly on EU/Major languages
  • Ollama: ollama pull translategemma (~3.1 GB) — substituted for 'towerinstruct' (doesn't exist in Ollama's registry)
  • LocalAI: not available — no dedicated translation-specialist model in the gallery — use a strong generalist (Qwen3 8B) with a translation prompt instead

Storytelling / Creative Writing

Tuned for fiction, roleplay, and narrative writing rather than factual accuracy or following instructions strictly.

Llama 3.1 8B

  • Good for: Roleplay, world-building, narrative generation
  • Watch out for: Context window (how much text it can "remember" in one conversation) eats RAM in long stories
  • Ollama: ollama pull llama3.1 (~4.6 GB)
  • LocalAI: local-ai run fusechat-llama-3.1-8b-instruct (~4.6 GB)
  • https://huggingface.co/bartowski/FuseChat-Llama-3.1-8B-Instruct-GGUF/resolve/main/FuseChat-Llama-3.1-8B-Instruct-Q4_K_M.gguf

Nous-Hermes-2 7B

  • Good for: Highly conversational, dramatic narrative flows
  • Watch out for: Slightly older dataset
  • Ollama: ollama pull nous-hermes2 (~5.7 GB)
  • LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — Nous-Hermes-2 not in gallery; Falcon3 3B for general conversational storytelling
  • https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf

16 GB RAM

4 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Gemma 2 9B (Q3/Q4)

  • Good for: Top-tier nuance
  • Watch out for: Will severely bottleneck a 4GB VRAM card
  • Ollama: ollama pull gemma2:9b (~5.1 GB)

Qwen 2.5 7B (Q4)

  • Good for: SOTA generalist in the 7B class
  • Watch out for: Requires at least 16GB RAM for split
  • Ollama: ollama pull qwen2.5:7b (~4.4 GB)
  • LocalAI: local-ai run qwen3-8b (~4.7 GB) — Qwen3 8B replaces Qwen2.5 7B, already verified
  • https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Qwen2 7B (Q4)

  • Good for: Multilingual (handles multiple languages) coding, robust mathematics
  • Watch out for: Heavy RAM usage when context window (how much text it can "remember" in one conversation) expands
  • Ollama: ollama pull qwen2 (~4.1 GB)
  • LocalAI: local-ai run qwen3-8b (~4.7 GB) — old-gen plain Qwen2 not in gallery at all; Qwen3 8B is the current equivalent-tier generalist
  • https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

DeepSeek Coder 6.7B

  • Good for: Advanced multi-file coding
  • Watch out for: Resource intensive
  • Ollama: ollama pull deepseek-coder:6.7b (~3.6 GB)

DeepSeek Coder V2 (Lite)

  • Good for: Reading complex source code, debugging FPC/Lazarus
  • Watch out for: Slower startup
  • Ollama: ollama pull deepseek-coder-v2 (~8.3 GB)
  • LocalAI: local-ai run deepseek-coder-v2-lite-instruct (~9.7 GB)
  • https://huggingface.co/LoneStriker/DeepSeek-Coder-V2-Lite-Instruct-GGUF/resolve/main/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

Phind-CodeLlama 34B (Heavy Q2) / 7B

  • Good for: Answering deep programming questions, analyzing user code inputs
  • Watch out for: The 7B version is less capable than Qwen Coder
  • Ollama: ollama pull phind-codellama (~17.7 GB)
  • LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — Phind-CodeLlama not in gallery; OpenCoder for lightweight code Q&A
  • https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Phind-CodeLlama 7B

  • Good for: Answering deep programming questions
  • Watch out for: Requires precise prompting
  • Ollama: ollama pull phind-codellama (~17.7 GB)
  • LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — Phind-CodeLlama not in gallery; OpenCoder for lightweight code Q&A
  • https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Qwen 2.5 Coder 7B

  • Good for: Advanced multi-file coding, analyzing existing codeblocks
  • Watch out for: Resource heavy for large context
  • Ollama: ollama pull qwen2.5-coder:7b (you already have this)
  • LocalAI: local-ai run deepseek-coder-v2-lite-instruct (~9.7 GB) — real larger code model already verified elsewhere in this doc
  • https://huggingface.co/LoneStriker/DeepSeek-Coder-V2-Lite-Instruct-GGUF/resolve/main/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

Reasoning / Logic

Built to "think through" harder problems step by step — math, logic puzzles, multi-step decisions — instead of just pattern-matching an answer. Often slower because it works through the problem first.

Qwen 2.5 7B

  • Good for: High accuracy fact retrieval, excellent multilingual (handles multiple languages) (Spanish/Hebrew/English)
  • Watch out for: Context offloading uses significant System RAM
  • Ollama: ollama pull qwen2.5:7b (~4.4 GB)
  • LocalAI: local-ai run qwen3-8b (~4.7 GB) — Qwen3 8B replaces Qwen2.5 7B, already verified
  • https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Qwen 3 8B

  • Good for: Advanced logic, agentic (able to plan and use tools on its own, not just chat) routing
  • Watch out for: Requires RAM offloading
  • Ollama: ollama pull qwen3:8b (~4.9 GB)
  • LocalAI: local-ai run qwen3-8b (~4.7 GB)
  • https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Vision Analysis

Can look at an image you give it and describe, answer questions about, or read text from it. Doesn't generate new images — just understands existing ones.

BakLLaVA 7B

  • Good for: High detail Img captioning
  • Watch out for: Requires high System RAM
  • Ollama: ollama pull bakllava (~4.4 GB)
  • LocalAI: local-ai run qwen3-vl-2b-instruct (~1.8 GB) — BakLLaVA not in gallery; Qwen3-VL 2B is a real, current small vision model
  • https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen3-VL-2B-Instruct-Q4_K_M.gguf
  • https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/mmproj-F16.gguf

Llava 1.5 7B

  • Good for: Analyzing uploaded Imgs, OCR (OCR (reading text out of images))
  • Watch out for: Slow projection phase
  • Ollama: ollama pull llava (~4.4 GB)
  • LocalAI: local-ai run qwen3-vl-2b-instruct (~1.8 GB) — Llava 1.5 not in gallery; Qwen3-VL 2B covers image+text QA
  • https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen3-VL-2B-Instruct-Q4_K_M.gguf
  • https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/mmproj-F16.gguf

Qwen-VL-Chat 7B

  • Good for: Excellent OCR (OCR (reading text out of images)) and multilingual (handles multiple languages) visual QA
  • Watch out for: Slightly less conversational than Llava
  • Ollama: ollama pull qwen3-vl (~5.7 GB) — substituted for 'qwen-vl' (doesn't exist in Ollama's registry)
  • LocalAI: local-ai run qwen3-vl-2b-instruct (~1.8 GB) — the old Qwen-VL-Chat naming isn't in gallery; Qwen3-VL is the current real version of the same family
  • https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen3-VL-2B-Instruct-Q4_K_M.gguf
  • https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/mmproj-F16.gguf

Image Generation

Creates brand-new images from a text description ("a cat wearing a top hat"). The opposite of Vision Analysis.

SDXL Turbo

  • Good for: Fast 1-4 step 1024x1024 generation
  • Watch out for: Pushes 4GB VRAM to absolute maximum
  • LocalAI: local-ai run sd-1.5-ggml (~1.5 GB) — SDXL Turbo not in gallery; sd-1.5-ggml is the real small image-gen model that fits 4GB VRAM (SDXL-class models generally don't)
  • https://huggingface.co/second-state/stable-diffusion-v1-5-GGUF/resolve/main/stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf

Music Generation

Creates instrumental music tracks from a text description.

MusicGen (Small)

  • Good for: Generates instrumental tracks from text
  • Watch out for: Takes a long time for 10 seconds of audio
  • LocalAI: local-ai run acestep-cpp-turbo-4b (~7.6 GB) — real, current music generation model in the gallery
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-5Hz-lm-4B-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/Qwen3-Embedding-0.6B-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-v15-turbo-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/vae-BF16.gguf

MusicGen (Small/Medium)

  • Good for: Generates short instrumental music tracks from text descriptions
  • Watch out for: Takes a long time for even 10 seconds of audio
  • LocalAI: local-ai run acestep-cpp-turbo-4b (~7.6 GB) — real, current music generation model in the gallery
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-5Hz-lm-4B-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/Qwen3-Embedding-0.6B-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-v15-turbo-Q8_0.gguf
  • https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/vae-BF16.gguf

Structured Data / Office

Good at pulling structured information (tables, JSON, forms) out of messy text, or formatting output to fit a specific structure.

Command-R 8B (Q4)

  • Good for: Excellent at RAG (RAG (feeding it your own documents to answer from)) and formatting enterprise JSON
  • Watch out for: Rigid conversational style
  • Ollama: ollama pull command-r (~17.4 GB)
  • LocalAI: local-ai run command-r-v01:q1_s (~7.9 GB) — real quantized build, already applied
  • https://huggingface.co/dranger003/c4ai-command-r-v01-iMat.GGUF/resolve/main/ggml-c4ai-command-r-v01-iq1_s.gguf

Storytelling / Creative Writing

Tuned for fiction, roleplay, and narrative writing rather than factual accuracy or following instructions strictly.

Dolphin 2.9 (Llama 3 8B)

  • Good for: Uncensored (no built-in content filters — use responsibly) creative writing, raw sci-fi/thriller generation
  • Watch out for: Can produce overly graphic content unprompted
  • Ollama: ollama pull dolphin-llama3 (~4.3 GB)
  • LocalAI: local-ai run dolphin-2.9-llama3-8b (~4.6 GB)
  • https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b-gguf/resolve/main/dolphin-2.9-llama3-8b-q4_K_M.gguf

Gemma 2 9B (Q4)

  • Good for: Exceptional creative writing, deep character nuance (Sci-Fi, Thrillers)
  • Watch out for: Strict safety alignment (built-in safety behavior) can interrupt creative flow
  • Ollama: ollama pull gemma2:9b (~5.1 GB)

24 GB RAM

4 GB VRAM

Video Generation

Creates short video clips from a text description. Still an early, slow, low-resolution capability on consumer hardware.

Zeroscope v2 576w

  • Good for: Short (2-3 second) low-res video generation
  • Watch out for: Generates frame-by-frame; heavily relies on System RAM
  • LocalAI: local-ai run wan-2.1-t2v-1.3b-ggml (~7.3 GB) — Zeroscope not in gallery; Wan2.1 1.3B is a real, current small text-to-video model
  • https://huggingface.co/calcuis/wan-gguf/resolve/main/wan2.1_t2v_1.3b-q8_0.gguf
  • https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/vae/wan_2.1_vae.safetensors
  • https://huggingface.co/city96/umt5-xxl-encoder-gguf/resolve/main/umt5-xxl-encoder-Q8_0.gguf

Structured Data / Office

Good at pulling structured information (tables, JSON, forms) out of messy text, or formatting output to fit a specific structure.

Mistral Nemo 12B (Q3)

  • Good for: Massive context (128k) for processing large tabular text/CSVs
  • Watch out for: Very slow on 4GB VRAM
  • Ollama: ollama pull mistral-nemo (~6.6 GB)

Storytelling / Creative Writing

Tuned for fiction, roleplay, and narrative writing rather than factual accuracy or following instructions strictly.

Mythomax-L2 13B (Q2)

  • Good for: Exceptional multi-character roleplay
  • Watch out for: Extremely slow due to heavy system RAM usage
  • Ollama: ollama pull nous-hermes2 (~5.7 GB) — substituted for 'mythomax' (doesn't exist in Ollama's registry)
  • LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — MythoMax roleplay finetune not in gallery; Falcon3 3B for general storytelling/chat (won't match its uncensored roleplay style specifically)
  • https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf