Mistral Small 3.2 24B Instruct

LicenceFACT

apache-2.0

Fresh·Retrieved 25 Aug 2026

ParametersFACT

24.0B

Fresh·Retrieved 25 Aug 2026

Context lengthFACT

not returned by the registry

Evidence not verified

Downloads (30 days)FACT

266,292

Fresh·Retrieved 25 Aug 2026

LikesFACT

603

Fresh·Retrieved 25 Aug 2026

GatedFACT

no

Fresh·Retrieved 25 Aug 2026

Peak VRAM for serving this model, computed by lib/deployment/hardware.ts from weights + KV cache + 15% runtime overhead. Inputs: 24.0B parameters, 8,192 tokens of context (assumed — none fetched yet), 4 concurrent requests (assumed).

• On-premise single 48 GB GPU server

  GPU

  NVIDIA L40S 48 GB (rack) or NVIDIA RTX 6000 Ada 48 GB (office workstation)

  VRAM

  48 GB

  System RAM

  128 GB

  Storage

  4000 GB

  CPU

  24–32-core x86 server CPU (AMD EPYC or Intel Xeon Scalable)

  Form factor

  Rack server

  Indicative costUS$12,000 – US$22,000

  indicative build cost for the complete machine, USD, Aug 2026 — verify with a local supplier

  The step up when answer quality matters more than price: a 32B model at 4-bit with generous KV cache, or a 14B model at fp16 with a long context. ECC memory and a standard 2U chassis mean it belongs in a rack or comms room rather than under a desk.

  Sizing basis

  • Weights: 24.0B parameters × 2 bytes per parameter (fp16) = 48.02 GB.
  • No model configuration was supplied, so the KV cache uses the size heuristic for this band: 0.262 MB per token, from Qwen2.5-32B-Instruct (64 × 8 × 128).
  • KV cache: 0.262 MB per token × 8,192 tokens of context × 4 concurrent requests = 8.59 GB, sized for every user holding a full context at once.
  • Overhead: 15% of weights plus KV cache for the runtime, activations and memory fragmentation = 8.49 GB.
  • GB means 10⁹ bytes, matching how GPU memory is advertised.
  • KV cache held at fp16 (2 bytes per element); quantising weights does not by itself quantise the cache.
  • Peak figure, not average: prefix caching and paged attention usually keep real usage lower, while speculative decoding, CUDA graphs and a second resident model push it higher.
  • "Concurrent" counts requests generated at the same instant, not people: 4 simultaneous requests is the worst case this estimate is built on — measure yours before buying.

• Apple Silicon workstation (unified memory)

  GPU

  Apple M4 Max (up to 128 GB unified) or M3 Ultra (up to 256 GB unified)

  VRAM

  96 GB

  System RAM

  128 GB

  Storage

  2000 GB

  CPU

  Apple M4 Max / M3 Ultra

  Form factor

  Workstation

  Indicative costUS$4,500 – US$12,000

  indicative build cost for the complete machine, USD, Aug 2026 — verify with a local supplier

  Quiet, standard-socket power draw, and a large memory pool — about 75% of installed unified memory is addressable by the GPU, which is where the 96 GB figure comes from on a 128 GB M4 Max. Memory bandwidth, not capacity, sets the ceiling: excellent for a pilot, a partner workstation or a single heavy user, weak for 40 people at once. Runs Ollama, llama.cpp and MLX; does not run CUDA builds of vLLM.

  Sizing basis

  • Weights: 24.0B parameters × 2 bytes per parameter (fp16) = 48.02 GB.
  • No model configuration was supplied, so the KV cache uses the size heuristic for this band: 0.262 MB per token, from Qwen2.5-32B-Instruct (64 × 8 × 128).
  • KV cache: 0.262 MB per token × 8,192 tokens of context × 4 concurrent requests = 8.59 GB, sized for every user holding a full context at once.
  • Overhead: 15% of weights plus KV cache for the runtime, activations and memory fragmentation = 8.49 GB.
  • GB means 10⁹ bytes, matching how GPU memory is advertised.
  • KV cache held at fp16 (2 bytes per element); quantising weights does not by itself quantise the cache.
  • Peak figure, not average: prefix caching and paged attention usually keep real usage lower, while speculative decoding, CUDA graphs and a second resident model push it higher.
  • "Concurrent" counts requests generated at the same instant, not people: 4 simultaneous requests is the worst case this estimate is built on — measure yours before buying.

No hosted provider has been fetched for this model. OpenRouter’s catalogue is read by pnpm sync; until it runs, this is empty rather than assumed.

• Self hostedInvoice capture and extractionInvoices arrive by email or watched folder, Docling converts them (OCR included), a local vision-capable model fills a JSON schema under constrained decoding, code checks the arithmetic and the supplier against your master data, and n8n posts the clean ones while routing the rest to a person.

1. 01–07
   Tier 4·official repository / model cardHugging Face Hub

   Hugging Face model card data — mistralai/Mistral-Small-3.2-24B-Instruct-2506

   https://huggingface.co/api/models/mistralai/Mistral-Small-3.2-24B-Instruct-2506

   Fresh·Retrieved 25 Aug 2026sha256:0cddc49c6103

   7 records01 license ·02 gated ·03 lastModified ·04 downloads ·05 total ·06 likes ·07 library name