AI Models

Every AI model Sponic can call — the local stack on ALPUCA's external SSD plus every API-hosted model wired into the apps. One page to answer "what should I use for X, and what does it cost?"

Summary — what to use for what

Pick the smallest, cheapest tool that hits the quality bar. Local is free at the margin but slower; cloud is faster and smarter but priced per token / per second / per image.

Local on ALPUCA (PortoSams2T)

All local models live on the always-connected Samsung T7 external SSD /Volumes/PortoSams2T/. ALPUCA's internal 245 GB drive does not hold any models. Three runtimes load them: Ollama (LLMs), HuggingFace transformers / diffusers (image gen, embeddings), whisper.cpp (audio).

Ollama LLMs — 107 GB

Stored at /Volumes/PortoSams2T/models/llm/ollama/ (legacy alias alpuca-offload/ollama/models still resolves). List with ollama list. Run with ollama run <name> or via the OpenAI-compatible HTTP API on http://alpuca:11434/v1/.

Whisper.cpp — speech-to-text, 5.2 GB

Stored at /Volumes/PortoSams2T/models/stt/whisper/ (legacy alias models/whisper still resolves — the live-translation server's hardcoded path keeps working) as ggml-*.bin files. Loaded directly by whisper.cpp / pywhispercpp consumers. Pick the smallest tier that meets accuracy needs — quality climbs steeply, but so does latency on CPU.

HuggingFace cache (active) — 12.5 GB

Stored at /Volumes/PortoSams2T/models/embeddings/huggingface/hub/ (legacy alias huggingface-cache/hub still resolves). These are the embedding models the photo-indexer venv loads at runtime.

HuggingFace cache (offload, image gen) — 31 GB

Stored at /Volumes/PortoSams2T/models/image-gen/huggingface/hub/ (legacy alias alpuca-offload/huggingface/hub still resolves). Reached via the ~/.cache/huggingface symlink so any HF library on ALPUCA finds them transparently.

Image-tools venvs (Python runtimes that load the above)

Stored at /Volumes/PortoSams2T/image-tools/venvs/. Activate then run; full READMEs in each venv dir.

• venvs/mflux/ — FLUX.1 image gen (Apple Silicon MLX port). Output to ~/mflux-output/.
• venvs/moondream/ — Moondream VLM for photo captioning.
• venvs/photo-indexer/ — Face recognition + Flask search API. Loads siglip2 + facenet-pytorch.

Cloud APIs

Every paid / hosted model the apps actually call. Bold rows are wired into production code; non-bold rows are accessible but not yet integrated.

Cloudflare Workers AI

Account wingsiebird (9cd3a280a54ce2a5b382602f0247b577). Catalog: developers.cloudflare.com/workers-ai/models. Pay-as-you-go, billed per million tokens.

Azure OpenAI

Subscription 785e237b-… (Microsoft for Startups Founders Hub Phase 1, $1k credit, expires 2026-07-31). Resource group sponic-ai. Service principal claude-code-sp in BW item 40b98339-3dce-4cec-9eaf-b43f00f43e80.

Google AI (Gemini) — the most-used cloud LLM in this stack

Key in env var GOOGLE_AI_API_KEY. BW item TODO — key currently lives in each app's .env.local. Gemini is doing more work in Sponic than any other cloud LLM — live translation, OCR, voice-note transcription, email classification, image generation. Three families in active use: 3.x (newest, live translation), 2.5 (production workhorse), 2.0 (legacy classifiers).

Anthropic (Claude)

Key in env var ANTHROPIC_API_KEY. Used by AI host, Vapi server, prompt-runner worker, and this Claude Code CLI.

Deepgram

Key in env var DEEPGRAM_API_KEY.

ElevenLabs Replaced by Gemini TTS

The AI dinner-host originally scaffolded around elevenlabs.TTS, but Sponic never provisioned an ElevenLabs account. On 2026-05-06 the TTS was swapped to gemini-3.1-flash-tts-preview (see Google AI section above) — reuses the existing Google AI key, supports inline audio tags, ~70 languages. The remaining ElevenLabs reference in apps/control/supabase/functions/vapi-server/index.ts:224 is just a config string passed to the Vapi platform (provider: "11labs"); Vapi handles its own ElevenLabs auth on their side — not used.

OpenRouter

One OpenAI-compatible endpoint (https://openrouter.ai/api/v1) that fronts ~300+ models from Anthropic, OpenAI, Google, Meta, Mistral, DeepSeek, Qwen, MiniMax, xAI/Grok, NVIDIA Nemotron, Cohere, Moonshot, and more — with one API key. Pricing is provider-cost + a small markup; many models have free tiers. This is the right place to reach for "I just want to try model X" without provisioning a new account.

NVIDIA build.nvidia.com (NIM)

NVIDIA also hosts a catalog of open-weight models directly behind an OpenAI-compatible API at integrate.api.nvidia.com/v1 — useful when you want NVIDIA-tuned variants (Nemotron) or want to avoid the OpenRouter middleman. Free dev tier, usage-based after.

Storage layout — executed 2026-05-06

Models used to be spread across four sibling directories on PortoSams2T (organic growth, not design). On 2026-05-06 they were consolidated into a single canonical models/ root, with non-model state moved to a sibling runtime/ root. Every old path was preserved as a back-compat symlink so existing consumers (live-translation server, mflux, photo-indexer, anything with hardcoded paths) keep working unchanged.

Current layout (post-consolidation)

/Volumes/PortoSams2T/models/      ← canonical root for every model blob
├── llm/
│   └── ollama/                   107 GB — Ollama LLMs
├── stt/
│   └── whisper/                  4.9 GB — whisper.cpp ggml files
├── embeddings/
│   └── huggingface/{hub,modules,xet}    13 GB — siglip2 + gme-Qwen2-VL
└── image-gen/
    └── huggingface/{hub,xet,token,…}    31 GB — FLUX.1-dev

/Volumes/PortoSams2T/runtime/     ← non-model state
├── image-tools/                  2.9 GB — mflux/moondream/photo-indexer Python venvs
└── moondream-indexer/             10 GB — photo embedding DB

/Volumes/PortoSams2T/             ← back-compat symlinks at every old path
├── models/whisper                  → models/stt/whisper
├── huggingface-cache/{hub,modules,xet}    → models/embeddings/huggingface/*
├── alpuca-offload/ollama/models    → models/llm/ollama
├── alpuca-offload/huggingface/{hub,xet,token,stored_tokens} → models/image-gen/huggingface/*
├── alpuca-offload/moondream-indexer → runtime/moondream-indexer
└── image-tools                     → runtime/image-tools

What broke / what didn't

• Ollama: nothing — ~/.ollama/models is a directory hard-link (same inode 1983236 as alpuca-offload/ollama/models). Moving the SSD-side directory entry to the new location preserved the inode, so the hard-link from home still resolves. Daemon restarted, all 13 models listed.
• HF caches: nothing — both old paths are now symlinks to the new locations. ~/.cache/huggingface still resolves through alpuca-offload/huggingface → new image-gen path.
• Whisper: nothing — the live-translation server's hardcoded /Volumes/PortoSams2T/models/whisper/... path still resolves via the symlink to models/stt/whisper.
• Python venvs (mflux, moondream, photo-indexer): nothing — the image-tools top-level path is still a valid entry (now a symlink).

How to refresh this doc

Run these on ALPUCA when local-model state changes; paste new totals into the tables above.

ssh alpuca@alpuca '
  ollama list
  du -sh /Volumes/PortoSams2T/models/llm/ollama
  du -sh /Volumes/PortoSams2T/models/embeddings/huggingface/hub/*
  du -sh /Volumes/PortoSams2T/models/image-gen/huggingface/hub/*
  ls -la /Volumes/PortoSams2T/models/stt/whisper/
'

For cloud entries, check each provider's pricing page and the relevant BW item under DevOps-sponicgarden:

infra/bin/bw-sponic search anthropic
infra/bin/bw-sponic search azure
infra/bin/bw-sponic search nvidia
infra/bin/bw-sponic search deepgram
infra/bin/bw-sponic search elevenlabs