Runs on your siliconOwned
A whole inference cluster — ~14 language models plus image, video, music and speech — on one desktop-class box. No model provider ever sees a prompt.
← all projects
project deep-dive
GaudeauAI
Every model — language, vision, voice, music, and code — runs locally on a single NVIDIA DGX Spark. No prompt, document, or memory is ever sent to a third-party model provider. Reachable from an iPhone, a Mac, and CarPlay over a private mesh.
The one exception, stated plainly: web search calls an external API (Brave) — deliberately isolated behind a server-side proxy so the assistant's sandbox can never even see the key. Everything else is inference on the box.
121 GB unified memory~14 language models120B-class models on one box128K assistant context5 generative studios~250 news items/day enriched8 Xcode targets · iOS · Mac · CarPlay848 backend tests green0 cloud LLM calls
At a glance · The premise
Owned. Private. Complete.
Reachable only by youPrivate
Devices connect over a private mesh, and every request passes through one authenticated entry point before anything answers.
A platform, not a chatbotComplete
Chat, a tool-using agent, agentic coding, generative studios, RAG project workspaces, a news-intelligence pipeline, voice, and CarPlay — one integrated system.
Live service status
| Service | Detail | Status |
|---|---|---|
| Primary model | Tier-1 · Qwen3 35B MoE · always-on | active |
| Model fleet | 14 models · on-demand | idle |
| Embeddings | RAG vectors · always-on | active |
| Speech | neural speech | active |
| Image & video | FLUX.2 · SDXL · video | active |
| Music | music generation | active |
| Agent | Nemo · isolated pod | active |
| Orchestrator | API & app backend | active |
Subsystem · The model fleet
Fourteen language models, plus a full generative stack — all local.
Each language model is its own llama.cpp service on a dedicated port. A memory-aware router loads them on demand, evicts idle ones, and keeps the assistant model permanently hot — treating 121 GB of shared memory as a live budget across LLMs, diffusion, video, music, and speech.
Language models
| Model | Role | Params · active | Quant · memory | Lifecycle |
|---|---|---|---|---|
| Qwen3.6-35B-A3B — Tier-1, the brain behind Nemo | Primary assistant & premium chat | 35B · 3B MoE | Q4_K_M · ~28 GB | always-on |
| Mistral-Small-3.2-24B (+ vision projector) | Workhorse — voice, translation, enrichment | 24B | Q4_K_M · ~21 GB | on-demand |
| Qwen3-Coder-Next 80B | Default coding · interactive & roadmap implementation | 80B MoE · active unknown | UD Q4_K_M · 57 GB @ 256K | on-demand · 5m idle |
| Qwen3.6-27B-MTP | Coding reasoning · roadmap planning & review | 27B dense | UD Q4_K_XL · 35 GB @ 256K | on-demand · 30m idle |
| Qwen3-30B-A3B | Deep Research / reasoning | 30B · 3B MoE | Q4_K_M · ~26 GB | on-demand |
| Qwen3-VL-30B-A3B | Vision — OCR, receipts, describe-image | 30B · 3B MoE | Q4_K_XL · ~22 GB | on-demand |
| Qwen3.5-122B-A10B — the largest LLM | Premium — maximum quality | 122B · 10B MoE | Q4_K_M · ~86 GB | manual |
| gpt-oss-120B | Open 120B alternative | 120B | MXFP4 · ~70 GB | manual |
| Gemma-3-27B-it | Mid-size general | 27B | Q4_K_M · ~20 GB | manual |
| Cydonia-24B | Creative / general | 24B | Q4_K_M · ~18 GB | manual |
| LFM2-8B-A1B | Scientific & factual specialist | 8B · 1B MoE | Q4_K_M · ~8 GB | on-demand |
| Gemma-3n-E4B | Quick answers | E4B | Q4_K_M · ~12 GB | on-demand |
| Gemma-3n-E2B | Educational / lightest | E2B | Q4_K_M · ~8 GB | on-demand |
| EmbeddingGemma-300M | RAG vectors — 768-dim embeddings | 300M | Q8_0 · ~1 GB | always-on |
| Gemma-3n-E2B — on-device (Apple MLX) | Offline fallback — runs on the iPhone | E2B | 4-bit · ~1.5 GB | on phone |
Model identity is verified by readlink, never filename — the Tier-1 path is a stable symlink so ~15 downstream references never churn on a model swap.
Generative media & voice
| Model | Capability | Runtime | Performance |
|---|---|---|---|
| FLUX.2-dev | Flagship text-to-image (fp8, 35.5 GB DiT + Mistral text encoder) | ComfyUI v0.27 | ~75s / 1024px |
| SDXL 1.0 (+ refiner, ControlNet, IP-Adapter) | Fast image · img2img · inpaint · pose-match | ComfyUI | ~14s / image |
| SUPIR-v0Q | Gold-standard photo restore & upscale (2–4×) | ComfyUI + KJNodes | 1024→2048 ~316s |
| LTX-2.3-22B | Fast video — t2v / i2v / HQ 2× upscale (≤201 frames) | ComfyUI | ~46 GB cold-load |
| Wan 2.2 TI2V-5B | Balanced text/image-to-video | ComfyUI | ~96s / test clip |
| HunyuanVideo 1.5 | Highest-quality 720p video (t2v / i2v) | ComfyUI | 1280×720 |
| ACE-Step 1.5 XL-SFT + 5Hz LM 4B | AI music — 5–600s, 320 kbps, lyrics, vocals, seeds | uvicorn service | elapsed time recorded per run |
| Kokoro-82M | Neural text-to-speech — 27 voices, streamed MP3 | FastAPI service | ~0.8s to first audio |
| Whisper.cpp (ggml-medium.en) | Speech-to-text — podcast/video transcription & ad-trim | DGX GPU | 45-min ep ~1–2 min |
ComfyUI runs pinned at v0.27 with restored DynamicVRAM (a one-line comfy-aimdo bump that ended a five-crash bring-up saga) and 14 SHA-pinned custom-node packs — FLUX.2 got ~3.4× faster in the upgrade. Voice-mode STT is separate: it runs on-device with Apple's Speech framework, never leaving the phone.
Subsystem · Inference & memory
Running a whole fleet on one box is a systems problem before it's an AI problem.
On unified memory the CPU and GPU draw from the same 121 GB pool — cgroup limits are cosmetic and torch never sees an OOM, so the kernel starves first. Everything that allocates has to coordinate through one shared budget. This is the layer that makes 120B-class models coexist without the box hard-resetting.
A cross-service memory ledgerAdmission control
One reservation surface both the LLM router and the image/video guard read and write, so a model cold-load and a FLUX.2 render can't each independently pass a free-memory check and then both allocate. Each side registers its in-flight loads and can ask the other to evict idle work. This is the fix that closed the box's first-ever hard crash.
An allocator-aware GPU reaperLive reclamation
After the v0.27 upgrade, model weights moved into GPU memory the container's cgroup can't see — docker stats read 2.7 GB while nvidia-smi showed the same process pinning 51 GB. The reaper now decides on max(RSS, per-PID GPU), so it's correct regardless of which allocator is in play. A 20-second headroom watchdog evicts the largest idle model before any allocation can fail.
Automatic model selectionRouting
Every chat message is scored for topic (coding, scientific, research, vision…) and complexity, then routed to the best-fit specialist — you just type. Coding always lands on the coder; a hard question can escalate to the 122B. Ported from the retired n8n 'Question Analyzer' into native JS.
MoE + quantization by designEconomics
The fleet is built on mixture-of-experts models so nominally huge models cost little active compute — a 122B model activates only 10B parameters per token. Aggressive quantization (Q4_K_M / UD-Q4 / MXFP4) plus statically pre-allocated KV cache means memory budgets are load-time and calibrated from real journal telemetry, not guesses.
The loaded state, not the declared stateModel Manager
A live control surface reads systemd, process/container, queue, GPU and RSS state across the LLM fleet, ComfyUI, ACE-Step, Whisper, Kokoro and embeddings. It shows what's actually resident — context, reserved or held memory, activity, idle time and errors — and lets you unload only idle, unprotected models through the same reclamation machinery as the automatic reaper.
Runtime footprint
- Single Node ESM server · ~130 modules
- core.db · 38 SQLite tables
- Separate vectors.db for embeddings
- 13 functional health canaries
- 11-step hash-verified deploys
- 44 days uptime before the first crash
Subsystem · Chat modes
Pick a mode. Press and hold to tune it.
The chat isn't one-size-fits-all. A row of mode pills sits above the message box — tap one and your message routes to the local model best suited to the job; press and hold it to open that mode's Options sheet and dial in length, depth, effort, and tone. No config file, no restart.
| Mode | Routes to | Press-and-hold tunes |
|---|---|---|
| Quick Response | A fast, lightweight model | Conciseness, response length, creativity |
| Coding | Two 256K specialists (Qwen3-Coder-Next 80B / Qwen3.6 27B MTP) | Model, reasoning depth, temperature, comments, explanation depth, output size |
| Deep Research | The analytical reasoning model | Research depth (Surface → Exhaustive), reasoning effort (Low → Max) |
| Premium | The flagship 35B — maximum quality | Creativity, response length |
| Voice Chat | Hands-free, spoken back | Creativity, response length |
| Translate | Mistral-Small 24B + a purpose-built translation prompt | Source/target language, formality, literal-vs-creative |
Every setting persists per mode and per conversation, and syncs across devices — so each mode remembers how you like it. Prefer to steer manually? Pin a specific model to a conversation from the model picker; otherwise each mode auto-selects the best fit.
Translate mode
Multi-language translation is built right into the chat — no separate app, no cloud service, nothing leaving the rack. Tap the Translate pill, press and hold for options, then type or paste; GaudeauAI returns the translation only — no commentary, no echoed source — preserving tone, formatting, and technical terms.
Auto-detect in, anything out32 languages
Leave the source on auto-detect or set it explicitly, then translate into any of 32 languages (English by default) — from Spanish, French and German to Japanese, Arabic, Hindi, Ukrainian, Swahili and Tamil.
Formality + approachDialed to taste
Set the register (Informal, Neutral, Formal) and how faithful the rendering stays (Literal, Natural, Creative) right from the Options sheet.
Runs on your own hardwareLocal · private
Translation runs on a local open-weight model — Mistral-Small-3.2 (24B) on the DGX — driven by a dedicated translation prompt. Every language pair stays on the box.
Subsystem · Nemo
Nemo — a genuine tool-using agent, not a stateless chatbot.
The primary assistant runs the openclaw agent framework inside an isolated Kubernetes pod, driven by the always-hot 35B model. It has a persistent identity, a durable transcript, a long-term memory that survives context overflow, a least-privilege tool sandbox, and outbound filtering — engineered end-to-end for prompt-injection resistance.
128K context--ctx 131072
Holds roughly a full novel of conversation before anything is summarized away — GQA + q8 KV cache keep that buffer at just 1.36 GB.
Rotation-with-recall memory
When a session gets long it's distilled into a digest that seeds the fresh session, and chunked + embedded so past exchanges can be recalled later by meaning.
Editable personalitySOUL · AGENTS
Character, self-image, and behavior contracts live in Markdown files re-read every turn — editable from the app, effective on the next message with no restart.
Least-privilege sandbox
The web-search key was pulled out of the agent's sandbox entirely (injected text could have cat'd it) and put behind a rate-limited server proxy. Injection can't exfiltrate what it can't see.
Outbound allowlist
The agent can only reach destinations on an explicit allowlist — a compromised agent can't phone home.
Reminders & recaps via chat
Ask in plain language and Nemo emits a hidden JSON block the server turns into a real scheduled reminder, or a background job that renders a spoken topic recap.
Drafts new skills
Nemo can draft a whole automation from a conversation — but it lands as an inert pending item; every activation gate stays behind your manual tap. It can suggest, never self-install.
Self-healing recovery
On a reboot or pod recreation Nemo comes back on its own — container auto-restart, preserved pod, and wiped workspace files restored from an hourly backup. A live 8-minute cold boot recovered with zero manual steps.
Subsystem · Forge (now standalone)
The agentic coder that grew up and moved out.
GaudeauAI incubated an agentic coding client — Forge Lite — that drove a real tool-use loop (read, write, edit, run, search, patch) against the box's local coding models, sandboxed in throwaway git worktrees with approve-before-apply diffs. It has since become its own native Apple app, Forge Pro, with its own server, provider choice, and iPhone/iPad clients.
Forge is now a separate product. Explore Forge Pro → — the standalone native Apple workspace for AI-assisted coding. Prefer the original? The first-generation prototype is archived: see Forge Lite →.
What carried forward: worktree isolation, review-first unified diffs, run controls (model, reasoning depth, temperature, tool-iteration and output ceilings), live context telemetry, autonomous roadmaps, and an admin-gated, audited execution model. Forge Pro rebuilds those ideas as a dedicated app with a server you control and coding providers you choose.
Subsystem · Media Workshops
Self-hosted generative studios — image, video, music, documents.
All diffusion runs through one containerized ComfyUI driven by a custom Node WebSocket bridge; music runs on a local ACE-Step server; documents stream from the model fleet. Every generation is background-safe — lock the phone or restart the server mid-render and the finished result still lands in the gallery.
ImageFLUX.2-dev fp8 · SDXL
Text, camera/library captures or one-to-many references feed img2img, masked inpainting and ControlNet pose-match workflows. Seeds pin back into the form; device-local prompt recents and model/resolution/step/guidance/timing stats keep the run reproducible. Save, share, copy, redo, enhance, reuse as a reference, or send to Nemo.
EnhanceSUPIR-v0Q · SDXL
Photorealistic restore-and-upscale from 1.5–4× (2× default) on any server-stored image — entirely on the DGX; pixels never round-trip through the phone.
VideoLTX 2.3 · Wan 2.2 · Hunyuan 1.5
All three families run text- or image-to-video; LTX adds an HQ tier with 2× spatial upscaling. Control prompt, negative prompt, resolution, frames, FPS, CFG and seed, then inspect the full run stats and save, share, redo or send the result to Nemo. Queued work can be removed and the active Comfy prompt interrupted without unloading warm model weights.
MusicACE-Step 1.5 XL-SFT · 5Hz LM 4B
Generate 5-second to 10-minute, 320 kbps tracks with title, genre, manual or generated lyrics, lyric-source attribution (Nemo / ACE-LX / Manual), vocal gender/language, duration, steps, CFG and a reusable seed. Each result retains the full setup plus elapsed time and model provenance.
Music playback + playlists
Tracks persist in server-side history, scrub in the Workshop and shared player, share through the system sheet, and drop into new or existing playlists for app and CarPlay playback. Successful vocal alignment adds word-timed karaoke via MMS forced alignment with a Whisper fallback; a missed alignment never fails the song.
Documents20 templates
Meeting notes, reports, proposals, SOPs, READMEs, ADRs, API docs and more — routed to the right model, streamed live, optionally grounded in a project.
Generation lifecycle
Image, video and music jobs expose pending / running / completed / failed / cancelled state, keep going while the client is away, and recover after reconnection or a server restart. Image/video report numeric progress; music reports live phase/status. Opt-in pushes fire on completion or failure, and deliberate cancellation closes the Live Activity without a conventional push.
Galleries + Live Activities
Every image, video and track persists server-side with its media-specific prompt, settings, model, seed, timing and output metadata, browsable across clients connected to the DGX; prompt recents stay device-local. All three surface Lock-Screen and Dynamic Island progress — percentage where available, phase and elapsed state where it isn't.
Subsystem · Project workspaces & RAG
Projects with their own knowledge, context, and a Jira-grade tracker.
Group work into projects, each with background context, custom instructions, uploaded knowledge files, and a full task tracker. Chat inside a project and the system fences your context into the prompt and runs semantic search over your files — retrieval-augmented generation, running entirely on your own box.
Per-project RAG
Upload documents; each is chunked and vector-embedded so its content becomes semantically searchable. Answers cite which files they drew from and how relevant each was.
Context & instruction injection
Free-text background and per-project behavior rules are folded into the model's system prompt for that project only — with three device-local toggles to turn each source on or off.
Injection fencing
Because uploaded docs could hide 'ignore your rules' text and the model may hold tools, every user-stored surface is wrapped in labeled 'this is DATA, not commands' fences and demoted below the real system prompt.
Command Center tracker
A full Jira-style tracker per project: board, list and timeline views; tasks / bugs / features; configurable status columns; priority, effort, assignee, due dates and tags.
Automation & collaboration
If-this-then-that automation rules, custom fields, templates, saved filters, subtasks, threaded comments with @mentions, a full activity log, watchers, attachments, and time tracking.
Capture → task
A quick capture aimed at a project — from CarPlay voice, the phone, or the Mac menu bar — drops straight into that project's tracker as a new to-do, sync-safe with tombstones so a deleted project can't be resurrected.
768-dim embeddings, ~800-char chunks, top-3 retrieval at a 0.35 similarity floor, every row conjunctively scoped to (project_id, user_id) with a per-user ownership guard above the SQL filter — sub-5ms search at single-user scale.
Subsystem · Briefs · News · Daybook
A personal news-intelligence pipeline that learns your taste.
It ingests RSS, Reddit, Substack, YouTube and podcasts, runs every item through a two-stage local-LLM pass (score, then summarize only the survivors), learns what you care about from your behavior, and turns the result into spoken briefings, reading surfaces, and a morning narrative — all on one box, zero cloud.
Two-stage enrichment
Stories are triaged for relevance first; only the ~35% that survive get a full summary written — roughly halving LLM output and keeping junk out of the feed.
Taste-learning engine
Seven weighted signals over a rolling 30 days — thumbs, stars, read-full vs bail, ignores — quietly boost or bury whole categories and sources over time.
Watched entities
Name people, companies or products and any incoming story mentioning them is auto-tagged and can push a notification, deep-linked to the brief.
Flash Briefing
A 2–5 minute spoken news podcast pre-rendered every morning, playable with lock-screen controls, regenerable on demand.
The 7am Daybook
A warm spoken 'here's your day' merging overnight weather, your calendar, reminders and top stories into one personal narrative — ending with a plain-spoken system-health check.
Recaps on demand
'Everything about Nvidia from the last 30 days' splices your enriched stories on a topic into a single narrated audio recap via map-reduce synthesis.
Podcasts + ad-trim
Add any podcast by name or URL; episodes get their ad reads transcribed (Whisper) and spliced out automatically, and YouTube items skip sponsors via SponsorBlock.
Drop-a-link ingest
Paste any article URL and it's fetched, cleaned to reader view, summarized, tagged, and saved — behind an SSRF guard and rate limit.
HEARD / READ / SEEN
Every brief is stamped for whether you heard, read, or just saw it, and audio playback position resumes across iPhone, iPad, CarPlay and Mac — the server is the single source of truth.
Subsystem · Clients
One Swift codebase — iPhone, iPad, Mac, and CarPlay.
A single SwiftUI codebase across eight Xcode targets ships as three first-class clients of the DGX backend, plus widgets, Live Activities, Siri intents, and system extensions — sharing view and service code behind #if os() shims.
iPhone & iPad
Five locked tabs
Today, Chats (model-routed with mode toggles), Nemo, Playlists, and Tools — plus a universal Settings gear. Deep links survive lazy tab mounting via a stash-drain router.
Voice Mode
Tap the orb and talk: on-device speech recognition → the 35B model streams a reply → gapless Kokoro speech back, first audio in ~0.5s. Interrupt it just by talking.
On-device offline LLM
When the DGX is unreachable the phone answers from a 4-bit Gemma model running on-device via Apple MLX — with an offline workspace and queued mutations.
macOS · native app
11-section sidebar
A sandboxed, hardened-runtime Mac app sharing ~55 source files with iOS: Today, Chats, Projects, Notes, Nemo, Briefs, Playlists, Workshops, Skills, Health, Settings.
Command panel + ⌘K search
A Spotlight-style global-hotkey panel for quick capture, clipboard AI and drag-in image OCR, plus a ⌘K universal search across briefs, notes, conversations and files.
Full parity
Streaming chat, briefs reader, media workshops with cancel/abort, and the voice orb — all sharing server state with the phone.
CarPlay · Widgets · Voice engineering
Full CarPlay app
A five-tab Driving-Task interface — Today, Playlists, Nemo (voice-first), Ideas (capture), More — with a full-screen player, motion-aware read-only-while-driving, and graceful offline degradation.
Audio-exclusivity arbiter
One coordinator guarantees only one thing ever makes sound at a time across 4+ engines — a Nemo reply, a brief, a flash briefing and voice mode can never overlap, in the car or on the phone.
Barge-in detection
While the assistant speaks, a per-turn auto-calibrated VAD listens for you to start talking and stops playback instantly — no hardware echo cancellation needed.
8 widgets + Live Activities
Home- and Lock-Screen widgets (tasks, projects, briefs, quick-capture, next-skill countdown, Nemo, smart home, quick actions), Dynamic Island generation progress, and 6 Siri App Intents.
Share & Notification extensions
Share a URL/text/image into the app from anywhere (images route to OCR or receipt extraction); pushes are enriched in-flight with the real sender and decrypted preview.
Offline-first sync
Pair once with a code, then conversations, projects, captures, notes and settings sync across devices with field-level last-write-wins merges and conflict handling.
Subsystem · Security · Skills · Privacy
A small attack surface, on purpose — and a threat model, not an afterthought.
Every request passes through one authenticated entry point, and the box is reachable only over a private mesh. On top runs a permission-gated Skills system where the AI can draft automations but nothing runs until the owner reviews the full source and explicitly approves exactly what was reviewed.
One gate, mesh-onlyPerimeter
A single entry point authenticates every call before any handler runs, so the API can't be mapped from outside. The box is reachable only over a private mesh, and the model servers behind it are never exposed directly.
A privilege ladder for untrusted textInjection doctrine
Before trusting any stored or fetched text, the surface is placed on a ladder of how much power the receiving model holds — tool-enabled chat at the top, then tool-less workshops, then read-aloud. Secrets never sit where a tool-holding model could read them.
Four gates before AI-drafted code executesSkills — approve-before-run
Drafted skills land inert. Approving requires all four: you review the complete source on the phone; the approval must match exactly what you reviewed; dangerous permissions demand a typed confirmation; and a static analyzer must find no undeclared capabilities. Rollback re-checks the code so a tampered skill can't be trusted.
Restore-verified backups + disaster runbookResilience
Nightly backups capture the whole system, keep an off-site copy, and keep secrets out of it. A disaster runbook covers the failure scenarios that matter, and backups have restored deleted media byte-identical.
Per-user data and vector isolation · ~37 permission-gated skills with sick-machine health, a real-cron scheduler, an autonomy dial, and semantic search.
By the numbers
Telemetry.
121 GB
Unified memory
14
Language models
122B
Largest model
128K
Assistant context
~250
News items/day
8
Xcode targets
38
SQLite tables
13
Health canaries
848
Backend tests
~2 yr
In development
1
External API · sandboxed
0
Cloud LLM calls
How it's built
Built like a product. Run for one.
A single-tenant deployment architected to multi-user-correct standards, shipped with the rigor of a system that has to stay up — because it does.
Zero-assumption
Docs and comments are leads, not truth. Every claim is verified against the live system before it ships.
Verified deploys
Hash-checked releases, journal boot-scans, and a live probe on every change. Green tests are the floor, not the finish line.
Adversarial review
Changes are reviewed to refute, not to rubber-stamp — the way a real bug gets found before a user does.
Self-hosted personal AI · iOS ⇄ DGX Spark ⇄ Nemo · every token generated on owned hardware.