KasprakStudio
Start a project
← Home
Personal AI assistant · local

Jarvis

A personal AI assistant that lives on its owner’s computer, not in someone else’s cloud. Built for a very demanding client.

Local modelsClaude on a budgetRisky actions need a yes
Watch the film →How it works

It talks, runs the computer, TV and soundbar, keeps track of projects, remembers what matters and comes up with ideas on its own. It thinks with local models on its own GPU and calls Claude only when a task is truly hard.

Jarvis in numbers
1,058commits in 6 weeks of development
~108,000lines of Python
410test files, thousands of checks
93core tools, each with a risk level and audit
7 + 3local model roles and 3 Claude tiers
32 / 32on the LongMemEval long-memory test
97.5%tool selection accuracy
24project windows around the sphere
21rounds of independent security review
Core principles
One core, many doors

Every action goes through one MCP core. Voice, Telegram, web chat, phone, TV and the Claude app are just entrances to it. So Jarvis is equally smart and equally careful wherever a command comes from.

Local by default

The main model, gpt-oss-20b, runs on its own GPU. Helper models handle vision and memory. The cloud is used only to escalate, on a strict budget, and private data never leaves the computer.

Security is a requirement

The model is treated as untrusted. Every risky action waits for confirmation in a separate channel: a Telegram button, a fingerprint on the phone or a dialog on the PC.

Promo film

The sphere is rendered by the real hologram engine, not drawn.

Jarvis’s memory

Memory as a neural network

Notes, documents, projects and entities are glowing nodes and links. White nodes are shared by several projects, bright lines connect different projects. It is the real structure, just without names on screen.

  • Keyword and semantic search at once, finds past conversations
  • “Hot” and “cold” facts, topic summaries, an owner profile
  • At night, without the cloud, it tidies its memory and reviews its mistakes
  • “Forget this” really deletes the fact
A brain with several layers

Fast things run locally and instantly, hard ones climb the ladder.

  1. Reflexesrules and semantic lookup, no large model
    instant
  2. Fast loopgpt-oss-20b on its own GPU calls the tools
    local
  3. Reviewa second look before acting
    local
  4. Long tasksa plan, steps, questions to the owner
    local
  5. Claude Haiku · Sonnetconversation and planning — on a budget
    cloud, quotas
  6. Claude Opusthe final solver for the hardest cases
    cloud, quotas
Everything Jarvis can do

Voice and conversation

  • Whisper speech recognition on the GPU; the reply starts while still being written
  • Jarvis’s own voice, a cache of ready phrases, end-of-turn detection
  • “Jarvis, stop” works instantly, even when the brain is busy
  • Character: restrained, with a light sense of humour

Jarvis everywhere

  • Telegram bot: chat, confirmation buttons, digests
  • Web chat: up to 10 files, model choice, long tasks, canvas
  • Android app: push-to-talk, the sphere, tasks, fingerprint and emergency stop
  • TV box remote: talk through its microphone

Computer and home

  • Apps, files, clipboard, screenshots, media, web search
  • TV box, TV and soundbar over HDMI-CEC
  • PC sleep and wake-on-LAN, safe start after a power cut
  • Power watchdog: outage → pause, unload models, sleep

Hologram and control center

  • A transparent 3D sphere over the windows: hundreds of golden threads, a star core
  • “Listening”, “thinking”, “speaking” states; up to 24 project windows
  • Video messages: advice as a clip with a talking sphere
  • Dashboard, daily feed, memory, schedules, “Radar and advice”

Proactive

  • Morning and evening digests, reminders, weather
  • Opus advice on projects — ratings teach Jarvis
  • “Radar”: dependencies, releases, vulnerabilities, GitHub finds
  • Sensor rules: “disk almost full”, “rain soon”

Agents that write code

  • A “writer → coder → review” pipeline
  • Offline sandbox: imports, linter, tests — only verified code lands
  • JarvisBench: a model is replaced only when it measures better
  • Nightly prompt optimisation (GEPA) and fine-tuning data

Project hub and games

  • One GPU for many projects: a shared priority queue
  • Heavy jobs at night, a report in the morning
  • Game mode: during a match Jarvis switches off completely
  • After the game it asks whether to restore everything

Knowledge and content

  • Obsidian: notes from the phone, semantic search, old ideas resurface
  • “Read later”: article summaries overnight
  • YouTube transcripts, photo and scan recognition, email drafts
  • Studio: “make a cartoon” → storyboard → approval → video
Risky actions only on the owner’s “yes”

The model is treated as untrusted: any text may turn out to be an injection.

Out-of-band confirmation

a Telegram button, a fingerprint on the phone or a dialog on the PC; single-use and bound to exact arguments

Audit of everything

a log without secrets; every source has its own token and risk ceiling

Kill switch

hotkey, voice or button; lifted only at the PC

Hard bans

government sites, banks, payment pages — forbidden in code

Game mode

during a match Jarvis is fully off and never touches the anti-cheat

Reliability

watchdog, self-check, daily backup, dependency and leak scans

What’s next

The roadmap: part of it is already written and waiting to be switched on.

Soon

  • Claude on the phone controls the PC through a secure tunnel
  • Jarvis TV: a voice over the film, with ducking
  • Health from the watch — local only
  • Its own “Jarvis” wake word in the owner’s voice
  • Sound: soundbar at home, phone outside
  • Remote desktop with approval of every connection

Later

  • Self-learning from its own mistakes (QLoRA, DPO)
  • Agents pick tasks from the plan on their own
  • Local vision: PDFs, photos, scans
  • A secret vault that never goes to the cloud
  • Morning TV briefing and bedtime stories
  • Brain 2: experience and link ranking
Technology
  • Python 3.12
  • MCP SDK
  • llama.cpp
  • Ollama
  • Whisper
  • gpt-oss-20b
  • Gemma
  • EmbeddingGemma
  • Claude Haiku · Sonnet · Opus
  • three.js
  • Electron
  • Kotlin · Android TV
  • SQLite FTS5
  • Vector search
How we build it

AI agents write and test the code, separate Opus-level agents review architecture and security, and every decision is logged. Hardware probe first, tool second. Every tool has a schema, a risk level, an audit trail and a test.

Want a Jarvis of your own?

We will build a personal AI assistant for your home, office or business — with your devices, data and rules. Local, private, no compromises.

Write to us →