Memtrix

A self-hosted, privacy-first personal AI agent with persistent memory and agentic tool use. Not a chatbot - an agent.

Memtrix runs entirely on your hardware. It communicates through the Matrix protocol, remembers every conversation through semantic search, executes tools autonomously, and evolves its personality based on your interactions.

What is Memtrix?

Memtrix is a personal AI agent that lives in a Docker container on your machine. You chat with it through any Matrix client (Element, FluffyChat, etc.) and it can:

• Search the web and fetch URLs
• Manage files, clone repos, download content
• Remember everything - searchable conversation history, semantic search, and a reasoning-memory layer that distills durable facts about you
• Create specialist sub-agents with their own identity and memory
• Update its own personality and knowledge of you over time

An optional Web Control Panel at localhost:8800 lets you configure everything, manage secrets, and browse memory from your browser.

Who is it for?

Developers and power users who want a personal AI assistant they control completely - no cloud dependency, no data leaving your machine, no subscriptions.

What makes it different?

• Self-hosted - LLM, homeserver, search engine, vector DB - everything on your hardware
• Multi-provider - local models via Ollama, 200+ cloud models via OpenRouter, or any OpenAI-compatible endpoint
• Persistent memory - searchable conversation-history RAG plus a reasoning-memory layer (auto-curated peer cards & conclusions), all on-device
• Agentic - iterative reasoning loop with 35 auto-discovered tools, parallel tool execution, and schema-validated calls
• Multi-agent - create specialist sub-agents that can consult each other
• Web Control Panel - configure, test connections, manage secrets, and curate memory from the browser
• Security hardened - non-root, read-only filesystem, all capabilities dropped

What you need

Quick start

# Clone and enter the project
git clone https://github.com/nnxmms/Memtrix.git && cd memtrix

# Create directories, build image, start Conduit
./setup.sh

# Interactive wizard - configure LLM, model, channel
./onboard.sh

# Launch everything
docker compose up -d

Then open Element → connect to http://localhost:6167 → log in → invite @memtrix:memtrix.local to a room.

Features

Everything Memtrix brings to the table - from persistent memory to multi-agent orchestration.

Fully Self-Hosted

Every component runs on your hardware. The LLM (via Ollama or OpenRouter), the Matrix homeserver (Conduit), the search engine (SearXNG), the vector store (the chroma service) - nothing phones home.

Persistent Memory

Memtrix layers two complementary memory systems:

• Conversation Memory (sessions/) - raw conversation transcripts, chunked and searchable via on-device RAG embeddings
• Reasoning Memory - a background deriver distills conversations into typed conclusions and an auto-curated profile card (USER.md)

Semantic search is powered by nomic-embed-text-v1.5 running entirely on-device. No external API calls. See Reasoning Memory.

Web Control Panel

A local web app at localhost:8800 for running Memtrix without editing JSON: a validated config editor, one-click connection tests, secret management, full memory administration, and safe restarts with live progress. See Web Control Panel.

Agentic Tool System

40 built-in tools auto-discovered at startup. The orchestrator runs an iterative reasoning loop (up to 25 rounds, configurable) where the LLM can call tools, observe results, and continue reasoning. Independent read-only tool calls run in parallel (up to 8 at once), every call is validated against the tool's JSON schema before it runs, and provider calls retry transient failures with exponential backoff. New tools are just Python files dropped into src/tools/. See The Agentic Loop.

Self-Documenting

Memtrix can research its own documentation. The search_docs and ask_docs tools let the agent - and every sub-agent - answer "how does Memtrix work?" questions from the bundled docs, with citations. The docs are parsed into a vector-indexed documentation collection at startup and refreshed automatically. See Tools Reference.

Multi-Provider LLMs

Run local models via Ollama (Llama, Mistral, Gemma, etc.), tap 200+ cloud models through OpenRouter (OpenAI, Anthropic, Google), or point at any OpenAI-compatible endpoint - llama.cpp, vLLM, LM Studio, an OpenAI-shim, a self-hosted gateway, or OpenAI itself. Configure multiple providers and switch models per-agent. The control panel can discover the models a provider exposes so you pick from a list instead of typing names.

Evolving Persona

Memtrix's identity is defined by markdown files. It freely edits SOUL.md and BEHAVIOR.md as it learns your preferences, while USER.md is auto-curated by the reasoning-memory deriver. Its personality genuinely grows with every conversation.

Multi-Agent System

Create specialist sub-agents with their own identity, memory, Matrix presence, and workspace. Agents communicate via ask_agent - the main agent can consult sub-agents, and sub-agents can consult each other or the main agent.

SSH Remote Administration

Memtrix can act as a sysadmin on your remote hosts. It generates its own SSH key, registers hosts, and opens a persistent interactive session it works inside across many commands (so cd and environment changes persist), then closes it. Key-only auth, trust-on-first-use host keys, confirmation for destructive commands, and in-memory-only sudo passwords. See SSH Remote Admin.

Self-Authored Skills

Memtrix builds its own skills - short, reusable task workflows it writes for itself so it handles recurring work better over time. After finishing a non-trivial task (5+ tool calls, error recovery, a user correction, or a non-obvious workflow), it captures the approach as generalized steps using the skill_manage tool - same model, no second reasoner. At the start of every turn it sees a catalog of all its skills (each as name: description) and decides for itself which, if any, fits the task, then loads that skill's full instructions on demand and follows them - progressive disclosure, no embeddings or vector search. Instructions only, no separate code execution; each agent keeps its own isolated skill store. See Skills.

Matrix Chat Protocol

Communicates through the Matrix protocol via a local Conduit homeserver (or any external homeserver). Use Element, FluffyChat, or any Matrix client. Each room maintains its own conversation session, and Memtrix shows a native typing indicator while it works on a reply.

Local Voice Notes

Send a voice message and Memtrix transcribes it on-device with a local faster-whisper model, then answers as if you had typed it. No audio leaves your machine. Off by default - see Voice & Transcription.

Email Access

Connect a mailbox and Memtrix can read it over IMAP and send over SMTP. It can stay passive (only checking when asked) or, with reactive mail on, watch the inbox in the background and act on genuinely new messages the moment they arrive - triaging and only interrupting you when something matters. A trusted-sender allowlist can restrict it to mail from specific addresses, and every message is screened for prompt injection. Off by default - see Email & Mail.

Security Hardened

Defense-in-depth: non-root container, read-only filesystem, all capabilities dropped, no shell access for the LLM, SSRF protection, human-in-the-loop for destructive operations, path traversal prevention, and prompt injection mitigation.

Private Web Search

Built-in SearXNG instance for privacy-respecting web searches. Fetch, read, and summarize any URL - all self-hosted, no tracking.

Message Reactions

Memtrix can react to your messages with emoji in Matrix - just like a human would. The LLM decides when and what to react with naturally.

Architecture

How the pieces fit together - every component runs locally in Docker.

     ┌──────────────────┐        ┌────────────────────────┐
     │  Element Desktop │        │   Web Browser         │
     │ (Matrix Client)  │        │ localhost:8800 (Panel)│
     └────────┬─────────┘        └────────────┬──────────┘
              │                               │
┌─────────────┼───────────────────────────────┼──────────────────┐
│  Docker Compose                             │                  │
│             │                               │                  │
│  ┌──────────┴──────┐   ┌──────────────┐   ┌─┴────────────┐     │
│  │    Conduit      │◄─►│   Memtrix    │◄─►│  web (panel) │     │
│  │ (Matrix Server) │   │   (Agent)    │   │  FastAPI+SPA │     │
│  └─────────────────┘   └──┬────────┬──┘   └──────────────┘     │
│                           │        │                           │
│         Sub-Agents ◄──────┤        ├──────► SearXNG (search)   │
│      (background threads) │        │                           │
│                           │        └──────► chroma (vectors)   │
│                           │                  shared store      │
└───────────────────────────┼────────────────────────────────────┘
                            │
                            ▼
        Ollama (local LLM)  /  OpenRouter (cloud LLMs)

Components

Tech Stack

Message Flow

1. You send a message in Element (or any Matrix client)
2. The homeserver (Conduit or external) delivers it to Memtrix via matrix-nio
3. Memtrix's orchestrator builds the system prompt (injecting persona & memory files)
4. Relevant long-term memory is recalled (conversation-history RAG + reasoning-memory cards) and injected as context
5. The LLM is called with conversation history + tool schemas
6. If the LLM requests tools → tools execute (in parallel where safe) → results returned → loop continues (up to max_iterations, default 25)
7. Final response is sent back through the homeserver to your Matrix client
8. After the reply, the reasoning-memory deriver runs in the background to update peer cards and conclusions

Onboarding

A detailed walkthrough of the interactive setup wizard that configures your Memtrix instance.

How it works

The onboarding wizard (./onboard.sh) runs the Python onboarding module inside a Docker container connected to the Conduit network. It uses Rich for a polished terminal UI.

Step 1: Name your agent

Choose a name for your main agent. This name is used for:

• Matrix bot username (e.g. @memtrix:memtrix.local)
• Display name in Matrix rooms
• System prompt identity
• Sub-agent naming conventions
• Persona file templates

Default is "Memtrix" - but you can name it anything.

Step 2: Configure a provider

Providers are dynamically discovered from src/providers/. Built-in options:

Ollama (local)

For running models on your own hardware.

• Required: base_url - URL of your Ollama instance (e.g. http://host.docker.internal:11434)
• Make sure Ollama is running and has a model pulled (ollama pull llama3)

OpenRouter (cloud)

For accessing 200+ cloud models.

• Required: api_key - your OpenRouter API key
• Get one at openrouter.ai/keys

You can configure multiple providers - the wizard will ask if you want to add more.

Step 3: Set up a model

Select a provider, enter the model name (e.g. llama3, anthropic/claude-sonnet-4-20250514), and give it an instance name for reference.

Step 4: Configure the channel

Choose between Matrix (recommended) or CLI. For Matrix, you then pick the homeserver:

Bundled Conduit (default)

The wizard talks to the local Conduit homeserver and automatically:

1. Registers an admin account on Conduit
2. Registers the bot account (e.g. @memtrix:memtrix.local)
3. Registers your user account
4. Sets display names
5. Collects the bot access token

Passwords are generated with Python's secrets module (cryptographically secure, 24 characters).

External homeserver

Already have a Matrix account (e.g. on matrix.org or your own server)? Point Memtrix at any homeserver instead of Conduit:

• Homeserver URL - e.g. https://matrix.org
• Bot user ID & access token - an existing account Memtrix logs in as
• Your user ID - so the bot knows who to talk to

What onboarding produces

• data/config.json - full configuration with all providers, models, channels
• .env - all secrets (API keys, access tokens, registration token)
• workspace/AGENT.md - system prompt updated with your agent's name

First Conversation

What to expect when you send your first message - and how Memtrix learns about you.

Starting a chat

After setup, open Element and invite the bot to a room. Memtrix auto-joins room invites. Each room gets its own independent conversation session - multiple rooms mean multiple contexts.

What happens behind the scenes

When you send a message, Memtrix:

1. Builds the system prompt - injects SOUL.md, BEHAVIOR.md, and USER.md into the AGENT.md template
2. Recalls memory - silently pulls relevant context from past conversations and the reasoning-memory store (peer cards & conclusions)
3. Calls the LLM with conversation history and all available tool schemas (40 built-in, minus any gated off)
4. Executes tools if the LLM requests them (web search, memory lookup, etc.)
5. Responds with the final answer
6. Learns in the background - after replying, the reasoning-memory deriver updates the auto-curated profile card (USER.md) and records conclusions, while the raw conversation transcript is saved and embedded for future recall

Verbose & reasoning modes

Want to see what's happening under the hood?

• /verbose on - shows tool call notifications in real-time
• /reasoning on - shows the LLM's thinking process

Both are off by default. Turn them on to watch Memtrix's thought process.

First conversation tips

• Introduce yourself - Memtrix will save your name and details to USER.md
• Tell it about your preferences - it updates BEHAVIOR.md if you correct its style
• Ask it to search the web - confirms tools are working
• Try /help to see available commands
• The more you talk, the better it gets - memory and persona evolve continuously

Sub-Agents

Create specialist agents with their own identity, memory, and Matrix presence. Agents can consult each other autonomously.

What sub-agents get

Creating a sub-agent

Ask Memtrix to create one. It needs a real human name:

You:    Create me a cooking expert. Call him Dennis.

Memtrix: [confirm] Create a new sub-agent?

         Name: Dennis
         Expertise: Cooking and recipe specialist

         Allow? (yes/no)

You:    yes

Memtrix: Dennis is ready! His Matrix user is @dennis:memtrix.local.
         Invite him to a room to start chatting.

From the Web Control Panel

You can also create a sub-agent from the Sub-Agents page of the control panel - no chat required. Enter the agent's name, its area of expertise, and the model, and Memtrix provisions everything: it registers a fresh Matrix account automatically on the bundled local homeserver, or - when you run against an external homeserver - asks for the user ID and access token of an account you pre-created. The panel detects which case applies and shows the right fields. The new agent comes online after the next restart. The page also lists your existing sub-agents and lets you delete them (workspace, memory, and sessions are cleaned up; the Matrix account stays on the homeserver). Both the panel and the chat tool share the same provisioning path, so either way you get a complete, bootable agent.

Chatting with a sub-agent

Invite the agent's Matrix user (e.g. @dennis:memtrix.local) to a room and chat like you would with Memtrix. Dennis has his own memory - he'll remember your conversations and preferences independently.

Inter-agent communication

Agents can consult each other using the ask_agent tool:

• Main agent → sub-agent
• Sub-agent → main agent
• Sub-agent → sub-agent

You:    I'm planning a dinner party. Ask Dennis for a menu.

Memtrix: (internally uses ask_agent to consult Dennis)

         Dennis suggests a three-course menu: roasted tomato soup
         to start, herb-crusted salmon as the main, and a lemon
         tart for dessert.

Safety guards

• Depth limit: 2 hops maximum (prevents infinite recursion)
• Deadlock prevention: non-blocking 5-second lock timeout per agent
• Isolated sessions: inter-agent conversations use dedicated sessions, not user-facing history
• No human-in-the-loop: destructive operations are denied during inter-agent calls
• Context injection: the target agent sees recent user conversation for context (capped at 10 pairs / 4000 chars)

Managing sub-agents

• list_agents - see all registered sub-agents and their status
• delete_agent - permanently remove a sub-agent (workspace, memory, sessions all cleaned up)
• ask_agent - query another agent by name
• spawn_worker - hand a self-contained task to an ephemeral background worker that runs without blocking the conversation; the result is delivered back automatically when it finishes

Memory exchange

After an inter-agent call, a summary of the exchange is queued for the target agent and surfaced in its user-facing conversation on its next turn. If Agent B asks Agent A something, A can later tell the user what B asked and what it answered - even if A had never spoken to the user before (a brand-new sub-agent) or the user resumes the conversation in a different room.

Background workers

Separate from persistent sub-agents, the main agent can spawn ephemeral background workers with spawn_worker for one-off, self-contained tasks - without blocking the conversation. A worker is a lightweight orchestrator on a background daemon thread with an in-memory session, no Matrix identity and no memory. The agent gets a worker id back immediately and keeps talking to you while the worker runs.

• Non-blocking - spawning returns instantly; the worker runs on its own thread.
• Ephemeral & isolated - fresh context, no persistent workspace, no memory; the session is discarded when the task ends.
• Restricted toolset - web, file, git and docs tools only. Workers cannot manage agents, use memory, SSH, email or skills, send files, react, or spawn further workers (no recursion).
• Automatic delivery - when a worker finishes, a single watcher thread blocks on a shared result queue and fires an in-process trigger that wakes the main agent with a synthetic notification carrying the result. No polling, no HTTP, no external event bus. The agent then delivers the outcome to the originating room.

Concurrency is bounded by workers.max_concurrent (default 4), and the feature can be disabled with workers.enabled: false in config.

The Agentic Loop

Memtrix is not a single LLM call - it is an orchestrated reasoning loop. Each message runs an iterative cycle of "call the model, run any tools it asks for, feed the results back" until the model produces a final answer.

The cycle

When a message arrives, the orchestrator runs this loop:

Parallel tool execution

When the model fans out independent work - several reads, multiple searches - those calls run concurrently, up to 8 at a time, cutting latency. Tools that mutate state or depend on order are detected and always run sequentially to keep behaviour deterministic. Sequential tools include:

• Core/persona writes (read_core_file, write_core_file)
• File operations (str_replace_editor, delete_file, git, send_file, directory ops)
• Agent management (create_agent, delete_agent, ask_agent, spawn_worker)
• Memory writes (memory_conclude), skill edits (skill_manage), and all SSH operations

Schema validation & error tolerance

• Argument validation - missing required parameters and basic type mismatches are returned to the model as a precise, correctable error instead of failing deep inside a tool.
• Malformed JSON - if the model emits invalid JSON for a tool's arguments, the arguments become an empty object and the model is told to fix it next round, rather than the whole turn crashing.
• One-to-one tool IDs - every tool call is paired exactly with its result, so strict providers never see an orphaned tool result.
• Secret redaction - tool-call notifications redact argument values whose names look like secrets (password, passwd, token, api_key, apikey, secret, passphrase, credential, auth, private_key).

Iteration budget

Each request may take at most max_iterations tool-call rounds (default 25, set under the agent config block). When only a few rounds remain, the model receives a transient "tool-round budget warning" so it can wrap up cleanly instead of being cut off mid-task. At the hard cap it is asked for a final answer; that forced nudge is not persisted into history.

History trimming

Long-running rooms are bounded by max_history (default 60 messages). Once a session grows past the limit, the oldest turns are trimmed - but trimming is tool-pairing-safe (it never leaves an orphaned tool result) and the system prompt is always preserved. This prevents context-window overflow without losing the agent's identity or recent context.

Interruption

The /stop command is checked at every iteration of the loop, so it interrupts instantly - even mid-reasoning or mid-tool-call. Session history is preserved; just send your next message to continue. See Commands.

Tuning the loop

{
  "agent": {
    "max_iterations": 25,  // tool-call rounds per request
    "max_history": 60      // messages kept before trimming
  }
}

Both keys apply to the main agent and every sub-agent. Installs without an agent block use the defaults.

Persona System

How Memtrix defines its identity - and how it evolves over time.

Persona files

Memtrix's identity is defined by markdown files in workspace/:

How injection works

AGENT.md contains placeholders like {{BEHAVIOR}}, {{SOUL}}, {{USER}}, plus {{DATE}} for today's ISO date. At system prompt construction time, the orchestrator reads each file and replaces the placeholder with its contents.

Live updates

BEHAVIOR.md and SOUL.md are live-editable by Memtrix itself. When you tell it to behave differently or correct its style, it updates them with write_core_file - and the system prompt is rebuilt immediately after.

Default personality

Out of the box, Memtrix's personality (from SOUL.md) is:

• A companion, not a product
• Values privacy
• Honest and direct
• Curious, remembers things
• Not trying to impress
• Grows with the user

Its behavior (from BEHAVIOR.md) is:

• Keep it short, casual - like texting a friend
• No emojis, no "As an AI..." disclaimers
• Have opinions, be measured
• Ask questions only when necessary

All of this evolves as you interact. You can edit SOUL.md and BEHAVIOR.md directly; USER.md is best managed through the deriver or the Web Control Panel's memory admin.

Memory & RAG

Memtrix remembers across sessions through a layered memory system - auto-curated peer cards, distilled conclusions, and searchable conversation history powered by on-device embeddings.

Conversation Memory

Every conversation is saved as a raw session transcript under sessions/. The agent writes no journals by hand - transcripts are split into windowed chunks (~800 tokens each), embedded, and stored in the vector store so the agent can recall any past exchange. Recall works two ways and can combine them: by meaning (a natural-language query) and by date (a single date or a start_date+end_date range). A date can't be matched semantically, so date questions filter on each chunk's day metadata instead; the agent is given today's date and resolves "yesterday" or "last Wednesday" to an ISO date itself. Inter-agent and internal sessions are skipped, and each sub-agent only indexes its own conversations.

Semantic Search (RAG)

Conversation transcripts are embedded using nomic-embed-text-v1.5 via sentence-transformers and stored in ChromaDB. The model runs entirely on-device - no external API calls.

• Embedding dimensions: 768→256 (Matryoshka truncation for efficiency)
• Sync interval: every 300 seconds
• Singleton model: loaded once, shared across all agents
• Local-only loading: if cached, skips all HuggingFace Hub network calls
• Persistent hash cache: a .chunk-hashes.json stored alongside the index survives restarts, so warm starts re-embed only new or changed chunks and prune deleted sessions

How search works

User:     "Remember that cake recipe I told you about?"
  → search_memory(query="cake recipe")
  → Finds a conversation from 2026-03-12 (distance: 0.23)
  → Returns the transcript excerpt where you discussed it

User:     "What did we talk about on the 15th?"
  → search_memory(date="2026-06-15")   // resolved from today's date
  → Returns that day's conversation in order - no query needed

Automatic memory management

Memtrix manages memory without you asking. After each exchange:

• Conversation memory - the session transcript is saved automatically and its new chunks are embedded for future recall
• Profile card - the reasoning-memory deriver updates USER.md in the background (see Reasoning Memory)
• Behavior - BEHAVIOR.md is updated if you correct the agent's communication style

These operations are silent - you only see them if /verbose is enabled.

Documentation Index

Memtrix indexes its own documentation so it can answer questions about how the system works. The documentation site is parsed into sections, embedded with the same on-device model, and stored in a separate documentation collection on the shared chroma service.

• Source: the docs are bundled into the agent image at build time and parsed into searchable sections
• Change detection: the index is content-hashed and only rebuilt when the docs actually change
• Tools: search_docs returns matching sections with citations (no LLM), and ask_docs synthesizes a grounded answer with sources - see Tools Reference
• Availability: always on for the main agent and every sub-agent

Reasoning Memory

A background reasoning layer that turns raw conversation into durable, structured knowledge - an auto-curated profile card, a searchable store of typed conclusions about you, plus profile cards for the people, projects, and places you mention and the dated events tied to them.

The user model

Reasoning memory models a single peer:

• user - everything the agent learns about you

The user has a finite, always-injected profile card and a growing store of conclusions.

Profile card

The profile card is always injected into the system prompt, capped at peer_card_max_chars (default 1,500). It is re-curated from accumulated conclusions every few reasoning passes - so it stays compact and current instead of growing forever. Cap enforcement is boundary-safe, so the card is trimmed at clean boundaries instead of being cut mid-bullet.

Conclusions

A conclusion is a single durable fact about a peer, vector-indexed in the shared ChromaDB store (the representations collection). Each conclusion has a kind:

Conclusions also carry a provenance: derived (produced by the deriver) or manual (added by the agent via memory_conclude or by you in the panel). Each conclusion additionally carries a confidence - high, medium, or low - reflecting how certain it is: explicit statements and certain deductions are high, well-supported inferences medium, and tentative patterns low. Confidence ranks which memories surface first and how the cards are curated. Near-duplicate conclusions are merged automatically using embedding similarity, and a memory that is independently re-derived is promoted in confidence rather than merely re-counted, so repeatedly-observed facts rise to the top.

The deriver

The deriver is a background worker thread. It never blocks your reply - it works after the conversation continues.

Consolidation (distillation)

When you talk to Memtrix a lot, the deriver produces many conclusions. A background consolidation pass runs once a day - like memory consolidation during sleep - and distills each peer's stored conclusions into a smaller, cleaner set.

• Merges duplicates and near-duplicates into single clear statements
• Resolves contradictions by keeping the more-reinforced or more-recent fact and dropping the superseded one
• Drops anything outdated, trivial, or ephemeral
• Synthesizes higher-order patterns from several related items
• Decays weak memories - derived conclusions that are stale, never reinforced, and low confidence are pruned, while reinforced and high-confidence facts persist
• Preserves your manually added (manual) conclusions untouched - only derived ones are distilled or decayed

The schedule is persisted to disk, so it survives restarts and runs roughly every consolidation_interval_hours. After a pass the peer card is re-curated from the distilled set. Run /consolidate in any room to trigger a pass immediately. Consolidation respects the deriver pause toggle.

Recall at message time

When you send a message, memory is brought back in two ways, controlled by recall_mode:

Peer cards are always injected regardless of mode. With auto-injection, the top inject_top_k (default 5) conclusions are considered, but only those within a relevance threshold of your message are actually injected - weakly-related memories are suppressed so off-topic recall never crowds out the live conversation. Each injected memory is labelled with its confidence, and the agent is reminded that recall may be stale and should be verified before acting on anything critical.

Memory tools

When enabled, the agent can actively work with reasoning memory through five tools:

People & events

Beyond modelling you, the deriver also learns about the people, projects, and places you mention and tracks dated events tied to them - automatically, in the background.

• Entity cards - when you talk about someone (e.g. your sister Jenna), the deriver records facts about them and, once enough signal accumulates (entity_promote_threshold, or a single medium-plus-confidence fact), promotes them to a compact profile card stored under people/<slug>.md. These cards are deriver-owned just like USER.md.
• Contextual injection - when a known person, project, or place is named in your message, their card is injected as transient context so the agent already knows who you mean.
• Events - statements like "Jenna's birthday party is this Saturday" are resolved to a concrete date and stored. Upcoming events within event_lookahead_days are surfaced every turn under a 📅 Upcoming block, and once an event passes you get a one-time 🔔 Just passed nudge (within event_followup_days). Recurring annual events (birthdays) roll forward automatically.
• Privacy - set entity_memory to false to disable people/event learning entirely, or forget a single person (and their facts, card, and events) from the Web Control Panel's People tab.

Browse, add, and remove people and events from the Web Control Panel's memory admin (People and Events tabs).

Configuration

All options live under the memory key in config.json. Defaults are safe - installs without a memory section keep working.

{
  "memory": {
    "backend": "native",         // native | off
    "recall_mode": "hybrid",    // hybrid | context | tools | off
    "write_frequency": "async", // async | turn
    "reasoning_level": "low",   // minimal | low | medium | high | max
    "reasoning_model": null,    // model instance name, or null = main model
    "batch_tokens": 1000,       // flush threshold per peer
    "peer_card_max_chars": 1500,
    "inject_top_k": 5,         // conclusions auto-injected per message
    "consolidation": true,     // daily memory distillation pass
    "consolidation_interval_hours": 24,
    "consolidation_min_items": 12, // skip below this many derived items
    "entity_memory": true,      // learn about people / projects / places + events
    "entity_card_max_chars": 800,
    "entity_promote_threshold": 2, // facts before an entity gets a card
    "event_lookahead_days": 7,    // surface events within this window
    "event_followup_days": 2,    // post-event nudge window
    "event_retention_days": 30 // prune old non-recurring events
  }
}

Freezing & pausing

• Freeze a card - set freeze_user_card to true to stop the deriver re-curating the profile card (handy after a manual edit you want to keep). Conclusions still accumulate.
• Pause the deriver - pause all background reasoning entirely (pending messages are retained, nothing is reasoned). Exposed as a toggle in the Web Control Panel's memory admin.

Skills

Memtrix writes its own skills - short, reusable task workflows it captures so it handles recurring kinds of work better over time. Skills are a distinct layer from persona (character) and memory (facts): they capture how the agent works.

What a skill is

Each skill lives in the agent's own workspace as a Markdown file with YAML frontmatter:

---
name: server-security-audit
description: Steps to audit a remote host's security posture
---

## Instructions
1. Connect to the host and check for OS updates
2. Review open ports and running services
3. Inspect sudoers and SSH config
4. Summarize findings with severity levels

A skill folder can also hold reference files alongside SKILL.md. Each agent (main and every sub-agent) keeps its own isolated skill store.

How skills are authored

Authoring happens inside the normal agent loop - the same model, no second reasoner. After finishing a non-trivial task, the agent evaluates whether it was skill-worthy and, if so, captures the approach as generalized steps. A task is considered skill-worthy when it:

• Took 5 or more tool calls
• Required error recovery
• Involved a user correction
• Followed a non-obvious workflow

If an existing skill proved suboptimal during the task, the agent improves it on the spot instead of creating a duplicate. This evaluation is a required last step of finishing a larger task - declining to save must be a deliberate judgment, not an omission.

Progressive disclosure

Skills use the progressive-disclosure model rather than embedding-based matching:

1. At the start of every turn, the agent sees a catalog of all its skills, each as name: description.
2. It decides for itself which skill, if any, fits the current task.
3. It loads that skill's full instructions on demand with skill_manage (view), then follows them.

There is no vector index, no embedding step, and no distance threshold that could wrongly reject a relevant skill - just a plain catalog the model reasons over.

The skill_manage tool

One tool drives the whole lifecycle, with these actions:

Configuration

Skills are controlled by the optional skills block in config.json:

{
  "skills": {
    "enabled": true  // load skill_manage + inject the catalog
  }
}

When disabled, the skill_manage tool is not loaded and no catalog is injected. The default is true.

Web Control Panel

A local web app for running Memtrix without touching JSON - edit configuration, test connections, manage secrets, browse and curate memory, and apply changes with a safe restart. It runs as the web service and is reachable at localhost:8800.

Opening the panel

docker compose up -d

open http://localhost:8800

What's in the panel

Configuration editor

Edit your whole config.json through a structured UI instead of hand-editing JSON:

• Edit individual values or whole sections (providers, models, channels, the memory block, agents, and more)
• Changes are validated before they are saved, so a typo can't break your config
• Secret fields are shown as $PLACEHOLDER references - actual values live in your secret store, never in config.json

Connection tests

Before committing a change, test that it actually works. The panel can verify:

Secrets management

View which secrets are configured and set or rotate their values without editing files by hand:

• Lists every $PLACEHOLDER referenced by your config and whether it currently resolves
• Set or update a secret - it is written to the managed data/secrets.env file (or your active backend)
• Values are never echoed back in plain text once stored

See Secrets & Bitwarden for how secrets are resolved.

Memory administration

The panel is a full front-end for the reasoning-memory store:

• Browse & search conclusions - filter by peer (user / agent), read what the agent has concluded
• Add a conclusion - inject a durable fact manually (stored with manual provenance)
• Edit or delete - fix a wrong conclusion, or clear all conclusions for a peer
• Profile card - view and edit USER.md directly
• Freeze a card - stop the deriver re-curating a peer card after a manual edit
• Pause the deriver - halt background reasoning while you make changes
• Export / import - back up the entire memory store to a file and restore it later

Apply & restart

Some changes (new provider, model swap, channel change) require a restart. The panel triggers a safe restart and streams live progress back to your browser so you can watch the agent come back up.

Environment variables

Security

• Local by default - the published port is bound to localhost, not exposed to your network
• Token-gated - set MEMTRIX_WEB_TOKEN so only holders of the token can call the API
• Same hardened container - runs read-only, non-root, with dropped capabilities like the rest of the stack
• No secret leakage - secret values are write-only through the API and never returned in plain text

Voice & Transcription

Send Memtrix a voice note in Matrix and it transcribes it locally, on-device, then treats the text as normal input. No audio ever leaves your machine.

How it works

1. You record a voice message in any Matrix client - it arrives as an m.audio event.
2. Memtrix downloads the audio file to the agent's attachments/ directory.
3. The local speech-to-text engine transcribes it to text.
4. The transcript is forwarded into the normal agent loop as your message context - so the agent answers as if you had typed it.

The model is loaded lazily on first use, so enabling voice does not slow startup. Failures and timeouts degrade gracefully - a transcription error never breaks message handling.

The engine

Transcription uses a local faster-whisper model (provider local). Choose a model tier to trade speed for accuracy:

The model is downloaded to data/models/ on first use and reused across restarts.

Configuration

{
  "voice": {
    "enabled": false,
    "provider": "local",        // currently the only provider
    "model": "base",            // tiny | base | small | medium | large
    "language": null,          // null = auto-detect, or an ISO code like "en"
    "max_audio_bytes": 25000000, // 25 MB cap; larger files are skipped
    "timeout_seconds": 180      // give up after this long
  }
}

Email & Mail

Give Memtrix a mailbox and it can read your mail over IMAP and send over SMTP - on demand, or reactively the moment new mail arrives. Off by default, main-agent only, and screened for prompt injection.

The tools

Configuration

Configure the mailbox on the Email page of the Web Control Panel, or edit the email block in config.json by hand. The mailbox password is never stored in config - it is supplied as the EMAIL_PASSWORD secret (from .env, the managed data/secrets.env, or Bitwarden) and resolved at startup.

{
  "email": {
    "enabled": false,
    "imap_host": "imap.example.com",
    "imap_port": 993,
    "imap_ssl": true,
    "smtp_host": "smtp.example.com",
    "smtp_port": 587,
    "smtp_security": "starttls", // starttls | ssl | none
    "username": "[email protected]",
    "password": "$EMAIL_PASSWORD",
    "from_address": "[email protected]",
    "from_name": "Memtrix",
    "mailbox": "INBOX",
    "auto_mark_read": true,
    "max_fetch": 10,
    "max_body_chars": 4000,
    "react_to_mail": false,
    "poll_interval_seconds": 60,
    "trusted_senders": []
  }
}

Reactive mail

By default Memtrix only touches your mailbox when you ask it to. Turn on react_to_mail and it watches the inbox in the background, acting on mail the moment it arrives:

• A background poller checks the mailbox over IMAP every poll_interval_seconds (default 60s, clamped to a 15s minimum).
• When genuinely new mail appears, it pings the agent with a system notification - the same in-process mechanism that delivers finished background-worker results - so the agent can read, triage, and only message you when something actually needs attention.
• The poll never marks mail read, so your own email_check still sees it as new.
• An existing unread backlog (and mail that arrived while Memtrix was offline) is never announced - turning the feature on won't trigger a flood. Only mail arriving after the baseline is seeded gets a notification.
• The notification is delivered to your most recently active room.

Trusted-sender allowlist

For a tighter security boundary, set trusted_senders to a list of addresses. When the list is non-empty, Memtrix only ever sees mail from those senders - every other message is filtered out at the tool level and never reaches the agent, the reasoning memory, or the reactive poller.

• Matching is on the exact parsed From address (case-insensitive), never the display name - so a message that merely puts a trusted address in its display name cannot slip through.
• Each entry may be a bare address or a Name <addr> form; only the address part is kept. Invalid entries are dropped so a malformed allowlist can never silently widen access.
• The allowlist also narrows the IMAP search server-side as an optimisation, while the exact in-process check remains the real boundary.
• Leave the list empty to allow mail from all senders.

SSH Remote Admin

Memtrix can act as a sysadmin on your remote hosts. It opens a persistent interactive session and works inside it across many commands - so cd and environment changes persist, just like a human at a terminal - then closes it.

Setting it up

The tools

sudo handling

Pass sudo as a parameter to ssh_run - never embed it in the command string (if you do, the tool auto-corrects and warns). When sudo=true:

• Memtrix first tries non-interactive sudo -n. If passwordless sudo (NOPASSWD) is configured, the command runs immediately - no prompt.
• Only if a password is actually required does it ask you via the human-in-the-loop prompt.
• The sudo password is kept in memory only and never written to disk.

Security model

• Key-only auth - you install Memtrix's public key on each host; it never uses passwords to authenticate.
• Host-key pinning - host keys are pinned trust-on-first-use with an explicit SHA256 fingerprint confirmation, stored in data/ssh/known_hosts.
• Destructive-command confirmation - rm, dd, mkfs, shutdown/reboot, recursive chmod/chown, block-device writes and similar require explicit approval.
• Blocked targets - SSH to Memtrix's own services (conduit, chroma, searxng, memtrix) and to loopback/link-local addresses is refused; private LAN hosts are allowed.
• Output capped - command output is truncated to max_output_chars (default 20,000).
• Open sessions are closed cleanly on shutdown.

Configuration

{
  "ssh": {
    "enabled": true,
    "connect_timeout": 15,   // seconds to open a connection
    "command_timeout": 120,  // seconds per command
    "max_output_chars": 20000 // output cap returned to the model
  }
}

Set enabled to false to remove the SSH capability entirely - the tools are never loaded.

Custom Tools

Extend Memtrix by dropping Python files into src/tools/.

Adding a tool

Create a new .py file in src/tools/ that subclasses BaseTool:

from src.tools.base import BaseTool

class MyTool(BaseTool):

    def __init__(self, workspace_dir: str) -> None:
        super().__init__(
            name="my_tool",
            description="Does something useful.",
            parameters={
                "type": "object",
                "properties": {
                    "input": {
                        "type": "string",
                        "description": "The input."
                    }
                },
                "required": ["input"]
            }
        )

    def execute(self, **kwargs) -> str:
        return "result"

Restart Memtrix and the tool is automatically discovered and available to the LLM.

BaseTool interface

• name - unique tool name (used by the LLM to call it)
• description - explains what the tool does (guides the LLM)
• parameters - JSON Schema defining the tool's input
• execute(**kwargs) - called when the LLM invokes the tool, returns a string result

Injected parameters

The orchestrator injects these kwargs automatically:

Custom Providers

Add support for any LLM API by dropping a provider file into src/providers/.

Built-in providers

Three provider types ship out of the box. Most setups never need a custom one:

The openai_compatible API key is optional, so key-less local servers work without one. When supplied it is sent as a standard Authorization: Bearer header and can be stored as a $SECRET reference. In the control panel, the Models page can Discover the model identifiers any provider exposes (Ollama's installed models, OpenRouter's catalogue, or an OpenAI-compatible /models route) so you pick from a list rather than typing names. Discovery resolves secret references server-side, so keys are never exposed to the browser.

Vision (image input)

Vision-capable models can see images the user sends in chat. Set "vision": true on the model (or tick the Vision checkbox on the Models page) and incoming pictures - PNG, JPG, GIF, WebP - are delivered to the model as actual images instead of a file path, so it can describe, read, or reason over them. The image is expanded into each backend's native multimodal format at send time: Ollama's images field, or OpenAI-style image_url data URLs for OpenRouter and OpenAI-compatible endpoints. Received images stay in context across turns so you can ask follow-ups, bounded to the most recent few (up to 4 images, 10 MB each) to keep requests lean. Leave the flag off for text-only models and nothing changes.

Adding a provider

from src.providers.base import BaseProvider

class MyProvider(BaseProvider):

    def __init__(self, api_key: str) -> None:
        super().__init__(name="myprovider")
        self._api_key = api_key

    def completions(self, model, history, tools=None):
        # Call your LLM API and return a message object
        ...

The onboarding wizard automatically discovers new providers and prompts for their constructor parameters. Secret fields (containing "key", "token", "secret" in the parameter name) are handled automatically.

Configuration

All configuration lives in data/config.json. Secrets are stored separately - in .env, the managed data/secrets.env file, or Bitwarden - and injected at runtime. You can edit everything by hand, or use the Web Control Panel.

Config structure

{
    "main-agent": {
        "name": "Memtrix",
        "provider": "my-ollama",
        "model": "my-model",
        "channel": "matrix",
        "sessions": {},
        "verbose": false,
        "reasoning": false
    },
    "agents": {},
    "workspace-directory": "/home/memtrix/workspace",
    "providers": {
        "my-ollama": {
            "type": "ollama",
            "base_url": "http://host.docker.internal:11434"
        },
        "my-openrouter": {
            "type": "openrouter",
            "api_key": "$OPENROUTER_API_KEY"
        },
        "my-openai-compatible": {
            "type": "openai_compatible",
            "base_url": "http://host.docker.internal:8000/v1",
            "api_key": "$OPENAI_API_KEY"
        }
    },
    "models": {
        "my-model": {
            "provider": "my-ollama",
            "model": "llama3",
            "think": true,
            "vision": false,
            "input_cost": 0.15,
            "output_cost": 0.60
        }
    },
    "channels": {
        "matrix": {
            "type": "matrix",
            "homeserver": "http://conduit:6167",
            "user_id": "@memtrix:memtrix.local",
            "access_token": "$MATRIX_ACCESS_TOKEN"
        }
    },
    "memory": {
        "backend": "native",
        "recall_mode": "hybrid",
        "write_frequency": "async",
        "reasoning_level": "low",
        "reasoning_model": null,
        "batch_tokens": 1000,
        "peer_card_max_chars": 1500,
        "inject_top_k": 5
    },
      "voice": {
        "enabled": false,
        "provider": "local",
        "model": "base",
        "language": null,
        "max_audio_bytes": 25000000,
        "timeout_seconds": 180
      },
    "ssh": {
        "enabled": true,
        "connect_timeout": 15,
        "command_timeout": 120,
        "max_output_chars": 20000
    },
    "skills": {
        "enabled": true
    },
    "agent": {
        "max_iterations": 25,
        "max_history": 60
    },
    "secrets": {
        "backend": "env"
    }
}

Secret management

Values starting with $ are resolved at startup from one of three backends: the .env / managed data/secrets.env file, the process environment, or Bitwarden Secrets Manager. For file/env backends, the lookup name is MEMTRIX_SECRET_ + the placeholder name.

Secrets are resolved once at boot and then cleared from the process environment - they can't leak via /proc or subprocess inspection. Full details, including the Bitwarden backend, are on the Secrets & Bitwarden page.

Main agent config

Model config

Channel config (Matrix)

Memory config

The memory block tunes the reasoning-memory system (backend, recall mode, derivation depth, peer cards, and more). Every key is documented on the Reasoning Memory page.

SSH config

The optional ssh block controls the remote-administration tools. Omit it to run on the defaults below.

Skills config

The optional skills block controls the agent's self-authored skills. Omit it to run on the defaults below.

Agent loop config

The optional agent block tunes the core tool-calling loop. Each request runs the model, executes any tools, and repeats until a final answer is returned; this cap bounds how many rounds a single request may take. Omit it to run on the default below.

Email config

The optional email block connects a mailbox over IMAP/SMTP and is off by default. Full key-by-key details, reactive mail, and the trusted-sender allowlist are on the Email & Mail page.

Web Control Panel

The panel is configured through environment variables on the web service, not config.json:

See the Web Control Panel guide for full details.

Secrets & Bitwarden

API keys and access tokens never live in config.json. They are stored as $PLACEHOLDER references and resolved at boot from one of three backends - a managed file, the environment, or Bitwarden Secrets Manager.

How placeholders work

Anywhere a secret is needed, the config holds a reference like $OPENROUTER_API_KEY instead of the real value:

"api_key": "$OPENROUTER_API_KEY"

At startup Memtrix resolves every $PLACEHOLDER, then scrubs the secrets from the process environment so they can't be read back via /proc or a subprocess.

Resolution order

For each placeholder, Memtrix looks in this order:

1. Bitwarden - if the Bitwarden backend is active, the secret map is checked first (e.g. $MATRIX_ACCESS_TOKEN → the MATRIX_ACCESS_TOKEN secret)
2. Environment / managed file - the variable MEMTRIX_SECRET_<NAME> (loaded from data/secrets.env or the real environment)
3. Empty - if nothing matches, it resolves to an empty string

Managed file backend (default)

By default, secrets are kept in data/secrets.env - a simple KEY=value file holding the prefixed variables:

MEMTRIX_SECRET_MATRIX_ACCESS_TOKEN=syt_...
MEMTRIX_SECRET_OPENROUTER_API_KEY=sk-or-...
MEMTRIX_SECRET_REGISTRATION_TOKEN=...

You rarely edit this by hand - onboarding writes it, and the Web Control Panel lets you set or rotate values safely (write-only; values are never echoed back).

Bitwarden Secrets Manager

For a centralized, auditable secret store, point Memtrix at Bitwarden Secrets Manager. Add a secrets block to config.json:

{
  "secrets": {
    "backend": "bitwarden",
    "organization_id": "<org-uuid>",
    "project_id": "<project-uuid>",
    "api_url": "https://api.bitwarden.eu",
    "identity_url": "https://identity.bitwarden.eu"
  }
}

Machine-account token

Authentication uses a Bitwarden machine-account access token, provided via the BWS_ACCESS_TOKEN environment variable (it is the one secret not prefixed with MEMTRIX_SECRET_). Memtrix logs in with it, fetches all secrets in the project, and resolves placeholders by name.

Best practices

• Keep data/secrets.env and .env out of version control (they are git-ignored)
• Rotate tokens through the Web Control Panel rather than editing files
• For multi-host or team setups, prefer Bitwarden so secrets are centralized and revocable
• Never paste secrets into config.json - always use a $PLACEHOLDER

Commands

Slash commands available in any chat room. Prefix with /.

Checking costs

Run /costs in any room to see what you've spent. For OpenRouter providers it reports live credit usage (US dollars); if multiple providers share the same API key, they are grouped and counted once:

You:    /costs

Memtrix: OpenRouter usage (main):
         Today (UTC): $0.4213
         This week:   $2.8740
         This month:  $9.1502
         All-time:    $41.2280
         Limit:       $50.0000 ($8.7720 remaining)

Costs for any model

OpenRouter reports real spend through its API. For every other provider - Ollama, OpenAI-compatible endpoints, self-hosted gateways - Memtrix can still estimate cost by counting tokens. Give a model a price and /costs adds a token-based estimate for it. Set input_cost and/or output_cost on the model (USD per 1 million tokens), either in config.json or via the Input cost / Output cost fields on the Models page:

You:    /costs

Memtrix: Local usage (token-based estimate since first run):
         gpt-4o-mini:
           Prompt:      1,204,320 tokens
           Completion:    186,540 tokens
           Est. cost:   $0.2926  (in $0.15/M, out $0.60/M)
         Total estimate: $0.2926

Tools Reference

All 40 built-in tools, auto-discovered at startup. Some are gated by config (reasoning memory, SSH, skills) or by channel (Matrix-only). Drop a new file in src/tools/ to add your own - see Custom Tools.

Time

Persona Files

Memory (conversation history)

Reasoning Memory

Available when reasoning memory is enabled (recall_mode of hybrid or tools). See Reasoning Memory.

Documentation

Always on for the main agent and every sub-agent - lets Memtrix research its own docs. See Memory for how the index is built.

Web

File Management

Reactions

Agent Management

Remote Administration (SSH)

Memtrix can administer remote hosts over SSH through a persistent interactive session: it opens a connection and works inside it across many commands (so cd and environment changes persist), then closes it. These tools are available to the main agent only and load only when ssh.enabled is true.

Email

Available only when email.enabled is true (main agent only). Reads a mailbox over IMAP and sends mail over SMTP. Connection settings are configured in the control panel; the mailbox password is supplied as the EMAIL_PASSWORD secret (.env or Bitwarden).

Reactive mail

Turn on react_to_mail (off by default) to let Memtrix act on mail the moment it arrives instead of only when asked. A background poller checks the mailbox every poll_interval_seconds (default 60s, minimum 15s) and, when genuinely new mail appears, pings the agent with a system notification — the same in-process mechanism that delivers finished background-worker results — so it can read, triage, and only message you when something needs your attention. The poll never marks mail read, and an existing unread backlog (or mail that arrived while Memtrix was offline) is never announced, so enabling it won't trigger a flood. Configure it on the Email page of the control panel.

Trusted-sender allowlist

For a tighter security boundary, set trusted_senders (a list of addresses) on the Email page. When non-empty, Memtrix only ever sees mail from those senders: every other message is filtered out at the tool level and never reaches the agent, the reasoning memory, or the reactive poller. Matching is on the exact parsed From address (case-insensitive), never the display name, so a message that merely puts a trusted address in its display name cannot slip through. Leave the list empty to allow mail from all senders.

Skills

Available to every agent when skills.enabled is true. Lets the agent author and reuse its own task workflows. See Skills.

Security

Defense-in-depth - multiple independent layers that each limit what the system and the LLM can do.

Container Isolation

• Non-root user - runs as memtrix (UID 1000), never root
• Read-only filesystem - immutable root via read_only: true; only workspace/, data/, and /tmp are writable
• All capabilities dropped - cap_drop: ALL with no-new-privileges: true
• No shell tools - curl, wget, and other network utilities are not installed in the image
• Internal-only networking - the agent, Conduit, SearXNG, and chroma sit on a private Docker network; the bot itself publishes no ports. Only the Web Control Panel is published, bound to localhost:8800

No Arbitrary Code Execution

The LLM has no shell access on the host container. There is no local run_command tool - every action goes through a purpose-built tool with its own validation.

The optional SSH tools do let the agent run commands on remote hosts you have explicitly registered, but never on the Memtrix container itself. That capability is gated: it only loads when ssh.enabled is true, authenticates with a key you install, pins host keys trust-on-first-use, requires confirmation for destructive commands, and keeps any sudo password in memory only. See Tools Reference → Remote Administration.

SSRF Protection

All outbound tools (fetch_url, download_file, git) validate URLs against:

• A hostname blocklist of internal Docker service names (conduit, searxng, chroma, localhost, etc.)
• DNS resolution - hostnames are resolved and IPs checked against private, loopback, link-local, and reserved ranges

Human-in-the-Loop

Sensitive operations require explicit approval:

• File downloads - user sees URL and destination, must confirm
• File overwrites - overwriting existing files requires approval
• Agent creation - user must confirm name and expertise

During inter-agent calls, confirm_with_user() returns false (deny) when no human callback is available - no auto-approval of destructive operations.

File System Protection

• Path traversal prevention - every path validated with os.path.realpath()
• Core file protection - system files only accessible through dedicated tools with strict allowlist
• Memory protection - memory/ directory off-limits to general file tools
• Read-before-write - per-room tracking ensures reading before modifying

Prompt Injection Mitigation

• Untrusted content tagged - web search results, fetched URLs, remote SSH command output, downloads, and attachments are prefixed with disclaimers
• Active injection screening - the output of the web-fetching tools (web_search and fetch_url) — arbitrary content straight from external sites — is screened by ProtectAI's DeBERTa prompt-injection detector, a local classifier that runs entirely inside the container (openly licensed — no HuggingFace token required). If the content is flagged as a prompt-injection or jailbreak attempt, the tool result is replaced with a tool-error so the malicious text never enters the conversation, and the model is told the source is untrusted
• Filename sanitization - os.path.basename() with auto-increment on collision
• Sender name sanitization - brackets stripped, 50-char limit to prevent prompt injection via Matrix profile names

Screening is controlled by the prompt_guard config block: enabled (default true), model (a short name like deberta, or any full HuggingFace repo id of a prompt-injection sequence classifier), threshold (malicious-probability cutoff, default 0.5), max_chars (per-result cap), and fail_closed (block untrusted content if the classifier cannot run; default false fails open). The model downloads once to data/models/ and is reused across restarts.

Secret Management

• Secrets live outside config.json - in .env, the managed data/secrets.env, or Bitwarden - and are injected at container startup
• Resolved once at boot, then cleared from the process environment
• The Web Control Panel exposes secrets write-only - values are never returned in plain text
• SearXNG gets a randomly generated secret key during setup
• Conduit registration token is randomly generated - no hardcoded defaults

See Secrets & Bitwarden for the full resolution model.

Docker & Services

A small set of services orchestrated by Docker Compose - all on a private internal network, with only the Web Control Panel published to localhost.

Services

Volumes

Dockerfile

• Base: python:3.13-slim
• System deps: git only (minimal attack surface)
• User: memtrix (uid/gid 1000)
• Runtime: a supervisor entrypoint runs the agent (and the panel runs the same image as the web service)
• PYTHONUNBUFFERED=1 for real-time log output

Useful commands

# Start all services
docker compose up -d

# View agent logs
docker compose logs -f memtrix

# View web control panel logs
docker compose logs -f web

# Restart the agent only
docker compose restart memtrix

# Stop everything
docker compose down

# Rebuild after code changes
docker compose build && docker compose up -d

Performance & Startup

Memtrix is designed to connect and start responding within seconds, then do the heavy lifting - embedding, indexing, reasoning - in the background, off the request path.

Non-blocking startup

Rather than waiting 40-90 seconds for the embedding model and indexes before accepting messages, the agent connects to Matrix and starts responding right away. The expensive work is deferred:

• The on-device embedding model is loaded lazily on first use, on a background thread - and warmed at boot off the request path so the first message doesn't pay the one-time load cost.
• Initial conversation and documentation indexing runs on the periodic-sync thread, not the startup path.
• The embedding model is a thread-safe lazy singleton shared across the conversation, docs, and reasoning-memory indexes - it loads once even with concurrent callers.

Embedding & CPU

The local embedding model (nomic-embed-text-v1.5) runs in-process. To keep the agent responsive while it indexes:

• Indexing upserts in bounded batches that release the GIL between passes, so the Matrix event loop and request handler aren't starved.
• Embedding is capped to leave one CPU core free for the agent. Override the thread count with the MEMTRIX_EMBED_THREADS environment variable.
• Embeddings are truncated to 256 dimensions (Matryoshka) for efficiency without meaningful quality loss.

Incremental, restart-safe indexing

Conversation indexing is incremental and survives restarts via a per-chunk content-hash cache (.chunk-hashes.json stored alongside the index):

• Growing conversations only embed their newest chunks.
• Deleted sessions are pruned from the index.
• Warm starts skip re-embedding unchanged history.
• A chunk is re-embedded whenever its hash changed or it is missing from the collection, so a wiped or partially rebuilt index heals itself.

The documentation index is content-hashed too - it only rebuilds when the docs actually change. Both run a background sync roughly every 300 seconds.

Background work

First start vs. warm start

• First start downloads the embedding model (~100 MB) into data/models/ and, for voice, the whisper model into data/models/ on first use. The agent stays responsive throughout.
• Warm start reuses the cached models and the hash caches, so there is no re-download and only new/changed content is re-embedded.

Troubleshooting

Common issues and how to fix them.

Conduit won't start

Check if port 6167 is already in use:

lsof -i :6167

Verify the Conduit container is running:

docker compose ps conduit

Bot doesn't respond

• Check logs: docker compose logs -f memtrix
• Verify onboarding completed: the log should show "Starting Memtrix..." at boot
• Make sure you invited the bot to the room (check the exact username from onboarding)
• Check that the LLM provider is reachable (Ollama running? OpenRouter key valid?)

Can't connect to Ollama

If Ollama runs on your host machine, the container needs to reach it. Use:

http://host.docker.internal:11434

On Linux, you may need to add --add-host=host.docker.internal:host-gateway to the container or use your machine's LAN IP.

Embedding model download fails

Memtrix downloads nomic-embed-text-v1.5 on first launch. If it fails:

• Check internet connectivity from the container
• Ensure data/cache/ is writable (should be owned by uid 1000)
• Try restarting: docker compose restart memtrix

Permission errors

All mounted directories should be owned by uid 1000:

sudo chown -R 1000:1000 workspace/ agents/ data/

Session or memory issues

• /clear (or /new) - resets the current session
• Sessions are stored in data/ as JSON files organized by date
• Memory files are in workspace/memory/
• Reasoning-memory conclusions live in the chroma service - browse, edit, or export them from the Web Control Panel

Web Control Panel won't load

• Confirm the service is up: docker compose ps web
• Check logs: docker compose logs -f web
• Make sure nothing else uses port 8800 (lsof -i :8800), or change MEMTRIX_WEB_PORT
• If the API returns 401, set or re-enter the MEMTRIX_WEB_TOKEN the panel asks for

Memory not persisting / Chroma errors

• Check the vector store is running: docker compose ps chroma
• Verify both memtrix and web point at CHROMA_URL=http://chroma:8000
• Inspect logs: docker compose logs -f chroma
• If memory is paused, the deriver won't write new conclusions - check the panel's deriver toggle

Full reset

docker compose down
rm -rf data/ workspace/ agents/ .env
./setup.sh
./onboard.sh
docker compose up -d

Changelog

Every release, newest first. This page is rendered live from CHANGELOG.md in the repository - the single source of truth.