AI Chat App 384 (why I made one); Part 1
Magicspace (stylized as "magicspace.") is my attempt at a distinctive AI chat app. I had a few primary goals:
- The AI should have very strong yet efficient long-term memory
- You should be able to BYOK (Bring Your Own Key) so you can use LLMs from any platform, and you should be able to use these LLMs together.
- Chats should feel like flexible, composable, and interconnected branches of a knowledge tree / notebook rather than rigid siloed one-off instances.
Preview:
Two things might immediately catch your attention, one being the colored hierarchical chats on the left side, and another being the memory-related tool calls the AI is making, a bonus third might be my awesome CSS skills.
Let's start with the memory system and come back to the hierarchical chats in a moment.
The Memory System
The memory system is a zettelkasten-based hierarchical memory layout where nodes can be partially loaded into memory (context history / sysprompt). When unloaded, nodes are stored in SOTA embeddings: MongoDB Voyage (Yes, I was also somewhat shocked to hear that MongoDB runs laps around OpenAI and Cohere when it comes to vector embeddings1). The nodes are stored in PostgreSQL and use a recursive node layout to form a tree (meaning yes, we will need to use recursive CTEs & pgvector which i'll show in Part 2).
This is the general XML layout that loads in the models memory first of all:
<loaded_memory>
Currently loaded nodes:
<loaded_nodes>
<node_partial codename="[snake_case_codename]" has_children="[boolean]">
<name> [human_readable_name] </name>
<summary>[node_summary]</summary>
<subnodes>
<node codename="[snake_case_codename]">
<name>[human_readable_name]</name>
<summary>[node_summary]</summary>
<description>[node_description]</description>
<tags>
<tag>[tag_a]</tag>
<tag>[tag_b]</tag>
<tag>[tag_c]</tag>
<tag>[tag_d]</tag>
</tags>
</node>
<node_partial codename="[snake_case_codename]" has_children="[boolean]">
<name>[human_readable_name]</name>
<summary>[node_summary]</summary>
</node_partial>
</subnodes>
</node_partial>
<node_partial codename="[snake_case_codename]" has_children="[boolean]">
<name>[human_readable_name]</name>
<summary>[node_summary]</summary>
</node_partial>
</loaded_nodes>
Nodes can be loaded at different levels:
- PARTIAL: You know it exists and see immediate connections, but not full details
- FULL: Complete access to all properties and connections
Load nodes fully when you need to work with them actively. Unload to partial when no longer needed to manage context efficiently.
</loaded_memory>
<clipboard_system>
<clipboard/>
When nodes are deleted, their content moves to the clipboard temporarily to prevent information loss. Review and refactor clipboard contents into proper nodes. The only way to clear the clipboard is by integrating the content back into the graph structure.
</clipboard_system>You can see that the first node is just a container for other nodes. There are multiple types of nodes based on the zettelkasten:
Node Types
| Type | Purpose | Type-specific data | Example |
|---|---|---|---|
| Index | Organizes related memories into a hierarchy. | None | work_and_projects, relationships |
| Actor | Represents something with agency. | origin: external or internal; free-form actor_type |
A person such as sarah, or an internal part such as inner_critic |
| Event | Represents something that happened or may happen. | Fuzzy date fields and optional recurrence_days |
promotion, weekly_team_review |
| Pattern | Represents a recurring behavior, dynamic, or theme. | Occurrences containing a description and fuzzy date | approval_seeking, avoids_difficult_conversations |
| Fact | Represents a subjective or objective claim. | origin: subjective or objective; required citation |
A sourced launch date or a quoted user statement |
Quick Note: A common misconception is that actors must necessarily model human actors, or inner voices. But this is not true, actors can model abstract things such as: AIs, Countries, Corporations, the weather, and even philosophies. Anything that has a core loop and has agency to influence you is considered an actor. Philosophical worldviews can be actors because by simulating them in our minds (e.g. "What would a stoic person do here?"), we effectively create an actor out of them.
Types of Fact citations
Facts can come from many places, here is a quick table and its corresponding PostgreSQL.
| Citation type | Purpose | Required data | Optional data | PostgreSQL fields |
|---|---|---|---|---|
| URL | Cites a web page or other URL-addressable source. | title, url |
None | citation_title, citation_url |
| Book | Cites a book or a specific location within one. | title |
author, page |
citation_title, citation_author, citation_page |
| Document | Cites a report, file, paper, or other document. | title |
author |
citation_title, citation_author |
| User statement | Grounds a fact in something the user said. | title, quotes |
None | citation_title, citation_quotes |
| Other | Stores a source that does not fit the other variants. | title, raw |
None | citation_title, citation_raw |
CREATE TYPE fact_origin AS ENUM (
'objective',
'subjective'
);
CREATE TYPE citation_type AS ENUM (
'url',
'book',
'document',
'user_statement',
'other'
);
CREATE TABLE nodes_fact (
node_id integer PRIMARY KEY REFERENCES nodes (id),
origin fact_origin NOT NULL,
citation_type citation_type NOT NULL,
citation_title character varying NOT NULL,
citation_url character varying,
citation_author character varying,
citation_page character varying,
citation_quotes character varying[],
citation_raw character varying,
created_at timestamp without time zone NOT NULL,
updated_at timestamp without time zone NOT NULL
);Embedding Accuracy
In this model, keeping embeddings accurate is key. I'm using a set of broad tags to help semantic searches match a node. Having a separate summary rather than just a description avoids overwhelming the AI with context when simply searching for a node via a tool call, whilst still giving it a sense of whether this is the appropriate node to load.
Tool Calling
I use AutoAgents, a fantastic multi-agent framework that makes our tool calling easy, made by the team at liquidos-ai. In particular, an internal fork of AutoAgents to make tool calling easier by switching to JSON5 and adding in JSON healing. I also added in support for Gemini and Anthropic's encrypted reasoning blocks, and the addition of a few tool lifecycle hooks for us to forward live tool status to the user via WebSockets (e.g. "Web Search for 'Reasons NixOS is cool'
I hoped to contribute the changes upstream, but at the time I had tunnel vision towards the project, and was later bogged down by exams. AutoAgents upstream has since matured, adding in reasoning streaming and hooks, but our fork still has features specific to magicspace's architecture that aren't implemented upstream.
Two particularly interesting problems that cropped up while setting up tool calling were making sure agents were absolutely sure when deleting a node, and having agents be able to paginate through semantic search results.
Agent --> DeleteNode:
tool_call:
name: DeleteNode
arguments:
node_codename: q4_product_roadmap
subnode_action: DeleteAll
confirmation_code: nullDeleteNode (generates a confirmation code on the spot, never stored) --> Agent:
tool_result:
name: DeleteNode
result:
success: false
data: null
errors: []
warnings: []
notices: []
requires_confirmation:
message: "Confirm deletion of node 'q4_product_roadmap' (delete all subnodes)"
confirmation_code: "7fa2c19e"The agent then decides to confirm the action or not. If it does:
Agent --> DeleteNode (with the confirmation code this time):
tool_call:
name: DeleteNode
arguments:
node_codename: q4_product_roadmap
subnode_action: DeleteAll
confirmation_code: "7fa2c19e"The mechanism for generating the confirmation codes themselves is quite interesting, it is ENTIRELY STATELESS. Using HMAC (hmac) and a secret code, combining the hash of the inputs and the secret code creates a unique code. The code is always unique per set of inputs. Thinking it over, including the unique UUID of the node in the hash might be beneficial, or another narrowing hash. DefaultHasher is not always stable across rust versions, this is fine for this usecase and may be slightly beneficial to prevent reuse of old confirmation codes and staying up-to-date with the current preferred hasher.
/// Generate a confirmation code for the given inputs
///
/// The code is derived from:
/// 1. A hash of the inputs (stable representation)
/// 2. The server secret (from CONFIRMATION_SECRET env var)
///
/// This ensures the same inputs always produce the same code,
/// but different inputs produce different codes.
pub fn confirmation_code(inputs: &impl Hash) -> String {
// Hash the inputs to get a stable representation
let mut hasher = DefaultHasher::new();
inputs.hash(&mut hasher);
let input_hash = hasher.finish();
// Create HMAC with secret
let mut mac = HmacSha256::new_from_slice(&get_secret()).expect("HMAC can take key of any size");
// Include the input hash in HMAC
mac.update(&input_hash.to_be_bytes());
// Return first 8 hex chars as the code
hex::encode(mac.finalize().into_bytes())[..8].to_string()
}The next thing is pagination, so:
Agent --> SemanticSearch:
tool_call:
name: SemanticNodeSearch
arguments:
query: "research about improving customer onboarding"
include_messages: true
node_types: null
page: 1
limit: 3SemanticSearch --> Agent:
tool_result:
name: SemanticNodeSearch
result:
success: true
data:
results:
- node_id: 101
codename: customer_onboarding_research
name: Customer Onboarding Research
node_type:
Pattern:
occurrences: [...]
similarity: 0.861
node_similarity: 0.84
message_similarity: 0.91
- node_id: 138
codename: activation_interview_findings
name: Activation Interview Findings
node_type: Index
similarity: 0.822
node_similarity: 0.81
message_similarity: 0.85
- node_id: 204
codename: onboarding_configuration_friction
name: Onboarding Configuration Friction
node_type:
Fact:
origin: Subjective
citation:
Other:
title: "..."
raw: "..."
similarity: 0.785
node_similarity: 0.77
message_similarity: 0.82
page: 1
per_page: 3
total_pages: 2
total_count: 5
errors: []
warnings: []
notices: []And to paginate, simply set page=2,
Agent --> SemanticSearch:
role: assistant
tool_call:
name: SemanticNodeSearch
arguments:
query: "research about improving customer onboarding"
include_messages: true
node_types: null
page: 2
limit: 3I'll show how this works internally in Part 2.
Something shocking
In my testing, Grok models (particularly Grok 4 - if you look closely at the screenshot of the webapp, that is actually the default model) were the best at making tool calls and understanding this system. Other models at the time (such as Gemini 2.5 Pro) struggled significantly with the syntax. This is an interesting observation because at the time Grok models were considered worse than Google or Anthropic models.
Conclusion
This is the conclusion to Part 1 of this doc. I will make part 2 soon, discussing Recursive CTEs, pgvector and dig deeper into some interesting decisions I made with PostgreSQL tables in an attempt to structurally prevent malformed data (rather than require loose validation - although if I had to resort to it SeaORM's models made it possible at the data-layer, I'll talk about that too). I'll also explain the hierarchical chats & reasoning behind it, as well as how I integrated Remote MCPs via a Manager Singleton that handles lifetime & MCP deduplication as well as authentication. It'll include how I integrated BYOK with any OpenAI-compatible API key using AutoAgents. Lastly, I'll explain how I combined AsyncAPI and OpenAPI for a strongly typed API between Rust & TypeScript both with WebSockets and REST.