magicspace. 7 minute read

AI Chat App 384 (why I made one); Part 1

Magicspace (stylized as "magicspace.") is my attempt at a distinctive AI chat app. I had a few primary goals:

  1. The AI should have very strong yet efficient long-term memory
  2. You should be able to BYOK (Bring Your Own Key) so you can use LLMs from any platform, and you should be able to use these LLMs together.
  3. Chats should feel like flexible, composable, and interconnected branches of a knowledge tree / notebook rather than rigid siloed one-off instances.

Preview:

Two things might immediately catch your attention, one being the colored hierarchical chats on the left side, and another being the memory-related tool calls the AI is making, a bonus third might be my awesome CSS skills.

Let's start with the memory system and come back to the hierarchical chats in a moment.

The Memory System

The memory system is a zettelkasten-based hierarchical memory layout where nodes can be partially loaded into memory (context history / sysprompt). When unloaded, nodes are stored in SOTA embeddings: MongoDB Voyage (Yes, I was also somewhat shocked to hear that MongoDB runs laps around OpenAI and Cohere when it comes to vector embeddings1). The nodes are stored in PostgreSQL and use a recursive node layout to form a tree (meaning yes, we will need to use recursive CTEs & pgvector which i'll show in Part 2).

This is the general XML layout that loads in the models memory first of all:

XML
<loaded_memory>
	Currently loaded nodes:
	<loaded_nodes>
     <node_partial codename="[snake_case_codename]" has_children="[boolean]">
       <name> [human_readable_name] </name>
       <summary>[node_summary]</summary>
       <subnodes>
         <node codename="[snake_case_codename]">
           <name>[human_readable_name]</name>
           <summary>[node_summary]</summary>
           <description>[node_description]</description>
           <tags>
             <tag>[tag_a]</tag>
             <tag>[tag_b]</tag>
             <tag>[tag_c]</tag>
             <tag>[tag_d]</tag>
           </tags>
         </node>
         <node_partial codename="[snake_case_codename]" has_children="[boolean]">
           <name>[human_readable_name]</name>
           <summary>[node_summary]</summary>
         </node_partial>
       </subnodes>
     </node_partial>
     <node_partial codename="[snake_case_codename]" has_children="[boolean]">
       <name>[human_readable_name]</name>
       <summary>[node_summary]</summary>
     </node_partial>
   </loaded_nodes>

   Nodes can be loaded at different levels:
   - PARTIAL: You know it exists and see immediate connections, but not full details
   - FULL: Complete access to all properties and connections

   Load nodes fully when you need to work with them actively. Unload to partial when no longer needed to manage context efficiently.
</loaded_memory>

<clipboard_system>
<clipboard/>

When nodes are deleted, their content moves to the clipboard temporarily to prevent information loss. Review and refactor clipboard contents into proper nodes. The only way to clear the clipboard is by integrating the content back into the graph structure.
</clipboard_system>

You can see that the first node is just a container for other nodes. There are multiple types of nodes based on the zettelkasten:

Node Types

Type Purpose Type-specific data Example
Index Organizes related memories into a hierarchy. None work_and_projects, relationships
Actor Represents something with agency. origin: external or internal; free-form actor_type A person such as sarah, or an internal part such as inner_critic
Event Represents something that happened or may happen. Fuzzy date fields and optional recurrence_days promotion, weekly_team_review
Pattern Represents a recurring behavior, dynamic, or theme. Occurrences containing a description and fuzzy date approval_seeking, avoids_difficult_conversations
Fact Represents a subjective or objective claim. origin: subjective or objective; required citation A sourced launch date or a quoted user statement

Quick Note: A common misconception is that actors must necessarily model human actors, or inner voices. But this is not true, actors can model abstract things such as: AIs, Countries, Corporations, the weather, and even philosophies. Anything that has a core loop and has agency to influence you is considered an actor. Philosophical worldviews can be actors because by simulating them in our minds (e.g. "What would a stoic person do here?"), we effectively create an actor out of them.

Types of Fact citations

Facts can come from many places, here is a quick table and its corresponding PostgreSQL.

Citation type Purpose Required data Optional data PostgreSQL fields
URL Cites a web page or other URL-addressable source. title, url None citation_title, citation_url
Book Cites a book or a specific location within one. title author, page citation_title, citation_author, citation_page
Document Cites a report, file, paper, or other document. title author citation_title, citation_author
User statement Grounds a fact in something the user said. title, quotes None citation_title, citation_quotes
Other Stores a source that does not fit the other variants. title, raw None citation_title, citation_raw
SQL
CREATE TYPE fact_origin AS ENUM (
    'objective',
    'subjective'
);

CREATE TYPE citation_type AS ENUM (
    'url',
    'book',
    'document',
    'user_statement',
    'other'
);

CREATE TABLE nodes_fact (
    node_id integer PRIMARY KEY REFERENCES nodes (id),
    origin fact_origin NOT NULL,
    citation_type citation_type NOT NULL,
    citation_title character varying NOT NULL,
    citation_url character varying,
    citation_author character varying,
    citation_page character varying,
    citation_quotes character varying[],
    citation_raw character varying,
    created_at timestamp without time zone NOT NULL,
    updated_at timestamp without time zone NOT NULL
);

Embedding Accuracy

In this model, keeping embeddings accurate is key. I'm using a set of broad tags to help semantic searches match a node. Having a separate summary rather than just a description avoids overwhelming the AI with context when simply searching for a node via a tool call, whilst still giving it a sense of whether this is the appropriate node to load.

Tool Calling

I use AutoAgents, a fantastic multi-agent framework that makes our tool calling easy, made by the team at liquidos-ai. In particular, an internal fork of AutoAgents to make tool calling easier by switching to JSON5 and adding in JSON healing. I also added in support for Gemini and Anthropic's encrypted reasoning blocks, and the addition of a few tool lifecycle hooks for us to forward live tool status to the user via WebSockets (e.g. "Web Search for 'Reasons NixOS is cool' "), amongst other fallback mechanisms we needed.

I hoped to contribute the changes upstream, but at the time I had tunnel vision towards the project, and was later bogged down by exams. AutoAgents upstream has since matured, adding in reasoning streaming and hooks, but our fork still has features specific to magicspace's architecture that aren't implemented upstream.

Two particularly interesting problems that cropped up while setting up tool calling were making sure agents were absolutely sure when deleting a node, and having agents be able to paginate through semantic search results.

Agent --> DeleteNode:

YAML
   tool_call:
     name: DeleteNode
     arguments:
       node_codename: q4_product_roadmap
       subnode_action: DeleteAll
       confirmation_code: null

DeleteNode (generates a confirmation code on the spot, never stored) --> Agent:

YAML
   tool_result:
     name: DeleteNode
     result:
       success: false
       data: null
       errors: []
       warnings: []
       notices: []
       requires_confirmation:
         message: "Confirm deletion of node 'q4_product_roadmap' (delete all subnodes)"
         confirmation_code: "7fa2c19e"

The agent then decides to confirm the action or not. If it does:

Agent --> DeleteNode (with the confirmation code this time):

YAML
   tool_call:
     name: DeleteNode
     arguments:
       node_codename: q4_product_roadmap
       subnode_action: DeleteAll
       confirmation_code: "7fa2c19e"

The mechanism for generating the confirmation codes themselves is quite interesting, it is ENTIRELY STATELESS. Using HMAC (hmac) and a secret code, combining the hash of the inputs and the secret code creates a unique code. The code is always unique per set of inputs. Thinking it over, including the unique UUID of the node in the hash might be beneficial, or another narrowing hash. DefaultHasher is not always stable across rust versions, this is fine for this usecase and may be slightly beneficial to prevent reuse of old confirmation codes and staying up-to-date with the current preferred hasher.

Rust
/// Generate a confirmation code for the given inputs
///
/// The code is derived from:
/// 1. A hash of the inputs (stable representation)
/// 2. The server secret (from CONFIRMATION_SECRET env var)
///
/// This ensures the same inputs always produce the same code,
/// but different inputs produce different codes.
pub fn confirmation_code(inputs: &impl Hash) -> String {
    // Hash the inputs to get a stable representation
    let mut hasher = DefaultHasher::new();
    inputs.hash(&mut hasher);
    let input_hash = hasher.finish();

    // Create HMAC with secret
    let mut mac = HmacSha256::new_from_slice(&get_secret()).expect("HMAC can take key of any size");

    // Include the input hash in HMAC
    mac.update(&input_hash.to_be_bytes());

    // Return first 8 hex chars as the code
    hex::encode(mac.finalize().into_bytes())[..8].to_string()
}

The next thing is pagination, so:

Agent --> SemanticSearch:

YAML
   tool_call:
     name: SemanticNodeSearch
     arguments:
       query: "research about improving customer onboarding"
       include_messages: true
       node_types: null
       page: 1
       limit: 3

SemanticSearch --> Agent:

YAML
   tool_result:
     name: SemanticNodeSearch
     result:
       success: true
       data:
         results:
           - node_id: 101
             codename: customer_onboarding_research
             name: Customer Onboarding Research
             node_type:
               Pattern:
                 occurrences: [...]
             similarity: 0.861
             node_similarity: 0.84
             message_similarity: 0.91

           - node_id: 138
             codename: activation_interview_findings
             name: Activation Interview Findings
             node_type: Index
             similarity: 0.822
             node_similarity: 0.81
             message_similarity: 0.85

           - node_id: 204
             codename: onboarding_configuration_friction
             name: Onboarding Configuration Friction
             node_type:
               Fact:
                 origin: Subjective
                 citation:
                   Other:
                     title: "..."
                     raw: "..."
             similarity: 0.785
             node_similarity: 0.77
             message_similarity: 0.82

         page: 1
         per_page: 3
         total_pages: 2
         total_count: 5

       errors: []
       warnings: []
       notices: []

And to paginate, simply set page=2,

Agent --> SemanticSearch:

YAML
   role: assistant
   tool_call:
     name: SemanticNodeSearch
     arguments:
       query: "research about improving customer onboarding"
       include_messages: true
       node_types: null
       page: 2
       limit: 3

I'll show how this works internally in Part 2.

Something shocking

In my testing, Grok models (particularly Grok 4 - if you look closely at the screenshot of the webapp, that is actually the default model) were the best at making tool calls and understanding this system. Other models at the time (such as Gemini 2.5 Pro) struggled significantly with the syntax. This is an interesting observation because at the time Grok models were considered worse than Google or Anthropic models.

Conclusion

This is the conclusion to Part 1 of this doc. I will make part 2 soon, discussing Recursive CTEs, pgvector and dig deeper into some interesting decisions I made with PostgreSQL tables in an attempt to structurally prevent malformed data (rather than require loose validation - although if I had to resort to it SeaORM's models made it possible at the data-layer, I'll talk about that too). I'll also explain the hierarchical chats & reasoning behind it, as well as how I integrated Remote MCPs via a Manager Singleton that handles lifetime & MCP deduplication as well as authentication. It'll include how I integrated BYOK with any OpenAI-compatible API key using AutoAgents. Lastly, I'll explain how I combined AsyncAPI and OpenAPI for a strongly typed API between Rust & TypeScript both with WebSockets and REST.

Footnotes

Footnotes

  1. https://blog.voyageai.com/2026/01/15/voyage-4/ ↩