Skip to content

Optimization.md

Latest commit

 

History

History
274 lines (263 loc) · 9.47 KB

Optimization.md

File metadata and controls

274 lines (263 loc) · 9.47 KB
Raw

This is the Final Architecture Plan. It represents the transition to a 100% self-hosted, privacy-first Enterprise Memory System. We are removing OpenAI entirely. We are moving the "Mathematics" (Embeddings/Reranking) to the VPS and the "Reasoning" to the Minisforum. The Architecture: "The Private Enterprise Stack"

  • The Brain (Reasoning): Minisforum MS-S1 Max (10.0.0.10) running GPT-OSS 120B.
  • The Search Appliance (Retrieval): VPS (10.0.0.1) running Qdrant + BGE-M3 + BGE-Reranker.
  • The Orchestrator: n8n (VPS) connecting them via localhost and VPN. Phase 1: VPS Deployment (The Search Appliance) Location: VPS (10.0.0.1) Task: Deploy the Database, the Embedder, and the Reranker. Run these three commands on your VPS.

  1. Qdrant (The Vector Database) docker run -d --restart always
    --name qdrant
    -p 6333:6333
    -v /opt/qdrant-data:/qdrant/storage
    qdrant/qdrant

  2. BGE-M3 (The Embedder) Replaces OpenAI text-embedding-3-small. docker run -d --restart always
    --name embeddings
    -p 8081:80
    -v /opt/embeddings-data:/data
    ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
    --model-id BAAI/bge-m3
    --dtype float16

  3. BGE-Reranker-v2 (The Ranker) Filters the results for accuracy. docker run -d --restart always
    --name reranker
    -p 8080:80
    -v /opt/reranker-data:/data
    ghcr.io/huggingface/text-embeddings-inference:cpu-1.5
    --model-id BAAI/bge-reranker-v2-m3
    --dtype float16

Phase 2: Minisforum Setup (The Brain) Location: Home PC (10.0.0.10) Task: Load the heavy logic model.

  • Open LM Studio.
  • Search: openai/gpt-oss-120b (or Qwen/Qwen2.5-72B-Instruct).
  • Download: The Q4_K_M.gguf version (Best balance for 128GB RAM).
  • Right Panel Settings:
    • GPU Offload: MAX (All layers).
    • Context Length: 32768 (32k).
    • Server Port: 1234.
  • Click: "Start Server". Phase 3: The "Retrieval" Workflow (n8n) Location: VPS n8n Task: Create the API that Open WebUI will call. This workflow replaces the standard n8n nodes with HTTP Requests to your local Docker containers. This gives you total control. Copy/Paste this JSON into n8n: { "name": "Private Memory Retrieval (BGE-M3 + GPT-OSS)", "nodes": [ { "parameters": { "httpMethod": "POST", "path": "retrieve-memory", "options": {} }, "id": "webhook", "name": "Webhook", "type": "n8n-nodes-base.webhook", "typeVersion": 1, "position": [460, 300] }, { "parameters": { "method": "POST", "url": "http://10.0.0.10:1234/v1/chat/completions", "sendBody": true, "bodyParameters": { "parameters": [ { "name": "model", "value": "gpt-oss-120b" }, { "name": "messages", "value": "={{ [{"role": "system", "content": "You are a query optimizer. Convert the user input into a specific keyword search query for a database. Output ONLY the query string."}, {"role": "user", "content": $json.body.query}] }}" }, { "name": "temperature", "value": 0.1 } ] }, "options": {} }, "id": "rewrite", "name": "GPT-OSS (Rewrite)", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4, "position": [660, 300] }, { "parameters": { "method": "POST", "url": "http://localhost:8081/embed", "sendBody": true, "bodyParameters": { "parameters": [ { "name": "inputs", "value": "={{ $json.choices[0].message.content }}" } ] }, "options": {} }, "id": "embed", "name": "Local Embed (BGE-M3)", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4, "position": [860, 300] }, { "parameters": { "method": "POST", "url": "http://localhost:6333/collections/memory_collection/points/search", "sendBody": true, "bodyParameters": { "parameters": [ { "name": "vector", "value": "={{ $json[0] }}" }, { "name": "limit", "value": 25 }, { "name": "with_payload", "value": true } ] }, "options": {} }, "id": "qdrant-search", "name": "Qdrant Search", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4, "position": [1060, 300] }, { "parameters": { "jsCode": "// Format for Reranker\nconst query = $('GPT-OSS (Rewrite)').first().json.choices[0].message.content;\nconst hits = $input.all()[0].json.result;\n\n// Map Qdrant results to text array\nconst texts = hits.map(hit => hit.payload.text);\n\nreturn { json: { query, texts } };" }, "id": "format-rerank", "name": "Format Data", "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [1260, 300] }, { "parameters": { "method": "POST", "url": "http://localhost:8080/rerank", "sendBody": true, "bodyParameters": { "parameters": [ { "name": "query", "value": "={{ $json.query }}" }, { "name": "texts", "value": "={{ $json.texts }}" } ] }, "options": {} }, "id": "rerank", "name": "Local Rerank", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4, "position": [1460, 300] }, { "parameters": { "jsCode": "// Filter Top 5 High Precision\nconst scores = $input.first().json;\nconst docs = $('Format Data').first().json.texts;\nconst top = scores\n .map((s, i) => ({ score: s.score, text: docs[s.index] }))\n .sort((a, b) => b.score - a.score)\n .slice(0, 5)\n .map(x => x.text)\n .join("\n---\n");\nreturn { json: { results: top } };" }, "id": "filter", "name": "Filter Best", "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [1660, 300] }, { "parameters": { "respondWith": "json", "responseBody": "={{ $json }}", "options": {} }, "id": "respond", "name": "Respond", "type": "n8n-nodes-base.respondToWebhook", "typeVersion": 1, "position": [1860, 300] } ], "connections": { "Webhook": { "main": [[{ "node": "GPT-OSS (Rewrite)", "type": "main", "index": 0 }]] }, "GPT-OSS (Rewrite)": { "main": [[{ "node": "Local Embed (BGE-M3)", "type": "main", "index": 0 }]] }, "Local Embed (BGE-M3)": { "main": [[{ "node": "Qdrant Search", "type": "main", "index": 0 }]] }, "Qdrant Search": { "main": [[{ "node": "Format Data", "type": "main", "index": 0 }]] }, "Format Data": { "main": [[{ "node": "Local Rerank", "type": "main", "index": 0 }]] }, "Local Rerank": { "main": [[{ "node": "Filter Best", "type": "main", "index": 0 }]] }, "Filter Best": { "main": [[{ "node": "Respond", "type": "main", "index": 0 }]] } } }

Critical Note: In the "Qdrant Search" node, I used the collection name memory_collection. Ensure you create this collection in Qdrant first with a vector size of 1024 (since BGE-M3 uses 1024 dimensions). Phase 4: The "Ingestion" Workflow (IMPORTANT) Since you switched embedding models, your old data is incompatible. You must re-ingest your data using BGE-M3. You need a second n8n workflow called "Save to Memory".

  • Trigger: Webhook (POST save-memory).
  • Action: HTTP Request (to localhost:8081/embed) -> Get Vector.
  • Action: HTTP Request (to localhost:6333) -> Upsert Point (Save Vector + Payload). If you need the code for this Ingestion workflow, let me know. Otherwise, I assume you can replicate the "Embed" logic from the Retrieval workflow above. Phase 5: Open WebUI Function (Frontend) This connects your Chat Interface to the new Private Stack. Go to Open WebUI -> Functions -> Create: import requests import json from pydantic import BaseModel, Field

class Tools: class Valves(BaseModel): N8N_URL: str = Field( default="http://10.0.0.1:5678/webhook/retrieve-memory", description="URL of the Private Retrieval Workflow" )

def __init__(self):
    self.valves = self.Valves()

def search_private_memory(self, query: str) -> str:
    """
    Search Essam's Private Knowledge Base.
    Use this for retrieving confidential project data, node/spoke info, or technical documentation.
    
    :param query: The question or topic to search for.
    :return: Context from the private Qdrant database.
    """
    try:
        # Post to n8n
        response = requests.post(
            self.valves.N8N_URL, 
            json={"query": query}, 
            timeout=15
        )
        response.raise_for_status()
        
        # Parse
        data = response.json()
        return f"RETRIEVED CONTEXT:\n{data.get('results', 'No data found.')}"

    except Exception as e:
        return f"Memory Error: {str(e)}"

Summary of Operations

  • User Asks: "What is the status of Node A?"
  • Open WebUI: Calls search_private_memory.
  • n8n (VPS):
    • Asks Minisforum (GPT-OSS 120B) to rewrite the query.
    • Asks VPS (BGE-M3) to vectorize it.
    • Asks VPS (Qdrant) to find 25 matches.
    • Asks VPS (BGE-Reranker) to find the best 5.
  • Open WebUI: Displays the answer generated by your local model using that private context.