Search Strategy
How Gitlas knowledge stays findable — from plain grep to semantic search to graph-based discovery.
Principles
- No lock-in — every phase builds on the previous one; you can stop anywhere.
- Local-first — no cloud dependency at any phase.
- Progressive — start with zero extra infra, add capability as needed.
Phase 0 — Built-in (no extra services)
| Layer | Tool | What it does |
|---|---|---|
| Published site | Quartz FlexSearch | Full-text keyword search over the static site |
| AI agent | grep / ripgrep | Regex search across Markdown files |
| Human | File tree | Direct filesystem navigation |
Cost: zero. Works out of the box.
Limitation: keyword-only, no semantic understanding, no cross-file relationships beyond what the file tree shows.
Phase 1 — Semantic Search
Add local embedding-based search for the AI agent. All candidates share the same pattern: Ollama embeddings + vector store + MCP server.
Option A: sqmd (recommended)
| Property | Detail |
|---|---|
| Backend | LanceDB (embedded, no separate service) |
| Chunking | remark AST — section-aware, respects Markdown heading structure |
| Search | Hybrid (BM25 + vector + RRF fusion) |
| MCP | Built-in |
| Embeddings | Transformers.js ONNX (default) or Ollama |
| Incremental | SHA-256 fingerprinting, filesystem watcher |
Why: No external DB, natively understands Markdown AST, single binary-style dependency.
Option B: Mnemos
github.com/Tanush1912/mnemos-mcp
| Property | Detail |
|---|---|
| Backend | PostgreSQL + pgvector |
| Chunking | Fixed-size with overlap |
| Search | Vector similarity |
| MCP | Built-in |
| Embeddings | Ollama |
| Incremental | SHA-256 hashing |
Why: Multi-collection isolation, good if you already run Postgres.
Option C: GNO
| Property | Detail |
|---|---|
| Backend | Own engine (SQLite + vector index) |
| Chunking | Markdown-aware |
| Search | Hybrid (BM25 + vector + reranker) |
| MCP | Built-in |
| Embeddings | Local ONNX models (bge-m3) |
| Incremental | SHA-256 tracking |
| Bonus | Built-in Web UI, wiki backlinks, knowledge graph viz |
Why: Most feature-rich out of the box; includes a visual graph explorer and document editor.
Phase 2 — Knowledge Graph
Build a browsable graph of how pages relate to each other.
Lightweight (zero extra infra)
Quartz already renders a graph from [[wikilinks]] and frontmatter tags.
Combined with backlinks in the page footer, this gives a visual map of
the knowledge base with no additional service.
Structured graph (optional)
If the knowledge base grows large and relationship-aware queries matter:
- Build a graph from frontmatter (
tags,category,status) - Add
[[wikilink]]edges between pages - Optionally push to a lightweight graph store (e.g. NetworkX JSON, or KuzuDB for larger graphs)
- Expose via a custom MCP tool (
search_related,find_path)
Graphify (github.com/Graphify-Labs/graphify) is the obvious choice here — but it is optimized for code (36 tree-sitter grammars), so its value is limited in a pure Markdown knowledge base. Revisit if the atlas later includes source code.
Phase 3 — AI-Driven Maintenance
Proactive quality-of-knowledge checks:
- Orphan detection — pages with no incoming links
- Stale content — pages not updated in >N months
- Broken links —
[[wikilinks]]and file references that point nowhere - Contradiction flagging — two pages claiming different facts about the same topic
These are straightforward to implement as custom MCP tools or a periodic script (e.g. pre-commit hook or scheduled CI job).
Recommendation
| Phase | What | When |
|---|---|---|
| 0 | Quartz FlexSearch + ripgrep | Now |
| 1 | sqmd MCP server | First upgrade |
| 2 | Quartz wikilink graph | Alongside content growth |
| 3 | AI maintenance checks | When content exceeds ~50 pages |
Graphify is not recommended for the current scope (pure Markdown knowledge). If the repo later includes source code, reassess.