Storage catalogue¶
agentgrep keeps an explicit catalogue of every on-disk store it knows
about, modelled as Pydantic
StoreDescriptor rows aggregated under one
StoreCatalog. The catalogue is descriptive:
it documents where each agent’s data lives and what the records look
like. Coverage decisions — whether agentgrep searches, inventories, or
only documents a store — are captured per-row so adding storage
knowledge does not automatically expand default prompt search.
The catalogue is the single source of truth that downstream adapters
consume. When upstream renames a path or changes a record shape, the
fix is to update one
StoreDescriptor and bump the
catalogue version; adapters pick the new metadata up automatically.
Catalog summary¶
- antigravity-cli:
8- antigravity-ide:
9- claude:
41- codex:
38- cursor-cli:
15- cursor-ide:
4- gemini:
11- grok:
17- opencode:
9- pi:
11- vscode:
4- windsurf:
6
- catalog_only:
97- default_search:
23- inspectable:
41- private:
12
- app_state:
79- cache:
16- instruction:
15- persistent_memory:
8- plan:
8- primary_chat:
16- prompt_history:
8- source_tree:
7- supplementary_chat:
14- todo:
2
- json_array:
2- json_object:
49- jsonl:
19- md_frontmatter:
11- opaque:
28- protobuf:
8- sqlite:
20- text:
36
Why a catalogue¶
Three reasons we did not bake paths into the adapters:
Provenance. Each row carries an
observed_versionandobserved_atstamp. A reader can tell at a glance whether the schema notes are still current or stale.Drift. Codex renames
history.jsonl, Cursor adds a CLI agent layout, Gemini reorganises itstmp/tree. With paths catalogued centrally, those changes diff cleanly in code review.Overlap. Several stores live in adjacent paths but play different roles — Codex
history.jsonl(user prompts only) vs.sessions/*.jsonl(full per-thread transcripts); Geminitmp/<hash>/chats/(live) vs.history/<timestamp>/(post-retention archive). Thedistinguishes_fromfield on each descriptor names the sibling and explains the difference.
Reading a descriptor¶
from agentgrep.store_catalog import CATALOG
claude_session = CATALOG.by_id("claude.projects.session")
claude_session.path_pattern
# '${HOME}/.claude/projects/<encoded_project>/<session_uuid>.jsonl'
Path patterns use ${HOME} and ${<ENV>} tokens; resolving them
against a concrete environment is the consumer’s job, so the catalogue
stays portable. env_overrides lists the env vars that change the
root (Claude respects CLAUDE_CONFIG_DIR; Codex respects
CODEX_HOME and CODEX_SQLITE_HOME; Gemini respects
GEMINI_CLI_HOME).
Coverage levels¶
Every descriptor has an effective coverage level:
Coverage |
Meaning |
|---|---|
|
Eligible for search without inventory opt-in; effort and scope still gate its role. |
|
May join exhaustive or broad-scope search and explicit inventory. |
|
Search never opens it. The path and schema remain documented, and a row may expose a conservative structural sample for explicit inspection. |
|
The store is documented but intentionally not enumerated from disk. |
Coverage sets search eligibility; effort and scope decide which eligible roles are opened. This distinction lets the catalogue describe auth files, runtime logs, shell snapshots, and file-history caches without making them part of ordinary prompt search.
Kinds of storage an agent keeps¶
Coding agents keep far more than transcripts, and StoreRole is how the
catalogue names each kind:
Role |
What it holds |
|---|---|
|
Settings, registries, session indexes, runtime markers |
|
The canonical transcript of a session |
|
Derived data reconstructible from a source of truth |
|
Skills, rules, and memory files that steer future sessions |
|
Subagent turns, delegations, secondary transcripts |
|
Long-lived notes an agent writes to itself |
|
Plan-mode Markdown |
|
Append-only logs of what you typed |
|
Working trees the agent created |
|
Task lists |
How many rows each kind carries is in the By role card of the
catalog summary above, rendered from the catalogue
itself so it cannot drift. app_state holds more than any other kind,
which is the honest signal that it is the catch-all: it absorbs anything
that is neither chat nor instruction. Splitting it is a future concern,
not a defect.
Kinds no row models yet¶
Three kinds exist on disk for several agents and have no representation here at all. They are recorded so a future adapter does not rediscover them as if they were chat:
Vector and embedding indexes. Agents increasingly ship a semantic
index beside their text one. Grok’s memory/<project>/index.sqlite
carries a sqlite-vec chunks_vec table alongside chunks_fts; the
Codeium and Windsurf trees carry an embedding_database.sqlite with
embeddings, snippets, and code_context_items. These index the
user’s source code, not their prompts, so they are a privacy surface
worth naming even though agentgrep has no reason to read them.
Checkpoints, snapshots, and rewind state. Several agents keep the
file state needed to undo their own edits: Gemini’s shadow-git roots
under history/, Grok’s per-session rewind_points.jsonl, Cursor’s
bare repositories under snapshots/. There is no snapshot role, so the
few rows that do cover this land under source_tree or cache.
Telemetry. Usage counters and analytics payloads — statsig/
directories, Grok’s memtrace/, per-agent analytics JSON. No row names
any of them.
Version detection strategies¶
Discovery payloads include a version_detection object for each source
agentgrep can enumerate. The object records the app version when local
metadata exposes one, the data-shape version, the strategy used, the
confidence level, and a short evidence string.
The strategies are:
Strategy |
Meaning |
|---|---|
|
The source itself carries a version field, such as a session metadata record. |
|
The file name, record keys, table names, or SQLite suffix identify the data shape. |
|
A local version file provides app-version context without spawning the upstream CLI. |
|
No concrete source evidence was available, so the catalog observation stamp is reported as a low-confidence fallback. |
The concrete data shape is authoritative. If a modern app-version hint coexists with an old unmigrated file, agentgrep parses the file by its own shape. See ADR 0001: Storage version detection for the full decision.
Discovery callers choose how much version evidence they need. Normal
grep, search, and find paths use metadata-free discovery so the
planner can prune source handles before opening files for evidence.
Inventory and MCP source-listing surfaces keep shape detection enabled,
while catalog-only detail remains available for callers that want a
cheap, low-confidence metadata stamp.
Search callers also narrow discovery by descriptor role before walking the
filesystem. Prompt effort enumerates only prompt_history rows. Targeted
selection searches those prompt sources first, then opens only exact
conversation sources resolved from proof-bearing Codex, Claude Code, Grok, or
Antigravity CLI evidence; it does not enumerate every chat source. Exhaustive
discovery adds the eligible primary_chat and supplementary_chat rows.
Conversation scope admits those chat roles plus the explicit
conversation-content app-state allowlist, while all scope admits prompt and
conversation record kinds.
Targeted routing has its own positive conversation_limit, currently 25
attempts by default. That policy is unmeasured and independent of result
limit; no result-limit migration to 25 occurred. Invariant agent and project
predicates run before candidate grouping and the bound. Unresolved or ambiguous
selected locators consume an attempt and are not backfilled. Locator traversal
is separately request-bounded, and routing evidence never establishes a result.
Stores by agent¶
Claude Code¶
observed_version: claude-code v2.1.226 (observed 2026-08-08).
Claude honours CLAUDE_CONFIG_DIR, falling back to ${HOME}/.claude.
Its global prompt-history audit log lives at
${CLAUDE_CONFIG_DIR or ${HOME}/.claude}/history.jsonl and is parsed by
claude.history_jsonl.v1. It stores the user-facing display text,
Unix-millisecond timestamp, project, sessionId, and
pastedContents; content-addressed text pastes resolve through
paste-cache/<contentHash>.txt when present.
Claude’s primary chat record lives at
${HOME}/.claude/projects/<encoded_project>/<session_uuid>.jsonl. The
file format is JSONL with multiple record types per line —
type: "user", type: "assistant", type: "attachment",
type: "permission-mode". Sub-agent dispatches nest under
<session_uuid>/subagents/, share the same parser, and are exposed as
the distinct runtime store claude.projects_subagents. __store.db,
session memory, project auto-memory, CLAUDE.md, tasks, todos, plans,
skills, legacy commands, project-local .claude instructions, plugin
instructions, and teams are inspectable but remain outside default
search because they either duplicate transcripts, represent derived
state, or steer future sessions. Tasks and todos emit their subject,
content, and description fields; teams emit team descriptions and
member prompts. Settings and app-state JSON expose only top-level key
and type summaries, so raw values such as env vars are not indexed.
Debug output and shell snapshots expose metadata-only file summaries.
Backups, uploads, file history, context/security state, credentials,
session environment, and cache payloads are catalogued or private
according to sensitivity.
Claude source version detection uses embedded_metadata for transcript
version fields, shape_inference for history records with display,
timestamp, and project, task/todo/team JSON keys, settings and
app-state key summaries, plugin manifests and hook event names,
instruction Markdown paths, file metadata summaries, and
catalog_observation as the fallback. Project-local discovery is
bounded to roots already present in Claude transcript metadata.
Cursor CLI and Cursor IDE¶
Cursor is modelled as two separate agents — cursor-cli (the
cursor-agent terminal binary) and cursor-ide (the desktop app) —
because they have disjoint data homes and on-disk formats.
cursor-clispans two home directories. The original${HOME}/.cursor/tree holds the JSONL transcripts (cursor_cli.transcripts_jsonl.v1, Anthropic-style{role, message.content[]}with no native timestamp, so agentgrep backfills the file mtime) and the AI-tracking SQLite summaries. The newer lowercase${HOME}/.config/cursor/home holdsprompt_history.json— a flat JSON array of typed prompts parsed bycursor_cli.prompt_history_json.v1, Cursor’s prompt-history store — and the per-session chatchats/<hash>/<uuid>/store.db. The chat store holds content-addressed protobuf blobs with no published schema;cursor_cli.chats_protobuf.v1walks the wire format best-effort and is inspectable (opt-in) rather than searched by default, since it overlaps the cleaner transcripts.cursor-cli.worktreesis catalogued withrole=SOURCE_TREEandsearch_by_default=Falseso the adapter never indexes multi-gigabyte git working trees as chat history.cursor-cli.skillscovers theSKILL.mddefinitions under~/.cursor/skills/and~/.cursor/skills-cursor/as inspectable instruction text.cursor-ideis parsed bycursor_ide.state_vscdb_modern.v1/cursor_ide.state_vscdb_legacy.v1via VS Code-stylestate.vscdbSQLite.cursor-ide.state_vscdbcovers the global database andcursor-ide.workspace_statecovers the per-workspaceworkspaceStorage/<hash>/state.vscdb. The global database reads known prompt/chat keys, while per-workspace stores contributeorigin.cwd_hashfrom the directory hash and can addorigin.cwdfrom siblingworkspace.json. On WSL, discovery also probes the Windows-host mount under/mnt/c/Users(overridable viaAGENTGREP_WSL_USERS_ROOT) for a Windows-side Cursor editing a WSL project, mirroring the VS Code backend; see ADR 0009: Cross-host discovery and remote-workspace path mapping.
Codex¶
observed_version: codex-cli 0.147.0 (observed 2026-08-08).
Codex honours CODEX_HOME for primary files. SQLite files resolve
through CODEX_SQLITE_HOME, then sqlite_home in config.toml, then
CODEX_HOME.
Schemas are pinned directly to the upstream Rust types:
JSONLhistory.jsonl→HistoryEntry { session_id: String, ts: u64, text: String }(codex-rs/message-history/src/lib.rs:56-60).Per-thread
sessions/YYYY/MM/DD/rollout-…jsonl→ tagged enumRolloutItemwith variantsSessionMeta,ResponseItem,Compacted,TurnContext,EventMsg(codex-rs/protocol/src/protocol.rs:2929).Legacy root
sessions/rollout-*.json→ JSON object withsessionmetadata and anitemsarray carrying message-like records.
The _N.sqlite files belong to the Codex CLI, not Cursor. Known
SQLite stores are state_5.sqlite, logs_2.sqlite,
memories_1.sqlite, goals_1.sqlite, thread_history_1.sqlite, and
queue_1.sqlite. Prompt-bearing fields such as
threads.first_user_message, threads.preview, memory summaries, and
goal objectives are inspectable storage rather than default search.
The numeric suffix is a schema generation, so a bumped file name is a
rename agentgrep must follow rather than a variant it can glob past.
agent_jobs no longer exists — state_5.sqlite reached migration 46
and migration 42 is named drop agent jobs — so job instructions are
no longer a Codex storage surface at all.
Codex source version detection uses shape_inference for
history.jsonl, legacy history.json, legacy root rollout JSON,
session_index.jsonl, external import ledgers, memory Markdown,
config TOML, project config TOML, app-state JSON summaries, plugin
manifests, plugin marketplace metadata, hook event names, instruction
Markdown/rule paths, file metadata summaries, and SQLite suffixes. It
uses embedded_metadata for session cli_version, and
version_check for models_cache.json.client_version app-version
context. Project-local .codex discovery is bounded to roots already
present in Codex session metadata.
Gemini CLI¶
observed_version: gemini-cli v0.54.4 (observed 2026-08-08); types
pinned at HEAD 927170fc. Three adapters cover the three on-disk
shapes:
gemini.tmp_chats_jsonl.v1parsestmp/<project_hash>/chats/session-*.jsonl. Each file opens with aSessionMetadataRecord(sessionId,projectHash,startTime,lastUpdated,kind); subsequent lines areMessageRecordturns interleaved withMetadataUpdateRecordupdates ({$set: {…}}). Real files surfacetypevaluesuserandgemini; upstream types also declareinfo/error/warningplusRewindRecordandPartialMetadataRecord, but those records did not appear in sampling.gemini-typed turns whosecontentis empty have their searchable text drawn fromthoughts[*].subject/descriptionandtoolCalls[*].name/description, joined into one record per turn.gemini.tmp_chats_legacy_json.v1parses pre-Feb 2026tmp/<project_hash>/chats/session-*.jsonsingle-file sessions. Upstream still reads this shape via theisLegacyRecorddiscriminator atchatRecordingService.ts:1041; the legacy file holds session metadata at the top level and the full conversation under amessagesarray.gemini.tmp_logs_json.v1parsestmp/<project_hash>/logs.json— a flat JSON array ofLogEntryrecords (user-prompt audit log).
The gemini.memory row covers ~/.gemini/GEMINI.md, the global
user-authored context file injected into sessions — the Gemini
analogue of Claude’s CLAUDE.md, parsed by gemini.memory_text.v1 as
an inspectable (opt-in) store rather than searched by default.
Gemini’s
sessionCleanup.ts
hard-deletes expired sessions via fs.unlink(), so an expired session
is gone rather than archived. ~/.gemini/history/ is a different
tree and does exist: it is the checkpointing shadow-git root, one
directory per project holding a .project_root marker and, where
checkpointing ran, a full .git/. It holds file snapshots, not
transcripts, and no store row covers it. The Antigravity files some
installs carry under
~/.gemini/antigravity/conversations/ are written by the
Antigravity IDE,
a separate Google product — Gemini CLI only detects Antigravity as
an IDE launcher and does not read or write the protobuf
conversation files. They are documented as the separate
Antigravity IDE and Antigravity CLI
backends, not as Gemini adapters.
The project_hash was sha256(absolute_project_root), and
gemini_project_hash() mirrors that
derivation. Newer releases name most project directories after a
slugified basename or a run stamp instead, so the helper answers
“which Gemini directory holds this repo?” only for the hash-named
share of a tree — 38 of 142 directories in the sample this catalogue
was observed against. projects.json is the registry that maps every
absolute project root to its directory name under both tmp/ and
history/, and it is the reverse lookup that still covers the whole
tree.
Because a slug is not a digest, gemini.tmp_logs and its siblings
publish origin.cwd_hash only when the directory name has a digest
shape. See ADR 0001: Storage version detection for why the concrete
shape wins over an app-version hint.
Grok CLI¶
observed_version: grok 1.0.0 (observed 2026-08-08).
Grok stores data under ${GROK_HOME or ${HOME}/.grok}/sessions/
using URL-encoded absolute project paths as directory keys
(e.g. %2Fhome%2Fd%2Fwork%2Fpython%2Fproj). Four adapters cover
the searchable store shapes:
grok.prompt_history_jsonl.v1parses per-projectsessions/<project>/prompt_history.jsonl. Each line is a{ timestamp, session_id, prompt, is_bash }audit record — one entry per user prompt, analogous to Codex’shistory.jsonl.grok.sessions_jsonl.v1parses per-sessionsessions/<project>/<uuid>/chat_history.jsonl. Lines carry atypefield (system/user/assistant/tool_use/tool_result) and acontentfield (string or content-blocks array). All record types are emitted.grok.session_search_sqlite.v1parses the globalsessions/session_search.sqliteFTS5 index. Tablesession_docshassession_id,cwd,updated_at(unix seconds),title(generated), andcontent(full-text indexed body).grok.subagents_json.v1parses per-subagentsessions/<project>/<uuid>/subagents/<subagent>/meta.json. The delegatedpromptis the only persisted record of the subagent, so it is emitted as supplementary-chat content, parity with the Claude and Cursor CLI subagent stores.
The grok.plans row covers per-session plan.md plan-mode Markdown,
and grok.memory covers the flat and per-project
memory/**/MEMORY.md subtree — both inspectable (opt-in) like
claude.plans. Documentary-only entries cover per-session
system_prompt.txt, prompt_context.json, hunk_records.jsonl
(edit attribution), updates.jsonl (ACP stream), terminal/*.log
(tool stdout), plus events, summaries, logs, worktrees, and config —
all carrying no user prompt payload and catalogued with
search_by_default=False or deferred.
Pi¶
observed_version: pi v0.84.1 (observed 2026-08-08).
Pi (earendil-works) stores each conversation as one append-only JSONL
file under ${PI_CODING_AGENT_DIR or ${HOME}/.pi/agent}/sessions/,
grouped by working directory (--<encoded_cwd>--, leading slash
stripped and / \ : replaced by -). It keeps no separate
prompt-history log or index for ordinary sessions, so their searchable
surface comes directly from the JSONL transcript:
pi.sessions_jsonl.v1parsessessions/--<cwd>--/<ts>_<uuid>.jsonl. Line one is atype:"session"header (versionmay be absent in v1 files); each later line is aSessionEntrytagged union.messageentries carry an LLM message (roleuser / assistant / toolResult,contentstring or content-blocks; assistant turns carrymodel), whilecompaction/branch_summarysummaries andsession_infonames are emitted as history text. User turns surface as prompts via the shared role-to-kind mapping.pi.context_mode_sqlite.v1parses the optional per-project~/.pi/context-mode/sessions/<project_hash>.dbstore. Itssession_eventsrows carry role, intent, decision, tool-call, file-read, blocker-resolution, and data events. This app-state store joins explicit conversation and all scopes, not deep prompt search.
Discovery resolves two roots: PI_CODING_AGENT_DIR (the agent dir,
default ~/.pi/agent) and the optional PI_CODING_AGENT_SESSION_DIR,
which holds session files flat with the cwd recovered from the header.
Context-mode discovery separately checks ~/.pi/context-mode/sessions/.
Documentary-only entries cover settings, auth (private credentials), models, themes, tools, managed binaries, prompt templates, the debug log, and the npm extension install root.
OpenCode¶
observed_version: opencode v1.18.15 (observed 2026-08-08).
OpenCode (anomalyco/opencode) stores conversations in a single SQLite
database under ${XDG_DATA_HOME or ${HOME}/.local/share}/opencode/,
unlike the JSONL-transcript backends:
opencode.db_sqlite.v1parses theopencode.dbSQLite database. It joins the relationalpart → message → sessiontables: each text-bearingpartrow becomes a record whosekindcomes from the joined messagerole(user→ prompt, else history). Searchable text ispart.dataof typetext/reasoning(thetextfield) andsubtask(theprompt); the sessiontitle,directory, and the messagemodel/timestamp are attached. Message times are unix-milliseconds, normalized to ISO-8601.
Discovery resolves the data root via XDG_DATA_HOME (default
~/.local/share) plus the opencode segment and finds opencode.db by
filename — not a glob, so the binary SQLite file bypasses the text
prefilter. An absolute OPENCODE_DB value is discovered as that exact
file, so channel installs are reachable by pointing OPENCODE_DB at
their opencode-<channel>.db.
OpenCode’s v2 event-sourced tables share the same opencode.db file.
session_input, session_message, and todo are still empty on
stable installs, but event/event_sequence are now populated and
mirror the part transcript rather than replacing it — every event
aggregate resolves to a session that already has part rows — so the
canonical transcript stays in session/message/part and the event
tables are left unsearched to avoid duplicate hits. The secret-bearing
account/credential tables are present but never enumerated.
OpenCode also writes a prompt-history log outside its data root, at
${XDG_STATE_HOME or ${HOME}/.local/state}/opencode/prompt-history.jsonl.
Each line is {input, parts, mode} with no timestamp and no session
id. It is not a duplicate of the database: prompts recalled there can
be absent from opencode.db entirely.
Documentary-only entries cover the legacy per-file JSON layout, config, auth (private credentials), snapshots, the repo cache, logs, and tool output.
VS Code (GitHub Copilot Chat)¶
observed_version: VS Code 1.132.0 (observed 2026-08-08).
VS Code’s built-in Copilot Chat stores readable JSON transcripts under
the workbench User/ directory, covered across the Code,
Code - Insiders, VSCodium, and Code - OSS editions:
vscode.chat_sessions_json.v1parses one session object perworkspaceStorage/<hash>/chatSessions/<uuid>.json(and the windowlessglobalStorage/emptyWindowChatSessions/*.json). Eachrequests[]turn yields a user prompt frommessage.textand an assistant record from the no-kindMarkdownStringresponse parts joined; tool names come fromresult.metadata.toolCallRounds[].toolCalls[].name, and the epoch-millisecondtimestampis normalized to ISO-8601. The siblingworkspace.jsonfolderURI resolves the projectcwd, mapping avscode-remote://wsl+<distro>/<path>remote to its Linux path.vscode.inline_history_sqlite.v1reads theinline-chat-historykey of the globalstate.vscdbItemTable— a JSON array of Ctrl+I inline-edit prompts. Only that key is read, so thesecret://…auth keys in the same database are never enumerated.
Discovery resolves User/ directories per OS (Linux ~/.config/Code,
macOS ~/Library/Application Support/Code, Windows %APPDATA%/Code) and,
on WSL, also probes the Windows host mount under /mnt/c/Users because a
WSL-remote workspace stores its chat client-side on Windows.
VSCODE_APPDATA pins one Roaming directory and AGENTGREP_WSL_USERS_ROOT
overrides the Windows users mount. See
ADR 0009: Cross-host discovery and remote-workspace path mapping for the cross-host design.
Documentary-only entries cover the per-chat chatEditingSessions/ edit
snapshots (a byproduct of the transcripts, keyed by the same session
UUID) and the secret://… auth keys in state.vscdb (private
credentials, never enumerated).
Observed record shapes¶
An observed_version stamp says when someone last looked. It does not say
what they saw, so bumping one destroys the previous observation and schema
drift stays invisible until a human re-audits by hand.
docs/_observations/<agent>/<version>.toml records the observation itself.
Each manifest holds, per store, the record discriminator, the key set seen for
each of its values, and the tables and columns of any SQLite file. Two
manifests for the same agent at different versions diff as text, so “did Grok
stop writing timestamp?” is a question you answer by reading rather than by
re-opening the store.
Backend pages render those shapes beside each store card, and the build warns when a catalogue stamp and the newest manifest disagree. The warning names both repairs, because either side can be the stale one: re-observing is a no-op when the manifest is already ahead.
Refresh one agent with uv run scripts/observe_stores.py observe --agent grok,
or compare live disk against what is recorded with
uv run scripts/observe_stores.py check --agent all. Those are written inline
rather than as console blocks on purpose — the documentation suite executes
every console fence under docs/, and observe writes into the manifest tree.
Manifests carry schema only. Source counts and the unclaimed-path list describe the machine an observation ran on rather than the agent it observed, so they stay out of the rendered page.
Adding or updating a store¶
Edit the per-agent module under
src/agentgrep/store_catalog/(e.g.vscode.py). Stampobserved_versionandobserved_atagainst the version you actually inspected.Add an
upstream_ref(preferred) or asample_recordso future readers can verify the schema.If the new store overlaps a sibling, name it in
distinguishes_fromand explain the difference inschema_notes.Capture a redacted fixture under
tests/samples/<agent>/<store_id>/.Bump
catalog_versionin the same commit that changes descriptor shape.Re-observe the agent so the recorded shape matches the stamp — see Observed record shapes.
Run
just test-all.
See also¶
agentgrep.stores— model definitionsagentgrep.store_catalog— concrete registry