---
generator: "vaultspec-marketing discovery"
title: "Search and index your project - vaultspec-rag documentation"
description: "vaultspec-rag searches vault records, source code, and explicitly routed extracted documents by meaning."
url: "https://vaultspec.neve.md/docs/rag/search-and-index.html"
component: "vaultspec-rag"
source_repository: "https://github.com/nevenincs/vaultspec-rag"
source_path: "docs/search-and-index.md"
source_commit: "d126f083802311e30300d2c5e98f087806172a52"
date_modified: "2026-10-04T20:56:03+02:00"

---

[View as Markdown](<https://vaultspec.neve.md/docs/rag/search-and-index.md>)

# Search and index your project

vaultspec-rag searches vault records, source code, and explicitly routed extracted documents by meaning. This guide covers running searches and keeping each independent index current.

This guide assumes the workspace is installed and provisioned. If it isn’t, see the [installation guide](<https://vaultspec.neve.md/docs/rag/installation.html>) first. For how search and indexing fit together, see the [architecture overview](<https://vaultspec.neve.md/docs/rag/architecture.html>). To run searches against a background daemon instead of in-process, see [service mode](<https://vaultspec.neve.md/docs/rag/service-mode.html>).

Examples use the `uv run` prefix, which runs the command inside a project environment. If you installed vaultspec-rag as a standalone tool, drop the prefix and call `vaultspec-rag` directly; see the [installation guide](<https://vaultspec.neve.md/docs/rag/installation.html>) for lane selection.

## Run a search

Nothing is searchable until the index exists. If this is a new project, run [Build and refresh the index](<https://vaultspec.neve.md/docs/rag/search-and-index.html#build-and-refresh-the-index>) first.

How you phrase a query matters more than any filter here: pair a short description of the behavior with the concrete words the target would contain. [Writing a query](<https://vaultspec.neve.md/docs/rag/query-craft.html>) covers it properly.

Search defaults to relevant ADRs and source code, ranked together:

**Command**

```bash
uv run vaultspec-rag search "how does the watcher coalesce changes"
```

Use `--type vault` for all vault record types, or `--type combined` for all three indexes. `--doc-type plan` (or inline `type:plan`) overrides the default ADR filter while retaining code results. The one-line advisory names the available filters; check `source` and `doc_type` in JSON to verify each hit’s kind.

A vault result has three parts. The first line is the document path with the lines that hold the passage. The second line gives the document’s type, feature, status, date, related records, and the section the passage sits under. Then comes the passage itself, cut to its first lines in human output. `--scores` adds a relevance figure after the location, which a later section shows.

To search source code instead, add `--type code`:

**Command**

```bash
uv run vaultspec-rag search "gpu lock around the forward pass" --type code --max-results 3
```

This is the start of a real run against this repository. A code result is a path with a line range, then the passage from those lines. Long passages are trimmed here with `...`:

**Captured output**

```text
1. src/vaultspec_rag/embeddings.py:1089-1108
       def _encode_dense_bucket(
           self,
           bucket_texts: list[str],
           gpu_lock: threading.Lock | None,
           ...
2. src/vaultspec_rag/memory_probe.py:691-709
   @contextlib.contextmanager
   def cuda_forward_peak_capture() -> Generator[None]:
       """Bracket one model forward with a job-local peak capture.
       ...
3. src/vaultspec_rag/job_control.py:85-106
   ...
```

Search extracted documents independently with `--type document`, or allocate candidates across all three domains with `--type combined`:

**Command**

```bash
uv run vaultspec-rag search "quarterly retention assumptions" --type document
uv run vaultspec-rag search "where is this policy implemented" --type combined
```

`docs` remains an alias for `vault`, `codebase` remains an alias for `code`, and `all` remains an alias for `combined`. The command rejects unknown source types rather than falling back to another corpus. A combined response preserves an outcome for every requested domain. If only some domains fail, successful results return with `partial=true`; if every requested domain fails, the command reports a failure instead of an empty success.

Each result is a record with a rank, a file location, and the matching text.

Search returns 10 results by default. Change the count with `--max-results` (or its alias `--limit`):

**Command**

```bash
uv run vaultspec-rag search "rerank inputs" --max-results 25
```

To see numeric relevance scores beside each record, add `--scores`:

**Command**

```bash
uv run vaultspec-rag search "why the service publishes a heartbeat" --type vault --scores --max-results 5
```

Each location then carries a `(score N.NNNN)` suffix. Two things to read in a vault result. The second line is the document’s own type, status and date, and the section the passage sits under, so a result says what kind of record it is and where in it the answer is before you open it. And the passage is the paragraph, list or block in that record that best answers the query - up to about 1,200 characters, verbatim from the lines the location names - not the opening of the record. The index scores whole chunks to rank records, then picks the passage to show from the best-ranked chunks of each. [Writing a query](<https://vaultspec.neve.md/docs/rag/query-craft.html#inspect-result-scores>) covers how to read the gap between two scores.

Search reports whether an empty result is authoritative. When publication cannot be verified, follow the response’s diagnostics before interpreting it as no matches.

A new worktree has its own index. A response with `reason_code=index_not_built` means that source has no published index in the selected backend; it does not by itself mean the service is degraded. Check `vaultspec-rag server status --verbose` and `vaultspec-rag server jobs --state active` first. If a job is already building the same root and source, inspect it and wait for publication. Otherwise, run `vaultspec-rag --target <worktree> index --rebuild --type code` (use `vault` or `document` for those sources). Follow the response’s `--port` when it names one. Verify that worktree’s index status, then retry the search. Job success or a zero record count alone does not prove that a searchable publication exists; a verified empty publication can legitimately have zero records.

## Choose immediate or bounded freshness

Search uses `immediate` freshness by default. It returns against the currently published generation without polling or waiting for an in-progress index job. A successful response can therefore contain results while its readiness facts say `freshness=updating`; those results come from the last usable publication.

Use a bounded wait when automation needs the publication target that existed when the request was admitted:

**Command**

```bash
uv run vaultspec-rag search "newly added cancellation path" --type code \
  --freshness-policy bounded --freshness-wait-seconds 5
```

The bound is in seconds and cannot exceed `VAULTSPEC_RAG_SEARCH_FRESHNESS_WAIT_MAX_SECONDS` (30 by default). It is not a job timeout and does not wait for job terminality. The service waits for publication evidence, unregisters the waiter on cancellation or disconnect, and returns a typed failure if the target cannot be proven before the bound.

Read the response as two separate axes:

- `availability=usable` means a published generation can answer; `unavailable` and `capacity_limited` explain why it cannot.
- `freshness=current`, `updating`, `unverifiable`, or `rebuild_required` describes that generation relative to the captured target.
- An empty result is authoritative only when `absence_authority=authoritative` for every requested source. A combined search retains each domain’s outcome and marks a useful partial answer with `partial=true`.

A bounded timeout is recoverable and carries canonical readiness facts and remediation. An HTTP `Retry-After` header appears only when the service has a future retry deadline; its absence means the service cannot truthfully promise when retrying will help. Treat `rebuild_required` as an operator action, not a reason to loop: inspect the reported remediation and run an explicit rebuild for the named source.

The service accepts the same policy directly. Token-gated routes take the `service_token` field of the discovery record as a bearer token. The record is `service.json` in the status directory, which is `~/.vaultspec-rag/` by default (`VAULTSPEC_RAG_STATUS_DIR` relocates it). Read the token from it, for example with `jq`:

**Command**

```bash
TOKEN=$(jq -r .service_token ~/.vaultspec-rag/service.json)
curl -sS http://127.0.0.1:8766/search \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"type":"code","query":"newly added cancellation path","top_k":10,"project_root":"/workspace/project","freshness_policy":"bounded","freshness_wait_seconds":5}'
```

For MCP automation, pass the typed fields to any search tool. Omit both fields for immediate mode:

**Captured output**

```json
{
  "name": "search_codebase",
  "arguments": {
    "query": "newly added cancellation path",
    "project_root": "/workspace/project",
    "freshness_policy": "bounded",
    "freshness_wait_seconds": 5
  }
}
```

MCP returns the canonical structured success or failure content. The adapter does not reclassify readiness, so HTTP, CLI JSON, and MCP automation can make the same decision from the same fields.

## Narrow code results by path

Use `--include-path` to keep only files matching a pattern, and `--exclude-path` to drop matching files. Both flags are repeatable and accept `fnmatch`-style globs:

**Command**

```bash
uv run vaultspec-rag search "lock ordering" --type code \
  --include-path "src/**" --exclude-path "**/tests/**"
```

A pattern with no glob character names a location, and matches that path and everything beneath it, so `--include-path src/vaultspec_rag/indexer` and `--include-path "src/vaultspec_rag/indexer/**"` select the same subtree. Repeating a pattern unions the selections.

The query string takes the same narrowing as a `path:` token. Reach for that form when the search travels as one string, through an agent or the Model Context Protocol (MCP) tools:

**Command**

```bash
uv run vaultspec-rag search "reopen a drifted indexed path path:src/vaultspec_rag/indexer/" --type code
```

Patterns match indexed project-relative paths, not files on disk. Matching uses Python’s `fnmatch`, where `*` also matches `/`, so `src/*.py` selects Python files at any depth under `src`, not only those directly inside it. When a pattern excludes every candidate the query matched, the empty result says so and names the pattern rather than reporting a plain no-match.

The path flags apply to code. So do `--language`, `--path`, `--structure`, `--function-name`, `--class-name`, `--dedup-locales`, and `--prefer`. They work with `--type code` and `--type combined`, where they narrow the code results. Passing any of them with `--type vault` or `--type document` is a usage error.

## Narrow by language, structure, or symbol

For code searches, filter by language, parse-tree node type, or symbol name.

Filter by language:

**Command**

```bash
uv run vaultspec-rag search "store lifecycle" --type code --language python
```

Filter by parse-tree node type with `--structure`:

**Command**

```bash
uv run vaultspec-rag search "encode" --type code --structure function_definition
```

Filter by function or class name:

**Command**

```bash
uv run vaultspec-rag search "encode" --type code --function-name encode_query
uv run vaultspec-rag search "store" --type code --class-name VaultStore
```

Target one exact project-relative path with `--path`. Unlike `--include-path`, it matches that one file and nothing under it:

**Command**

```bash
uv run vaultspec-rag search "lock" --type code --path src/vaultspec_rag/store_runtime.py
```

## Narrow vault results

For vault searches, filter by document type, feature, date, or tag.

**Command**

```bash
uv run vaultspec-rag search "concurrency" --doc-type adr
uv run vaultspec-rag search "concurrency" --feature server-supervision
uv run vaultspec-rag search "concurrency" --date 2026-06-12
uv run vaultspec-rag search "concurrency" --tag adr
```

Pass `--date` as `yyyy-mm-dd`, and pass `--tag` without the leading `#`.

## Collapse locale duplicates

Locale-variant collapse is on by default. Turn it off for a search with `--no-dedup-locales`, or force it on with `--dedup-locales`:

**Command**

```bash
uv run vaultspec-rag search "greeting" --type code --no-dedup-locales
```

## Prefer production, tests, or documentation

Use `--prefer production`, `--prefer tests`, or `--prefer documentation` to favor that kind of code in the ranking:

**Command**

```bash
uv run vaultspec-rag search "encode batch" --type code --prefer tests --max-results 12 --scores
```

The preference adds 0.05 to the score of results in the preferred category and subtracts 0.05 from all others. It doesn’t exclude other code results or guarantee that a preferred result ranks first. It also doesn’t remove the default demotion of tests, docs, locale, and vendored code, which is 0.3. To lift that demotion for one search, use `include:tests` in the query. Inspect the returned passages to judge their relevance.

To return only test code, use `only:tests` in the query instead. See [noise-domain filters](<https://vaultspec.neve.md/docs/rag/search-and-index.html#filter-noise-by-domain>) for other restrictions.

## Filter noise by domain

Each code chunk gets a *noise domain* from its path. This is the axis you use to cut noise inside the code content domain. The [glossary](<https://github.com/nevenincs/vaultspec-rag/blob/d126f083802311e30300d2c5e98f087806172a52/docs/glossary.md>) covers both senses of the word: `--type` selects a content domain, while the tokens here select noise domains.

| Noise domain | What it covers |
| --- | --- |
| `prod` | Production source - what a search usually wants |
| `tests` | Test files and directories, such as `tests/`, `*_test.*`, `conftest.py` |
| `docs` | Documentation (`docs/`, `README*`, `*.md`/`*.rst`) |
| `locale` | Localization tables, such as `locales/`, `i18n/`, `<lang>.yml` |
| `generated` | Machine-emitted files, such as `*_pb2.py`, `*.min.js`, `__generated__/` |
| `vendored` | Third-party trees, such as `vendor/`, `dist/`, `node_modules/` |
| `worktree` | Clones under `.claude/worktrees/` or `.git/worktrees/` |

By default, the search keeps production first. It hides `generated` output and `worktree` clones, demotes `tests`, `docs`, `locale`, and `vendored` below production, and collapses locale duplicates. The default demotion is 0.3 in score. At index time the indexer also skips `.claude/worktrees/`, along with `.git/`.

When a query still returns noise, narrow by domain rather than raising `--max-results` and reading past it.

Steer a single search with inline query tokens. They ride in the query string, so they need no flags and pass through the running service unchanged. Values are comma-separated and repeatable.

**Command**

```bash
# Hide one noise domain for this search
uv run vaultspec-rag search "retry backoff policy exclude:tests" --type code

# Hide several at once (comma-separated, or repeat the token)
uv run vaultspec-rag search "payment capture flow exclude:tests,docs,vendored" --type code

# Restrict to one or more domains - for example, find only the tests for a behavior
uv run vaultspec-rag search "fixture setup helpers only:tests" --type code

# Re-admit a domain the profile hides or demotes by default
uv run vaultspec-rag search "translation table lookup include:locale" --type code
```

Domain tokens compose with the path and locale controls, so you can scope precisely:

**Command**

```bash
# Production code under one subtree, with the legacy tree removed
uv run vaultspec-rag search "auth handler exclude:tests" --type code \
  --include-path "src/**" --exclude-path "**/legacy/**"

# Lift the default demotion of tests for one search
uv run vaultspec-rag search "encode batch include:tests" --type code

# Keep every locale variant for a translation audit
uv run vaultspec-rag search "greeting string include:locale" --type code --no-dedup-locales
```

The `search_codebase` MCP tool exposes the same control as typed `exclude_domains` / `only_domains` / `include_domains` parameters. Set the per-project defaults, meaning which domains hide, which demote, and how hard, with `VAULTSPEC_RAG_CODE_NOISE_HIDE_DOMAINS`, `VAULTSPEC_RAG_CODE_NOISE_DEMOTE_DOMAINS`, and `VAULTSPEC_RAG_CODE_NOISE_DEMOTE_PENALTY`, which the [configuration guide](<https://vaultspec.neve.md/docs/rag/configuration.html>) carries as rows. Through a config source they are the same three names with the prefix stripped, but the guide is keyed on the variables, so those are the names to search for there.

## Every filter in one place

The [CLI reference](<https://vaultspec.neve.md/docs/rag/cli.html>) lists every search flag with its type, default, and which content domain it applies to. The sections here cover the ones you reach for most.

## Use the service and MCP surfaces

The running service accepts the same fixed set of source names on `POST /search`, `POST /reindex`, and `POST /clean`. Send `vault`, `code`, `document`, or `combined`. Service requests don’t accept the CLI aliases. `GET /readiness` reports per-domain document counts and health without loading a model.

An assistant reaches the same operations through MCP. Combined operations keep one outcome per content domain, including failures, so a failure in one domain never erases a valid result from another. See [MCP integration](<https://vaultspec.neve.md/docs/rag/mcp.html>) for the tool list and [service mode](<https://vaultspec.neve.md/docs/rag/service-mode.html>) for operating the service.

## Build and refresh the index

Indexing keeps search results current with your files. By default, `index` uses the compatibility `combined` target and runs incrementally, processing only changed work in each domain:

**Command**

```bash
uv run vaultspec-rag index
```

If a service is running, the command hands the job to it. The work runs in the background; check progress with:

**Command**

```bash
uv run vaultspec-rag server jobs
```

If no service is running, the command indexes in the current process and returns when it’s done.

To scope the run, name `vault`, `code`, `document`, or `combined`:

**Command**

```bash
uv run vaultspec-rag index --type code
uv run vaultspec-rag index --type document
```

Code admission is not a recursive “all readable files” scan. It follows the configured source profile and explicit project routing. Configure non-source extraction and its owner in [preprocessing hooks](<https://vaultspec.neve.md/docs/rag/preprocessing-hooks.html>).

## Rebuild from scratch

To delete and recreate the code index, run:

**Command**

```bash
uv run vaultspec-rag index --rebuild --type code
```

`--rebuild` deletes the selected index data before rebuilding. Choose the domain carefully; an explicit `--type` is required. See the [index reference](<https://vaultspec.neve.md/docs/rag/cli.html#index>) for other targets and options.

## Clean index data

To delete the code index without rebuilding it, run:

**Command**

```bash
uv run vaultspec-rag clean code
```

Review the target before confirming. See the [clean reference](<https://vaultspec.neve.md/docs/rag/cli.html#clean>) for other targets and non-interactive use.

## Where to go next

- Run searches and indexing through a background daemon: [service mode](<https://vaultspec.neve.md/docs/rag/service-mode.html>).
- Every command, flag, and exit code: [CLI reference](<https://vaultspec.neve.md/docs/rag/cli.html>).
- Tune defaults like batch sizes, chunk budgets, and the data directory: [configuration](<https://vaultspec.neve.md/docs/rag/configuration.html>). The result count is not among them: `--max-results` defaults to 10 and is set per call rather than configured.

## Getting help

The [issue tracker](<https://github.com/nevenincs/vaultspec-rag/issues>) takes questions as well as bug reports.