aidp-knowledge-bases — RAG corpus + vector index

A Knowledge Base is AIDP's managed RAG store: it embeds source data into a vector index you can retrieve from (directly, or via a RAG_TOOL node in an agent flow). Runs over the LA AgentFlows family REST API.

Engine: oci raw-request --profile DEFAULT (no CLI group for KB in v1.0.0). Lake-scoped, live-verified 2026-06-10: GET …/dataLakes/<ocid>/knowledgeBases?catalogKey=<key>&schemaKey=<key> → 400 InvalidParameter means the route exists but needs real catalog/schema keys (resolve via aidp-catalog-explore); the workspace-scoped path 404s. Confirm with a live read before any write; record in references/rest-endpoint-map.md.

Live-verified 2026-06-10 on de-agent — correction: a bare GET …/knowledgeBases (no query string) returns 400 InvalidParameter requiring both schemaKey and catalogKey query params — the route is provisioned (lake-scoped), it is the missing params that 400, not a missing route.

When to use

• "Build a RAG / knowledge base / vector index", "embed these docs/tables", "semantic search over X", or any prerequisite for a RAG_TOOL agent-flow node.
• NOT ad-hoc LLM-in-SQL (→ aidp-ai-sql); NOT the agent that uses the KB (→ aidp-agent-flows / aidp-agent-highcode).

Create a KB (CreateKnowledgeBaseDetails, camelCase wire fields)

POST …/dataLakes/<ocid>/knowledgeBases

{
  "displayName": "policy_kb",
  "description": "RAG over policy docs",
  "catalogKey": "<catalog-key>", "schemaKey": "<schema-key>",
  "workspaceKey": "<ws-key>", "clusterKey": "<cluster-key>",
  "type": "...", "modality": "...",
  "embeddingModelSourceType": "...", "embeddingModelName": "<embedding-model>",
  "chunkSize": 512, "chunkOverlap": 64,
  "sourceFilePattern": "*.pdf",
  "indexDetails": { "type": "HNSW", "distance": "COSINE", "neighbors": 32, "efConstruction": 200, "targetAccuracy": 95 }
}

• indexDetails.type ∈ HNSW | IVF. Both index types accept the full 7-value distance enum: COSINE | DOT | EUCLIDEAN | HAMMING | JACCARD | L2_SQUARED | MANHATTAN (SDK KbVHnswIndexDetails / KbVIvfIndexDetails distance setter allowed_values — kb_v_hnsw_index_details.py:110, kb_v_ivf_index_details.py:110). Tuning values below are illustrative.

• HNSW tuning params (SDK KbVHnswIndexDetails, camelCase wire fields, all int — kb_v_hnsw_index_details.py:74-79):

  Field	Meaning
  neighbors	max neighbors each vector can have on any layer (the HNSW M parameter) — :146
  efConstruction	max closest-vector candidates considered during index construction — :170
  targetAccuracy	target accuracy percentage 1–100 — :122

• IVF tuning params (SDK KbVIvfIndexDetails, camelCase wire fields, all int — kb_v_ivf_index_details.py:74-79):

  Field	Meaning
  neighborPartitions	number of partitions (clusters) to divide the vector data into — :146
  neighborPartitionProbes	max partitions to probe during a search (higher = more accurate, slower) — :170
  targetAccuracy	target accuracy percentage 1–100 — :122

• embeddingModelName/SourceType — list available embedding models via aidp-models-catalog (modelType=EMBEDDING); needs a RUNNING clusterKey for embedding compute.

• Verify-first: the field names above are from the SDK CreateKnowledgeBaseDetails; the create was not round-tripped (it triggers embedding compute). Confirm the embedding-model + type/modality enums against a live read / aidp help before a production create — do not invent values.

Ingest + maintain

Wire it to an agent

Once the index is built, reference the KB from a RAG_TOOL node (aidp-agent-flows, references/agent-flow-nodes.md) or from high-code (aidp-agent-highcode). The KB must exist + be ingested before the RAG tool can retrieve.

Guardrails

• Mutation gate: KB create/ingest consumes embedding compute — show the body, confirm first, persist to .aidp/payloads/ (references/payloads.md).
• Resolve catalogKey/schemaKey/clusterKey (real keys) first via aidp-catalog-explore / aidp-cluster-ops.

References

• aidp-agent-flows (RAG_TOOL consumer) · aidp-models-catalog (embedding models) · aidp-catalog-explore (keys)
• references/oci-raw-request.md · references/rest-endpoint-map.md · references/payloads.md