Skip to content
Version: 1.0.0 Last Updated: 2026-05-14 Status: Active

Codebase-First Specification

Every spec must be grounded in what already exists.

The Problem

Bast AI is not a greenfield project. It is a mature platform with 33 core libraries, 13 API services, 19 pipelines, a full W3C PROV-O ontology pipeline, and deep architectural patterns embedded across every layer. When you write a spec without first understanding the existing codebase, three things happen:

  1. You spec what's already built. The platform already carries PROV-O lineage in every OWL entity via prov:wasDerivedFrom and prov:wasGeneratedBy. It already has prov_stack() for execution provenance. It already has BastRetrieverMongo for hybrid knowledge retrieval. If you don't know this, you write specs to build it again — wasting design time and creating confusion about what's canonical.

  2. You design against the wrong constraints. The codebase has established patterns: BastObject as a base class, ObjectEnforcer for validation, FileIO for all file operations, EnvIO for environment access. If your design doesn't use these, it won't pass review — not because of preference, but because the patterns exist to ensure consistency across 33+ modules that share these foundations.

  3. You miss integration points. A spec for "add provenance tracking to knowledge retrieval" sounds reasonable until you discover that build_owl already writes PROV triples, OntologyLoader already parses them at runtime, and CATOntologyContext already indexes them for query matching. The spec you actually need might be a 2-task wiring change, not a 14-task new capability.

The Rule

Before writing any proposal, design, or spec, you must be able to answer:

"What does the codebase already do in this area, and where does it do it?"

If you cannot answer this from memory, you are not ready to spec. You need to explore first.

How to Explore Before You Spec

Use /opsx:explore — It Exists for This

The explore phase is not optional brainstorming. For this codebase, it is where you build the understanding that makes your spec accurate. When you enter explore mode:

  1. State what you think needs to be built. Be specific.
  2. Ask the agent to trace what already exists. It will read the actual code, not guess.
  3. Look at the output. You will frequently discover that 60-80% of what you planned already exists in some form.

Example:

/opsx:explore

"I want to add provenance tracking so we can trace where
knowledge base answers come from."

The agent will find: - build_owl already writes prov:wasDerivedFrom on every entity - OWLGenerator in the PDF pipeline already captures sources_ref per entity - catmanifest.owl already carries source file paths as PROV annotations - OntologyConcept model already has handler routing based on ontology matches - prov_stack() in base utilities already tracks execution call chains

Now your spec changes from "build provenance tracking" to "surface existing PROV annotations in the chat response metadata" — a fundamentally different (and much smaller) piece of work.

What to Look For

When exploring a capability area, trace these layers:

Layer What to Find Where to Look
Data model Does a schema/class already exist? src/python/config/bast_ai_config/config/typedef.py, models.py files
Base utilities Is there already a utility for this? src/python/base/bast_ai/ — FileIO, TextUtils, EnvIO, Crypto, ObjectEnforcer
OWL/Ontology Is this already in the ontology? src/testdir/ontologies/*.owl, build_owl module, OntologyLoader
Agent framework Is there already an agent pattern? src/python/agno/bast_ai_agno/ — factories, workflows, retrievers
API layer Is there already an endpoint? src/apis/ — check existing routes before adding new ones
Pipeline Is there already a pipeline step? src/pipelines/ — especially process_pdf for document ingestion
Config Is there already a config field? resources/config/*.yaml, typedef.py

Minimum Exploration Checklist

Before running /opsx:propose, you should know:

  • Which existing modules touch this capability area
  • Which base library utilities are relevant (FileIO, TextUtils, EnvIO, etc.)
  • Whether the OWL/ontology layer already models this concept
  • Whether there's an existing pipeline or API endpoint that partially covers it
  • What config fields already exist for this feature
  • Which integration tests already exercise adjacent functionality

What a Codebase-Grounded Spec Looks Like

Bad: Spec written without codebase knowledge

### Requirement: Provenance tracking for knowledge base entries
The system SHALL track the source of each knowledge base entry
including the original document, page number, and extraction date.

#### Scenario: KB entry created from PDF
- WHEN a PDF is processed and KB entries are created
- THEN each entry SHALL include source_document, page_number,
  and extraction_timestamp fields

This spec ignores that: - prov:wasDerivedFrom already exists on every OWL entity - The PDF pipeline's OWLGenerator already captures source references per entity - AddEntities already writes these as PROV triples via add_derived_from() - The data model for this already exists in build_owl's entity schema

Good: Spec grounded in existing code

### Requirement: Surface PROV annotations in retrieval responses
The knowledge retrieval layer SHALL include `prov:wasDerivedFrom`
source references in BastRetrieverMongo responses, using the
existing PROV annotations written by build_owl's AddEntities
(via add_derived_from in graph_agent.py).

#### Scenario: Retrieved KB entry includes provenance
- WHEN BastRetrieverMongo returns a knowledge match
- THEN the response SHALL include the source_ref field
  from the matched OWL entity's prov:wasDerivedFrom annotation

#### Scenario: Provenance absent for legacy entries
- WHEN a KB entry predates PROV annotation
- THEN source_ref SHALL be null and the entry SHALL still
  be returned without error

This spec: - References the actual module that owns the capability (build_owl, AddEntities) - Names the actual code path (add_derived_from in graph_agent.py) - Identifies the actual integration point (BastRetrieverMongo) - Scopes the work to what's actually missing (surfacing, not building) - Handles the real edge case (legacy entries without PROV data)

For AI Agents Working on This Codebase

If you are an AI agent (Claude, Codex, Copilot) proposing or implementing changes:

  1. Always explore before proposing. Run /opsx:explore and trace the existing code. Read the actual files, not just file names.

  2. Reference existing code in your artifacts. Proposals should cite specific files and line numbers. Designs should name the actual classes and methods they extend. Tasks should reference the actual modules they modify.

  3. Assume it exists until proven otherwise. This codebase has been under active development with 33+ libraries. Before designing a new utility, check src/python/base. Before designing a new data model, check typedef.py. Before designing a new pipeline step, check src/pipelines.

  4. The OWL layer is not decorative. The PROV-O ontology structure is load-bearing infrastructure. Every manifest.owl carries provenance that anchors generation to curated expert knowledge. Specs that touch knowledge retrieval, answer generation, or entity management must account for the existing OWL pipeline.

  5. Use CLAUDE.md and the reference docs. The project's CLAUDE.md contains the canonical patterns. The MkDocs reference docs at docs/reference/ catalog every module, API, and pipeline. Read them.

Summary

Principle What It Means
Explore before you spec Run /opsx:explore and trace existing code before writing any artifacts
Name the code Proposals and designs must reference actual files, classes, and methods
Scope to the gap Spec only what's missing, not what already exists
Respect the patterns Use BastObject, ObjectEnforcer, FileIO, EnvIO — don't reinvent them
OWL is load-bearing PROV-O provenance, ontology routing, and manifest structure are core infrastructure
Assume it exists Check base libraries, config models, and pipelines before designing new ones

The codebase is the spec's foundation. If your spec doesn't demonstrate knowledge of what's already built, it's not ready for review.