Analysis of `parse()` Function¶

Overview¶

The parse() function extracts entities (ontology terms) from free-form text by matching against an OWL ontology. It processes text sentence-by-sentence, identifies ontology matches, and returns a deduplicated list of entity names.

Function Signature¶

def parse(
    ontology: str,        # Name of the ontology (e.g., 'catmanifest')
    ontology_path: str,   # Path to ontology directory
    input_text: str       # Text to analyze
) -> list[str]           # List of unique entity names found

High-Level Flow Diagram¶

┌─────────────────────────────────────────────────────────────────┐
│                         parse() Function                         │
└─────────────────────────────────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  1. Setup Logging                   │
        │     - Set bast_ai logger to ERROR   │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  2. Get Parser (Cached)            │
        │     getParser(ontology, path)       │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  3. Get swap_input_text method      │
        │     swap_synonyms = parser.swap...  │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  4. Initialize entities list        │
        │     entities = []                   │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  5. Segment Text into Paragraphs    │
        │     segment_text(input_text)        │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  6. For Each Paragraph              │
        │     for paragraph in paragraphs:    │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  7. For Each Sentence               │
        │     for sentence in paragraph:     │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  8. Parse Sentence                  │
        │     swap_synonyms(sentence)         │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  9. Extract Ontology Tokens          │
        │     Filter tokens with 'swaps' key  │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  10. Add to entities list           │
        │      entities.extend(tokens)        │
        └─────────────────────────────────────┘
                              │
                              ▼
        ┌─────────────────────────────────────┐
        │  11. Deduplicate & Return           │
        │      return list(set(entities))     │
        └─────────────────────────────────────┘

Detailed Component Flow¶

1. Parser Initialization (`getParser`)¶

getParser(ontology, ontology_path)
    │
    ├─► Check if parser exists in cache
    │   Key: f"{ontology_path}/{ontology}"
    │
    ├─► If NOT cached:
    │   │
    │   ├─► Create FindOntologyData
    │   │   ├─► ontologies = [ontology]
    │   │   ├─► absolute_path = ontology_path
    │   │   └─► namespace = 'http://bast.ai'
    │   │
    │   └─► Create MutatoAPI(finder)
    │       └─► Cache parser instance
    │
    └─► Return cached or new parser

2. Text Segmentation (`segment_text`)¶

segment_text(input_text)
    │
    ├─► Segment into Paragraphs
    │   └─► Split by double newlines (\n\n)
    │
    └─► For each paragraph:
        └─► Segment into Sentences
            └─► Use spaCy sentence segmentation
                └─► Returns: [[sentence1, sentence2], [sentence3]]

3. Entity Extraction (`swap_input_text`)¶

swap_input_text(sentence)
    │
    ├─► 1. Tokenize Input Text
    │   └─► Tokenizer().input_text(sentence)
    │       └─► Returns: list[str] of tokens
    │
    ├─► 2. Parse Input Tokens
    │   └─► ParseInputTokens().process(tokens)
    │       └─► Returns: dict with 'tokens' key
    │
    └─► 3. Swap Input Tokens (recursive matching)
        └─► swap_input_tokens(tokens, ctr=0)

4. Token Matching Pipeline (`swap_input_tokens`)¶

swap_input_tokens(tokens, ctr)
    │
    ├─► 1. Perform Exact Matching
    │   └─► Match tokens against ontology labels
    │       ├─► rdfs:label
    │       ├─► rdfs:seeAlso
    │       ├─► skos:altLabel
    │       └─► :inflection
    │
    ├─► 2. Perform Span Matching (if spans exist)
    │   └─► Match multi-word phrases
    │
    ├─► 3. Perform Hierarchical Matching
    │   └─► Match parent/child relationships
    │
    └─► 4. Recursive Call (if ctr < 2)
        └─► Re-process swapped tokens
            └─► Allows finding nested matches

Data Structures¶

Input¶

ontology = "catmanifest"
ontology_path = "../../testdir/ontologies"
input_text = """
I love traveling to many states especially seattle, Washington. 
I also love visiting my home state!
"""

Intermediate (Parser Result)¶

d_parser_result = [
    {
        'token': 'i',
        'normal': 'i',
        'swaps': [...],  # Only present if ontology match found
        ...
    },
    {
        'token': 'seattle',
        'normal': 'seattle',
        'swaps': [...],  # Ontology match!
        ...
    },
    ...
]

Output¶

entities = ['seattle', 'washington', 'many', 'home']

Step-by-Step Execution Example¶

Given input:

"I love traveling to many states especially seattle, Washington."

Step 1: Segmentation¶

paragraphs = [
    ["I love traveling to many states especially seattle, Washington."]
]

Step 2: Sentence Processing¶

sentence = "I love traveling to many states especially seattle, Washington."

Step 3: Tokenization¶

tokens = ['i', 'love', 'traveling', 'to', 'many', 'states', 
          'especially', 'seattle', ',', 'washington', '.']

Step 4: Ontology Matching¶

Exact Match: 'seattle' → matches ontology entity
Exact Match: 'washington' → matches ontology entity
Exact Match: 'many' → matches ontology entity
Hierarchical Match: 'states' → may match via hierarchy

Step 5: Result Extraction¶

o_tokens = ['seattle', 'washington', 'many']  # Only tokens with 'swaps' key

Step 6: Deduplication¶

entities = ['seattle', 'washington', 'many', 'home']  # From all sentences
return list(set(entities))  # Remove duplicates

Key Components¶

1. `FindOntologyData`¶

Loads OWL ontology file(s) from disk
Builds lookup structures for fast matching
Queries ontology using SPARQL
Caches label mappings (rdfs:label, skos:altLabel, etc.)

2. `MutatoAPI`¶

Main parsing API
Orchestrates matching pipeline:
Exact matching
Span matching (multi-word)
Hierarchical matching (parent/child)
Recursive processing for nested matches

3. `Segmenter`¶

Paragraph segmentation (by double newlines)
Sentence segmentation (using spaCy)
Returns nested list structure

4. Matching Strategies¶

Exact Matching¶

Matches individual tokens against ontology labels
Uses sliding window for n-grams (1-5 words)
Case-insensitive matching

Span Matching¶

Matches multi-word phrases
Handles discontinuous matches
Uses distance-based rules

Hierarchical Matching¶

Matches based on ontology class hierarchy
Uses rdfs:subClassOf relationships
Allows finding parent/child entities

Performance Optimizations¶

Parser Caching: Parsers are cached by {ontology_path}/{ontology} key
Segmenter Caching: Uses @lru_cache for repeated text segmentation
Ontology Lookup: Pre-built lookup structures for fast matching
Recursive Limit: Maximum 2 recursive passes to prevent infinite loops

Error Handling¶

Empty ontology: Raises ValueError("Empty Ontology")
Invalid input: Validation at each step
Missing ontology file: Handled by FindOntologyData

Dependencies¶

from bast_ai_segmenter import segment_text
from bast_ai_parseowl import MutatoAPI, FindOntologyData

Return Value¶

Returns a list of unique entity names (strings) that were found in the input text and matched against the ontology.

Example:

['seattle', 'washington', 'many', 'home']

Analysis of parse() Function¶