API Reference
Interface
IStorySourceTokenizer
Extension seam for optional language tokenizers. Implementations map parser-specific tokens into exact, renderer-neutral story source spans without adding parser dependencies to ChartForgeX.
Methods
public abstract StorySourceText Tokenize(String source) #Returns:
StorySourceTextTokenizes exact source text into renderer-neutral semantic spans.
Parameters
- source System.String
Properties
public abstract String Language { get; } #Gets the canonical language identifier handled by this tokenizer.