Under The Hood

You do not need this chapter to use Psy. You need it when you want to know why a value came out the way it did, or when you are building something on top.

source
  ↓
lexer
  ↓
parser
  ↓
AST
  ↓
semantic analysis
  ↓
resolved IR
  ↓
framework
  ↓
adapter / generator

No stage is skipped, ever. In particular the AST is always built, even for a plain psy check that will never print it — the formatter prints from it and semantic analysis derives everything from it rather than from the token stream. One representation, one source of truth.

What each stage refuses to do

The interesting part of a pipeline is not what each stage does. It is what each stage is not allowed to do.

The lexer is line driven. It emits layout tokens, tracks source ranges, recognises comments and interpolation, and tolerates arbitrary text so that unquoted strings keep working. Comments are collected outside the token stream, so the parser can attach them and strip them from string blocks without either of them being a special case.

The parser is syntax only. It never resolves a symbol. It never decides whether a bare identifier is a string or a reference. It never validates inheritance. Hand it a file that references forty things that do not exist and it will happily produce an AST.

Semantic analysis binds modules, activates keywords, linearizes inheritance and resolves every value, recording provenance as it goes.

Frameworks consume resolved IR and a diagnostic sink. That is all they get.

That last boundary is enforced by construction, not by documentation: @psy/framework exposes no route to the lexer, the parser or the resolver. There is no API to reach for. A framework cannot change how Psy parses or what a Psy program means, which is the property that lets you read any .psy file without first finding out what is installed.

The AST

The AST preserves source syntax and source ranges. Every node carries a range; declarations and block operations additionally carry trivia, so the formatter can reproduce your layout rather than approximating it:

interface Trivia {
    leadingComments: Comment[];
    trailingComment: Comment | null;
    blankLinesBefore: number;
    blankLinesAfterComments: number;
}

blankLinesBefore is why the formatter keeps your paragraph breaks between declarations. It is not inferring them from anything; it wrote them down.

The deliberate ambiguity

type ValueNode =
    | StringValueNode      // style: "bare" | "quoted" | "block"
    | NumberValueNode
    | BooleanValueNode
    | NullValueNode
    | IdentifierValueNode  // ambiguous: string or reference
    | ReferenceValueNode
    | SuperValueNode
    | ListValueNode
    | ObjectValueNode
    | BlockValueNode;

IdentifierValueNode is the one to look at. The parser cannot know whether Foo is the string "Foo" or a reference to the symbol Foo — that depends on the module namespace, which is a semantic question.

So it does not guess. It records "there was an identifier here" and lets semantic analysis decide. The ambiguity is represented honestly in the tree instead of being resolved early by a parser that does not have enough information.

Blocks are ordered operations

interface BlockValueNode {
    kind: "BlockValue";
    blockKind: "object" | "list" | "string" | "compose";
    operations: BlockOperation[];
    range: Range;
}

type BlockOperation =
    | AssignOperation   // key: value
    | ItemOperation     // - value
    | ComposeOperation  // + value / ++ value
    | TextOperation;    // raw text lines

Storing a block as an ordered list of operations rather than as a finished value is what makes local operation order expressible at all. strict: false then + super is a different sequence from + super then strict: false, and the tree can say so.

TextOperation keeps both the resolved string parts and a line-by-line rendering that retains comments, which is how a multiline string survives a format round-trip with its comments intact.

psy inspect --ast spec/framework.psy

Semantic analysis

The order, which matters:

1. Load and parse every reachable module.
2. Bind module namespaces — local specs, constants and imported names into one namespace.
3. Bind exports and re-exports.
4. Activate declaration keywords.
5. Bind inheritance and linearize.
6. Reject import cycles.
7. Resolve properties on demand, memoized per spec and property.
8. Check abstract contracts and types.

Step 2 happening before step 7 is what makes declaration order irrelevant. Step 7 being on demand is what keeps a large program cheap — nothing resolves a property nobody asked about.

The two binding rules, again

They are worth restating in implementation terms, because together they explain something about how resolution has to work.

super binds lexically. It is relative to the spec whose source is being evaluated, so a parent's definition keeps referring to that parent's own parents even when a grandchild inherits it.

self binds dynamically. It always names the spec a property is being resolved for, so an inherited definition yields a different value per descendant.

Put those together and you get a constraint: resolution works on definitions, not on cached parent values. When a child inherits a property, the parent's AST is re-evaluated in the child's context. There is no other way to satisfy both rules at once.

That is the reason for the memoization being keyed on spec and property rather than on property alone.

Composition folding

A block resolves by folding its operations left to right into an accumulator. The target kind is decided first: object if there is any assignment, else list if there is any item, else string if there is any text, else the kind of the operands.

For objects, the fold obeys the one rule that produces both documented behaviours:

A composition replaces a value that an earlier assignment put in place, but never a value that an earlier composition contributed.

For lists, + appends with stable deduplication against the accumulator and ++ appends verbatim. For strings, each contribution becomes a block joined by a blank line.

Cycles

A resolution stack rejects self-referential properties (PSY3007) and looping super chains. Import cycles (PSY3005) and inheritance cycles (PSY3006) are caught earlier, during binding.

The IR

Fully resolved. Inheritance applied, super expanded, compositions evaluated, references followed, interpolation rendered. It contains no unresolved references, and it is the only thing frameworks and adapters ever see.

interface SpecIR {
    id: string;                  // "<module path>#<name>"
    name: string;
    module: string;
    keyword?: string;            // declared with a framework keyword
    exposedKeyword?: string;     // exposes a keyword via `as`
    abstract: boolean;
    exported: boolean;
    bases: string[];
    baseIds: string[];
    linearization: string[];     // most specific first, including this spec
    properties: Record<string, PropertyIR>;
    abstractProperties: Record<string, AbstractPropertyIR>;
    source: SourceLocation;
}

type ValueIR =
    | { kind: "string";  value: string }
    | { kind: "number";  value: number }
    | { kind: "boolean"; value: boolean }
    | { kind: "null" }
    | { kind: "list";    items: ValueIR[] }
    | { kind: "object";  entries: Record<string, ValueIR> }
    | { kind: "specRef"; name: string; specId: string };

Seven kinds, matching the seven value forms in the language exactly. No surprises at the boundary.

specRef is the one that earns its place. It is what lets a framework tell use: [ReadPsySpec] from a list of strings that happen to look like names, and it is what makes the type Skill[] checkable at all. A resolved spec reference carries both the name and the spec id, so a renamed spec is a compile error rather than a string pointing at nothing.

Every ValueIR also carries provenance, which is the next section and the reason any of this was worth building.

Full type definitions, including PropertyIR, ModuleIR and ProgramIR, are in the reference.

Determinism

Module order, spec order, property order and object key order are all stable functions of the source. File paths are relative to the project root, so output does not depend on where the project lives on disk. Compiling the same project twice produces identical IR, and the test suite asserts it rather than hoping.

Provenance

Every resolved property retains enough information to explain where it originated, what parent supplied it, what composition contributed, what overrode it, and which source range was responsible.

interface Provenance {
    kind: ProvenanceKind;
    file: string;
    range: Range;
    spec?: string;
    property?: string;
    detail?: string;
    contributors?: Provenance[];   // in precedence order
    overridden?: Provenance[];     // values this one replaced
}

Kinds are literal, interpolation, reference, self, super, inherit, assign, item, compose, concat, block and unresolved.

psy inspect Api

  settings = {
    "cpu": "500m",
    "memory": "512Mi"
  }
  declared in Api at examples/deployment/main.psy
  provenance:
    block object in Api.resources (examples/deployment/main.psy:35:9)
      super BaseService.resources in BaseService.resources (...:13:5)
        block object in BaseService.resources (...:14:9)
          literal in BaseService.resources (...:15:13)

Read it outermost-first: the block that produced the value, then each contributor in precedence order, then what each of those was made of.

Overrides are recorded too

spec Child extends A:
    settings:
        strict: false
        + super

psy inspect Child.settings

Child.settings (child.psy)
  = {
    "strict": true
  }

  declared in Child at child.psy
  provenance:
    block object in Child.settings (child.psy:2:5)
      assign strict in Child.settings (child.psy:3:9)
        overridden by:
          super A.settings in A.settings (child.psy:4:9)
      super A.settings in A.settings (a.psy:2:5)
        literal in A.settings (a.psy:3:17)

Child.settings.strict resolves to the inherited value. Its provenance records the local strict: false under overridden, pointing at line 3.

So when you write a line and it does not take effect, the tool can tell you that your line was overridden and by what, rather than showing you a value and leaving you to work out that your line was never in play.

Coverage

Every resolved property carries provenance, as does every value nested inside it. Not a sample, not the top level — all of it.

This is what psy inspect renders, and it is the difference between "a surprising value" and "a surprising value I can trace to one source range in four seconds". It is also, honestly, most of why I gave up the features that would have made it impossible. No runtime, no dynamic evaluation, no cycles — every one of those non-goals buys this.

Host abstraction

All filesystem access goes through a CompilerHost, which lets the test suite drive the compiler from in-memory files with no temp directories.

Paths handed to the rest of the compiler are always display paths: POSIX style, relative to the project root. That is what makes diagnostics and generated artifacts byte-for-byte identical across machines, operating systems and CI runners — and byte-for-byte identical is the property that makes generated output reviewable.

And the architecture is specified in Psy

Everything on this page has a .psy file behind it:

export compiler-stage SemanticAnalyzer:
    input:
        AST

    output:
        Resolved IR

    responsibilities:
        - resolve symbols
        - resolve inheritance
        - resolve composition
        - validate abstract properties
        - type-check values
        - preserve provenance

{
  "SemanticAnalyzer": {
    "input": "AST",
    "output": "Resolved IR",
    "responsibilities": [
      "resolve symbols",
      "resolve inheritance",
      "resolve composition",
      "validate abstract properties",
      "type-check values",
      "preserve provenance"
    ],
    "layer": "compiler",
    "invariants": [
      "consumes only the declared input",
      "produces only the declared output"
    ]
  }
}

That is compiled by the real compiler on every check, which means this chapter cannot drift from the implementation without something going red.

Which brings us to the last chapter, and the thing I have been alluding to since the first page.

Next: Psy Describes Psy