New API for optionals

[pdubroy]

In the next major version of Ohm, we'd like to improve the API for optionals. For example, take the following rule:

  line = (key ":")? value

In the current version of Ohm (v17), you'd might write an action like this:

{
  line(keyOpt, _colonOpt, value) {
    const actualKey = keyOpt.childAt(0)?.sourceString ?? '<No key>';
    // …
  }
}

Arity change

In Ohm v18, the contents of an optional node will no longer be flattened — you'll get a single node representing the entire optional. So the action will have arity 2: line(opt, value) { ... }.

Getting the values

With the above change, you could still just use childAt or children — the node would either have 0 or 2 children. But we'd like to have a nicer API for working with option nodes. Here are some of the options I'm considering:

Pattern-match style

{
  line(opt, value) {
    const actualKey = opt.when({
      Some: (key, _colon) => key,
      None: () => '<No key>'
    });
    // …
  }
}

Alternative names: caseOf, unpack. I'm avoiding match because that already has a meaning in Ohm.
Alternative key names: isSome/isNone. Might make sense to align them with methods of the same name (isSome()/isNone()).

Bare callbacks

Effectively the same thing, but without explicit case names:

{
  line(opt, value) {
    const actualKey = opt.getOrElse((key, _colon) => key, () => '<No key>');
    // …
  }
}

Prior art

• The TC39 pattern matching proposal used match / when.
• patcom is a pattern-matching library for JavaScript. Also uses match/when.
• effect-ts pattern matching. Uses the same pattern-match style as above, but the method is named match, and the keys are onSome and onNone.

[pdubroy]

It's also worth mentioning that I'm considering similar handling for iter nodes:

• they will no longer flatten their children.
• there will be a convenience method for accessing the children.

Example

For the following rule:

  config = (key ":" value)+

Action:

{
  config(iter) {
    // allKeys has type `string[]`
    const allKeys = iter.collect((k, _colon, _v) => k.sourceString);
  }
}

[pdubroy]

Another approach would be to introduce a new SeqNode type, and then just use a "raw" representation for iter and opt nodes:

• a? → NonterminalNode | undefined
• (a b)? → SeqNode | undefined
• a+ → NonterminalNode[]
• (a b)+ → SeqNode[]

Then SeqNode would have an arity-checked way of getting the children out, e.g.:

aSeqNode.unpack((a: NonterminalNode, b: NonterminalNode) => { ... }));

(This would have a run-time check for the correct function arity, much like we check semantic actions at operation creation time.)

Of course, the unpack method (and SeqNode itself) is not strictly required, but from 10+ years of experience with Ohm's semantic actions, we do find that the arity checking prevents a lot of errors when you're evolving a grammar and the actions together.

Arguably in TypeScript a SeqNode could just be a tuple type (e.g. [NonterminalNode, NonterminalNode]), but then you lose the ability to distinguish seqs and iterations at runtime.

[pdubroy]

I've been prototyping this, and I've found a couple downsides to representing iter nodes as arrays:

• CstNode[] implies a mutable array, so it would be better to use readonly CstNode[] everywhere, which is more verbose. We could introduce an alias (CstNodeList) but that adds complexity.
• Because of implicit space skipping, it's not trivial to determine the correct matchLength / startIdx of the result of an iteration. For example, take the following grammar:

G {
  Start = digit*
}

…and the input 0 1 2. If you have a CstNode[] and you simply sum the match lengths in the iteration, you'll get 3. But the correct matchLength for the iteration, including spaces, is 6.

If we have our own CST node type representing iterations, it can handle this properly in its own matchLength implementation.