Scope hygiene

Massive Hygiene Simplification

So @mflatt of Racket fame has recently been working on a simplified hygiene algorithm for Racket that uses scope sets rather than marks and renames. His notes outlining the approach are here. It's crazy good.

Rather than do what I really should be doing (writing my thesis) I've hacked it up in sweet.js.

Previously sweet.js was using the standard Scheme/Racket approach of marks/renames and definition contexts. This was always hard to understand how it worked and we had a number of bugs that I could never track down. resolve was also super slow.

The basic idea of scope sets is that every syntax object is marked with the set of scopes it is in and each binding form (e.g. function) creates a new scope:

function foo(x^{a}) {
    function bar(x^{a, b}) {
        return x^{a, b};
    }
}

There is a global map that associates each identifier and scopeset with a new binding so the above might have a map like:

{
    x: [{
        scopeSet: [a],
        binding: 1
    }, {
        scopeSet: [a, b],
        binding: 2
    }]
}

and fully expanding it with resolve would then give

function foo(x1) {
    function bar(x2) {
        return x2;
    }
}

There are a few more details about macro invocation, recursive macro definitions, and let/var but that is the basic idea.

The big win here is that hygiene is mostly understandable now (check out how tiny resolve is now!) and faster (even without the memoization we were doing in the old resolve).

A few other things snuck into this PR.

New names for macro definitions

The current let name = macro {} is bad. I proposed some new syntax ideas in #437 (closed) and this is the start of implementing them. Internally there is now only one macro definition form stxrec which as the name suggests is recursive. On a suggestion from @mflatt the non recursive form is then built from the recursive form (implementation is here).

So right now the forms look like:

stxrec m {
    rule {} => {}
}
stxnonrec m {
    rule {} => {}
}

The stxnonrec is obviously a terrible name and just a placeholder. I think we want:

stxrec m = macro {
    rule {} => {}
}
stx m = macro {
    rule {} => {}
}
// using the `x = macro {}` is suggestive with normal `let`/`var`
// and also allows us to smoothly use the same syntax
// to put things other than macros into the compiletime env
stx x = 42;
stx x = function(x) { return x; };
// with maybe a convenient macro declaration form
macro m {
    rule {} => {}
}

Macro definitions no longer break hygiene to capture match

Probably fixes #430 (closed) and others.

So sorry I ever did that :(

macroclass currently doesn't work because it was breaking hygiene and I haven't updated it yet.

localExpand takes a stop list

At the moment it works by matching the name and the next delimiter token.

Phasing

Racket has begin-for-syntax that let's you define stuff for use inside a macro without going to the trouble of pulling it out into a separate module and importing it for syntax. I've added forPhase to do the same:

forPhase 1 {
    stxrec id {
        rule { $x } => { $x }
    }
}
stxrec m {
    case {_ } => {
        var x = id 42;
        return [makeValue(x, #{here})];
    }
}

There's probably some cleaner syntax we could come up with.

You can import into an arbitrary phase now:

import { m } from "mod.js" for phase 3;