Embed file source reference inside the AST?
This may be controversial, but I think it wouldn't be a bad idea to embed the location of a source file inside the rowan AST (at the root), in a way that is — obviously — hidden to text editors, LSP servers or pretty-printers.
What for?
This would help with diagnostic emitting, include
resolution and maybe other parts, as ASTs wouldn't need to be accompanied by a path (or require an external global mapping of Root
→ SourceFile
as it's the case right now.
Rationale
Due to include
s supporting relative paths, it's actually not absurd at all to consider the source code file's location as an intrinsic part of it, even if it's not "source code" strictly speaking. By embedding the location inside the file, an AST will contain absolutely everything any AST node consumer may possibly need.
Implementation
Attention should be given to the potential future need for "virtual" files, such as remote files provided by an LSP client or the embedded standard library file (c.f. #26).
Unfortunately, this idea looks a bit tricky to implement as rowan doesn't support adding arbitrary metadata to nodes. Two approches could be:
- Make the file source path an integral part of the "source" code (at the end so it doesn't mess up with character/line numbers), and filter it out as appropriate
- Use the fact that a
u16
can represent much more values than our currentToken
enum uses, and make the remaining values — inside token tree nodes — act as keys to a global table to arbitrary Rust objects. This requires manually freeing entries when ASTs are destroyed, though. This ends up being similar to what we have in place right now, but at least we're not relying on unreliable objectEq
identity as keys.