Ooh this is exactly what I've been thinking about. Text is such a slow, clunky medium. It'd be interesting if you could think of versions as events modifying a tree. Renaming a variable and inserting a character would both be an event. Also I wonder if structural editing will take over. IDEs are already so powerful that if you could create good keybindings, you could do so much with just IDE commands (generate expr, rename var, swap args, etc.). Then if your editor knows that it will always keep a valid AST, what can you do with your tooling?
Text is so clunky, especially in languages with superfluous syntax (semicolon, braces). My tree based outliner allows me to easily rearrange arbitrarily large blocks while never creating invalid syntax, why the heck doesn’t my IDE? Code is just a damn tree. Why can’t I arbitrarily choose to comment out/in code without breaking basically all the IDE tooling (collapsed a block? Well too bad!!)?
We should never have to think about syntax. Yet we (or certainly I) do a significant portion of the time.
The stuff I’m thinking of should be fairly possible to do as a Vscodium/VSCode plugin. Can somebody please tell me it’s already being done?
Those semicolons are redundant but not superfluous. Here are some good reasons why you might want to keep them around even in they aren’t strictly necessary in parsing your program.
I don’t think that always keeping a valid AST is important. Realtime highlighting of syntax errors already resumes parsing after invalid code, usually mapping to error nodes internally. That is, you still have an AST, just with additional node types. Having an interim state with error nodes isn’t really different from having intermediate states with temporary (possibly large) changes in valid code, e.g. where you move/cut/paste larger portions of code around, and then maybe decide to change it back (or just change back some parts). Creating a sensible history of AST operations doesn’t really depend on whether you have error nodes in your AST grammar or not.
On the other hand, allowing error nodes (i.e. invalid code) at least as an intermediate state arguably allows more freedom and creativity when editing code, and feels less coercive. It is also unavoidable in certain contexts, such as while typing an identifier, the identifier may be invalid in most intermediate states until you have finished typing it.
Therefore I’m unconvinced that restricting editing to valid ASTs is (a) critical to collaborative editing and versioning, and (b) strictly desirable from a usability perspective.
Agreed, structure editors must have interim/liminal representations that are only partially formed, or else the editing experience doesn't flow. I do think there's value in tracking the movement/mutation of AST nodes with the granularity and precision only achievable by a structure editor, so I hope someone cracks the formula.
One thing I think would be cool is if you could select a region of code and move it up/down the abstraction ladder e.g. convert an AST subtree into tokens, or all the way down to individual UTF8 characters. Then do some text manipulation on the plaintext, and finally "cook" them back together into AST nodes. The real cool trick would be to have the editor diligently track the flow of nodes/identifiers spanning the transmutation.
JetBrains MPS is such an IDE, for a while I was also fascinated by this approach (structured editing) but I now think that the freedom character level editing gives is non-negotiable for me
it is easier to ask for forgiveness than for permission, the same is true for syntax errors, it is easier to report errors (and fix them) than trying to always maintain a document in a valid state
