Month: January 2021

Context-Sensitive Tokenizing, Part Deux: Lexical States

(To get some prerequisite understanding of this topic, it might be a good idea to read this earlier blog post on context-sensitive tokenization from three months ago.) The Lay of the Land There are two quite useful ideas that have been in JavaCC from the very beginning: lookahead (particularly syntactic lookahead) lexical states Syntactic lookahead …

Context-Sensitive Tokenizing, Part Deux: Lexical States Read More »

Tastes just like home-made! (Some more tree building enhancements)

Before getting into what the minor enhancements to tree building are, I guess I should write a quick synopsis of the current state of affairs. When you have TREE_BUILDING_ENABLED set to true (this is the default in JavaCC21) the tree building machinery will build a Node if the production results in the creation of more …

Tastes just like home-made! (Some more tree building enhancements) Read More »

Happy New Year! Some Backward-Incompatible API Changes!

In conjunction with New Year 2021 (a landmark year for JavaCC 21 for sure!) I decided to scratch some longstanding itches — even at the expense of introducing some (minor) backward incompatibilities. The various option settings can only be set in the grammar file. There was a longstanding setup in legacy JavaCC (carried forward in …

Happy New Year! Some Backward-Incompatible API Changes! Read More »