A Glimpse of the Promised Land: Fault-tolerant parsing

For some time, it has been a goal of JavaCC 21 to provide the ability to generate fault-tolerant parsers. I started working on the problem about a year ago. However, I had not put in a comprehensive solution until now for several reasons. Basically these: The codebase, though already significantly refactored and cleaned up, was …

A Glimpse of the Promised Land: Fault-tolerant parsing Read More »

Context-Sensitive Tokenizing, Part Deux: Lexical States

(To get some prerequisite understanding of this topic, it might be a good idea to read this earlier blog post on context-sensitive tokenization from three months ago.) The Lay of the Land There are two quite useful ideas that have been in JavaCC from the very beginning: lookahead (particularly syntactic lookahead) lexical states Syntactic lookahead …

Context-Sensitive Tokenizing, Part Deux: Lexical States Read More »

Tastes just like home-made! (Some more tree building enhancements)

Before getting into what the minor enhancements to tree building are, I guess I should write a quick synopsis of the current state of affairs. When you have TREE_BUILDING_ENABLED set to true (this is the default in JavaCC21) the tree building machinery will build a Node if the production results in the creation of more …

Tastes just like home-made! (Some more tree building enhancements) Read More »

Happy New Year! Some Backward-Incompatible API Changes!

In conjunction with New Year 2021 (a landmark year for JavaCC 21 for sure!) I decided to scratch some longstanding itches — even at the expense of introducing some (minor) backward incompatibilities. The various option settings can only be set in the grammar file. There was a longstanding setup in legacy JavaCC (carried forward in …

Happy New Year! Some Backward-Incompatible API Changes! Read More »

Key Concept: JavaCC (21) is not a Java App!

One of my longstanding interests is language learning — I mean, human languages like Russian or Chinese, not computer languages. One striking thing about language learning is the extreme variability in people’s results. One observes that certain people, frequently very capable in other fields, will have some ongoing project of learning a language, Spanish for …

Key Concept: JavaCC (21) is not a Java App! Read More »