Announcement: JavaCC 21 includes parsers for the latest Java and Python versions!

As the title says: as of this writing, the included grammars for Java and Python both support the latest versions of the language, JDK 17 and Python 3.10 respectively. Contextual Keywords It turns out that, from a language support perspective, the only new stable feature in JDK 17 is the concept of sealed classes. This …

Announcement: JavaCC 21 includes parsers for the latest Java and Python versions! Read More »

Context-Sensitive Tokenization, Next Installment, Activating and De-activating Tokens

Sometimes, when you complete a major code cleanup, features that were previously pie in the sky become low-hanging fruit to pluck. The new feature that I describe here, the ability to activate and deactivate tokens is such a case. It resulted from my rewriting of the lexical code generation that I describe here. In an …

Context-Sensitive Tokenization, Next Installment, Activating and De-activating Tokens Read More »

Major Milestone: The Lexical Code Generation is completely rewritten

Dear Readers… This blog has been dark for about two months now, but not because I was inactive in the project. Quite the opposite actually. What happened over the last couple of months is that I went into full-blown obsessive mode and managed to rewrite the remaining part of JavaCC that had been resisting my …

Major Milestone: The Lexical Code Generation is completely rewritten Read More »

The Dreaded “Code too large” Problem is a Thing of the Past

"It’s too big! It doesn’t fit!" The above does not refer to any particular pornographic feature film, but rather, to a longstanding problem in JavaCC: if you write a very big, complex lexical grammar, the generated XXXTokenManager would fail to compile, with the compiler reporting the error: "Code too large". Well, this has now been …

The Dreaded “Code too large” Problem is a Thing of the Past Read More »

Moving Towards a Maximally Correct Reference Java Grammar

Now that the ability to generate fault-tolerant parsers is coming along so well, I have been thinking about what to do with the Java grammar included in JavaCC21. I decided that the best thing to do was to do the incremental work to make it maximally correct. Ideally, it will serve as a reference implementation …

Moving Towards a Maximally Correct Reference Java Grammar Read More »

A Glimpse of the Promised Land: Fault-tolerant parsing

For some time, it has been a goal of JavaCC 21 to provide the ability to generate fault-tolerant parsers. I started working on the problem about a year ago. However, I had not put in a comprehensive solution until now for several reasons. Basically these: The codebase, though already significantly refactored and cleaned up, was …

A Glimpse of the Promised Land: Fault-tolerant parsing Read More »

Is this Parsing theory just Bullshit: Part Deux

A Little Parlor Game As I pointed out earlier here, in this parser space, there is a great tendency to express concepts — that, once understood, are actually quite simple — in a very abstruse, obfuscated manner. Recently, I was musing about a little conceptual experiment with some real comic potential. Imagine if basic Mathematics …

Is this Parsing theory just Bullshit: Part Deux Read More »