| 00:02 | <jmdyck> | In general, the parser should, at any point, know the set of terminal symbols (input elements) that could validly occur next, given everything to the left. On the basis of that set, it can tell the lexer which InputElementFoo nonterminal to use as the start symbol for the next input element. |
| 01:24 | <sirisian> | Thanks for all that information. That helps clarify things for me. |
| 11:50 | <jmdyck> | A different formulation would have the parser, at each point, tell the lexer all the terminal symbols that it's expecting, and ask for one of those. That would eliminate the InputElementFoo nonterminals and the rules about when each is appropriate, and a few other nonterminals like CommonToken. So simpler in some sense. But the current formulation minimizes the information that goes from the parser to the lexer, so simpler in a different sense. |