Quex
From Wikipedia, the free encyclopedia
| quex | |
|---|---|
| Developed by | Frank-Rene Schäfer |
| Latest release | 0.23.8 / March 6, 2008 |
| OS | Cross-platform |
| Genre | Lexical analyzer generator |
| License | LGPL (with military use exclusion) |
| Website | quex.sourceforge.net |
Quex is a lexical analyzer generator implemented in the Python programming language that creates C++ lexical analyzers. Significant features include the ability to generate lexical analyzers that operate on Unicode input, the creation of direct coded (non-table based) lexical analyzers and the use of inheritance relationships in lexical analysis modes.
Contents |
[edit] Features
[edit] Direct Coded Lexical Analyzers
Quex uses traditional steps of Thompson construction to create nondeterministic finite state machines from regular expressions, conversion to a deterministic finite-state machine and then Hopcroft optimization to reduce the number of states to a minimum. However, instead of construction of a table based lexical analyzer where the transition information is stored in a data structure, quex generates C++ code to perform transitions.
Direct coding creates lexical analyzers that structurally more closely resemble typical hand written lexical analyzers than table based lexers. Also direct coded lexers tend to perform better than analogous table based lexical analyzers.
[edit] Unicode Input Alphabets
Quex can handle input alphabets that contain the full Unicode code point range (0 to 10FFFFh). This is augmented by the ability to specify regular expressions that contain Unicode properties as expressions. For example, Unicode code points with the binary property XID_Start can be specified with the expression \P{XID_Start} or \P{XIDS}. Quex can also generate code to call iconv to perform character conversion.
[edit] Lexical Analysis Modes
Like traditional lexical analyzers like Lex or Flex, quex supports multiple lexical analysis modes in a lexer. In addition to pattern actions, quex modes can specify event actions: code to be execute during events such as entering or exiting a mode or when any match is found. Quex modes can be also be related by inheritance which allows modes to share common pattern and event actions.
[edit] See also
- Lexical analysis
- Descriptions of Thompson construction and Hopcroft optimization can be found in most textbooks on compiler construction such as Compilers: Principles, Techniques, and Tools.
