3. Lexical Analysis and Its Trustworthiness Principles

  1. Vladimir O. Safonov

Published Online: 8 FEB 2010

DOI: 10.1002/9780470593387.ch3

Trustworthy Compilers

Trustworthy Compilers

How to Cite

Safonov, V. O. (2010) Lexical Analysis and Its Trustworthiness Principles, in Trustworthy Compilers, John Wiley & Sons, Inc., Hoboken, NJ, USA. doi: 10.1002/9780470593387.ch3

Author Information

  1. St. Petersburg University, USA

Publication History

  1. Published Online: 8 FEB 2010
  2. Published Print: 24 FEB 2010

ISBN Information

Print ISBN: 9780470500958

Online ISBN: 9780470593387



  • lexical analysis and trustworthiness principles;
  • models of lexical structure of programming language - automata grammars, finite-state automata, regular expressions;
  • lexical analyzer architecture and implementation principles


This chapter contains sections titled:

  • Token Classes

  • The Output of the Lexical Analyzer

  • Processing White Spaces, Comments, and New Lines

  • Theoretical Models of Lexical Analysis

  • Lexical Errors, Error Diagnostics, and Recovery

  • Processing Identifiers and Keywords

  • The Architecture of a Lexical Analyzer and the Principles of Its Implementation

  • The Lexical Analyzer Generator Lex

  • Lexical Analyzer Generation in ANTLR

  • Exercises to Chapter 3