Lexical Analysis

What Does Lexical Analysis Mean?

Lexical analysis is a concept that is applied to computer
science in a very similar way that it is applied to linguistics. Essentially,
lexical analysis means grouping a stream of letters or sounds into sets of
units that represent meaningful syntax. In linguistics, it is called parsing,
and in computer science, it can be called parsing or tokenizing.

Advertisements

Techopedia Explains Lexical Analysis

The idea of lexical analysis in computer science is that
lexical analysis breaks streams down into “lexemes” where a token represents
the basic unit of meaning. Tokens are strung together in such a way that the
language compiler must go back and isolate them to implement the right
computing instructions. Basically, both humans and computers do lexical
analysis, but computers do it differently, and in a much more technical way.
The way that computers do lexical analysis does not need to be transparent to
humans – it just has to be programmed into the computing system. Programs that
do lexical analysis in computer science are often called lexers, tokenizers or
scanners.

Advertisements

Related Terms

Latest Computer Science Terms

Related Reading

Margaret Rouse

Margaret Rouse is an award-winning technical writer and teacher known for her ability to explain complex technical subjects to a non-technical, business audience. Over the past twenty years her explanations have appeared on TechTarget websites and she's been cited as an authority in articles by the New York Times, Time Magazine, USA Today, ZDNet, PC Magazine and Discovery Magazine.Margaret's idea of a fun day is helping IT and business professionals learn to speak each other’s highly specialized languages. If you have a suggestion for a new definition or how to improve a technical explanation, please email Margaret or contact her…