ALERT

[FREE DEMO] Deploy Your Enterprise Cloud in Minutes

Lexical Analysis

Definition - What does Lexical Analysis mean?

Lexical analysis is a concept that is applied to computer science in a very similar way that it is applied to linguistics. Essentially, lexical analysis means grouping a stream of letters or sounds into sets of units that represent meaningful syntax. In linguistics, it is called parsing, and in computer science, it can be called parsing or tokenizing.

Techopedia explains Lexical Analysis

The idea of lexical analysis in computer science is that lexical analysis breaks streams down into “lexemes” where a token represents the basic unit of meaning. Tokens are strung together in such a way that the language compiler must go back and isolate them to implement the right computing instructions. Basically, both humans and computers do lexical analysis, but computers do it differently, and in a much more technical way. The way that computers do lexical analysis does not need to be transparent to humans – it just has to be programmed into the computing system. Programs that do lexical analysis in computer science are often called lexers, tokenizers or scanners.

Techopedia Deals

Connect with us

Techopedia on Linkedin
Techopedia on Linkedin
Tweat cdn.techopedia.com
"Techopedia" on Twitter


'@Techopedia'
Sign up for Techopedia's Free Newsletter!

Email Newsletter

Join thousands of others with our weekly newsletter

Resources
The 4th Era of IT Infrastructure: Superconverged Systems
The 4th Era of IT Infrastructure: Superconverged Systems:
Learn the benefits and limitations of the 3 generations of IT infrastructure – siloed, converged and hyperconverged – and discover how the 4th...
Approaches and Benefits of Network Virtualization
Approaches and Benefits of Network Virtualization:
Businesses today aspire to achieve a software-defined datacenter (SDDC) to enhance business agility and reduce operational complexity. However, the...
Free E-Book: Public Cloud Guide
Free E-Book: Public Cloud Guide:
This white paper is for leaders of Operations, Engineering, or Infrastructure teams who are creating or executing an IT roadmap.
Free Tool: Virtual Health Monitor
Free Tool: Virtual Health Monitor:
Virtual Health Monitor is a free virtualization monitoring and reporting tool for VMware, Hyper-V, RHEV, and XenServer environments.
Free 30 Day Trial – Turbonomic
Free 30 Day Trial – Turbonomic:
Turbonomic delivers an autonomic platform where virtual and cloud environments self-manage in real-time to assure application performance.