Tokenizer

Introduction

The tokenizer functions provide an interface to the PHP tokenizer embedded in the Zend Engine. Using these functions you may write your own PHP source analyzing or modification tools without having to deal with the language specification at the lexical level.

See also the appendix about tokens.

Installing/Configuring
- Installation
Predefined Constants
Examples
PhpToken — The PhpToken class
- PhpToken::__construct — Returns a new PhpToken object
- PhpToken::getTokenName — Returns the name of the token.
- PhpToken::is — Tells whether the token is of given kind.
- PhpToken::isIgnorable — Tells whether the token would be ignored by the PHP parser.
- PhpToken::__toString — Returns the textual content of the token.
- PhpToken::tokenize — Splits given source into PHP tokens, represented by PhpToken objects.
Tokenizer Functions
- token_get_all — Split given source into PHP tokens
- token_name — Get the symbolic name of a given PHP token