Let's say I'm trying to parse a string made out of a big list of tokens token 1, token 2, token N, all the way up to token last.

Invalid token.

Token missing

It's a token of their love.

This call, .token, returns the next token that's available.

Getting the token

Changing authentication token

Enter token pin

Send token password...

Need token to be insert. Please insert token in SLOT....

This is a small token of my gratitude.

By (the Token of) Time (through the ages),

Our first token is ' This ' our next token is a word, ' is '.

We're looking for parse token c, and the current token is exactly c.

He held up his hands in token of surrender.

He offered me some money in token of gratitude.

The next letters are the name of the token.

I say the type of this token is 'JAVASCRlPT'.

token password got from user

PIN for unlocking token requested...

It's only seen 1 token.

Your token is invalid. Would you like to get a new token to proceed?

He gave me this doll in token of his gratitude.

Getting the Token from the server

eToken pin for unlocking token requested...

The LANGLESLASH token is surprisingly important.

We've seen the first token, (a).

Then eventually you return the token.

Here's my first token definition rule.

Otherwise, we print out the token.

Our next token is the identifier.

I'm including everything to the left of the current token, making the 1 substitution, and then I'm including everything to the right of the token.

May I present this to you in token of my appreciation?

Here is a present for you in token of our appreciation.

If you surround sections of text that contain a token with curly braces, that section will be hidden if the token is empty.

And he showed him the tremendous token.

Checking if previous token is still valid

like the colon, token, blah, blah, blah.

So that's the one token we're missing.

I declare 1 token that's the email.

And then we return the token unchanged.

So step one you write down a bunch of these token definitions.

These other numbers, after lexical analyzer token STRlNG, with token value, ' This ' are the line and character number information.

I want it to be the LANGLESLASH (token) and not the Left Angle, followed by say a WORD(token).

For each of these possible token positions and for each grammar rule, we removed the starting token and replaced it with the right hand side of the grammar rule.