Tokenize (None)
This is an optional command.
You can set Tokenize
to None
to stop field tokenization. When Tokenize
is set to None
, the parsing grammar rule must include any spaces or other token separators within its rule definition.
To use this command:
- Position the cursor where you want the command inserted.
- Double-click %Tokenize(None) in the Commands list.