CustomTokenizer
CustomTokenizer = (
code,grammar) =>TokenStream
Defined in: prism/types.ts:97
A custom tokenizer for a grammar.
Parameters
Section titled “Parameters”string
A string with the code this grammar needs to tokenize.
grammar
Section titled “grammar”The grammar with the custom tokenizer
Returns
Section titled “Returns”A token stream representing the matched code.
tokenize symbol for more info.