* Fixes#1147: template strings not obeying -b ascii_only true
* Allow evaluation of template expressions by adding optimizers and
walkers
* Make sure tagged templates are never changed
* Remove template tokenizer in parser, add template tokenizer in
tokenizer. It is using a brace counter to track brace position of
templates
* Add tokens `template_head` and `template_substitution` but parsing
tokens stays mostly the same
* Do not output strings anymore in AST_TemplateString, instead use
AST_TemplateSegment
* Fix parsing tagged templates, allowing multiple templates behind
as spec allows this
These changes don't influence tagged templates because raw content
may influence code execution, however they are safe to do in normal
templates:
* Allow basic string concatenation of templates where possible
* Allow custom character escape style similar to strings, except in
tagged templates
Note that expressions are still compressed in tagged templates.
Optional things that may be improved later:
* Custom quote style for templates if it doesn't have expressions.
Making it obey the quote_style option if this is the case.
0af42d1831