[Jinja] GPT4All-Chat 3.5.1 Chat template syntax for MaziyarPanahi_Qwen2-1.5B-Instruct-GGUF

[ThiloteE]

Bug Report

I experience Jinja syntax errors.

Steps to Reproduce

Use the model with new default jinja template, introduced in GPT4All-Chat 3.5.1

Default (reset) template:

{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system
You are a helpful assistant.<|im_end|>
' }}{% endif %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}

Better template:

{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system
You are a helpful assistant.<|im_end|>
' }}{% endif %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}

Expected Behavior

No problems

Your Environment

• GPT4All version: 3.5.1
• Operating System: Windows 10
• Chat model used (if applicable): Qwen2-1.5B-Instruct.Q6_K.gguf (https://huggingface.co/MaziyarPanahi/Qwen2-1.5B-Instruct-GGUF/tree/main)

[ThiloteE]

So, I think many models have problems with space after '. In particular the ones that follow basic ChatML syntax (the ones that have im_start|> and <|im_end|>).

Example for another model with the same issues: https://huggingface.co/FPHam/Karen_TheEditor_V2_CREATIVE_Mistral_7B

Default jinja in GPT4All that causes error:

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}

Better:

{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}

Notice, I only added a space on the left side of the code block, right on the second character position of line 4.

[ThiloteE]

Even better Jinja chat template:

{%- for message in messages %}
    {%- if loop.first and messages[0]['role'] != 'system' %}
        {{  '<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n' }}
    {%- endif %}
        {{ '<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n' }}
{%- endfor %}

{%- if add_generation_prompt %}
    {{ '<|im_start|>assistant\n' }}
{%- endif %}