[Jinja] Mistral Small Instruct 2409 Chat Template and Possible Fix

[Phil209]

Bug Report

The default Mistral Small 2409 chat template (after hitting reset) errors, even though it appears to be correct. And fixing the supposed error (removing the colon in messages[1:]) drastically degrades performance.

According to AI (I'm not a technical user) the colon is needed to subsequently alternate between the user and assistant while skipping the system prompt, yet the colon isn't supported in the included code block prior to the bos token. So it's required, but unsupported, if that makes any sense.

Steps to Reproduce

1. Just load Mistral Small Instruct 2409
2. Hit reset chat template
3. See error message

Expected Behavior

No red box with an error message.

Possible Fix: At the end of this message is the chat template taken from Mistral Small's tokenizer_config.json, which appears to work, although there may be an issue with the system prompt and minor performance degradation, which is probably why you guys added the code that errors out before the bos token, plus a couple other changes.

Your Environment

• GPT4All version: 3.5.1
• Operating System: Windows 11

Below is an apparently working chat template taken from Mistral Small's tokenizer_config.json

{{ bos_token }}
{% for message in messages %}
    {% if message['role'] == 'user' %}
        {{ '[INST] ' + message['content'] + '[/INST]' }}
    {% elif message['role'] == 'system' %}
        {{ '[SYSTEM_PROMPT] ' + message['content'] + '[/SYSTEM_PROMPT]' }}
    {% elif message['role'] == 'assistant' %}
        {{ ' ' + message['content'] + eos_token }}
    {% else %}
        {{ raise_exception('Only user, system and assistant roles are supported!') }}
    {% endif %}
{% endfor %}

Below is the default chat format in GPT4All (when hitting reset) that errors out and I can't figure out how to fix.

{%- if messages[0]['role'] == 'system' %}
    {%- set system_message = messages[0]['content'] %}
    {%- set loop_messages = messages[1:] %}
{%- else %}
    {%- set loop_messages = messages %}
{%- endif %}

{{- bos_token }}
{%- for message in loop_messages %}
    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}
        {{- raise_exception('After the optional system message, conversation roles must alternate user/assistant/user/assistant/...') }}
    {%- endif %}
    {%- if message['role'] == 'user' %}
        {%- if loop.first and system_message is defined %}
            {{- ' [INST] ' + system_message + '\n\n' + message['content'] + ' [/INST]' }}
        {%- else %}
            {{- ' [INST] ' + message['content'] + ' [/INST]' }}
        {%- endif %}
    {%- elif message['role'] == 'assistant' %}
        {{- ' ' + message['content'] + eos_token}}
    {%- else %}
        {{- raise_exception('Only user and assistant roles are supported, with the exception of an initial optional system message!') }}
    {%- endif %}
{%- endfor %}

[manyoso]

We'll have a PR today with documentation for known working templates for different families of models