[Jinja] GPT4All-Chat 3.5.1 breaks RichardErkhov/occiglot_-_occiglot-7b-de-en-instruct-gguf

[ThiloteE]

Bug Report

Model does not work.

Steps to Reproduce

1. Download and sideload the model.
2. Check model settings:

Default Jinja Template:

{{'<s>'}}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% set system_message = 'You are a helpful assistant. Please give a long and detailed answer.' %}{% endif %}{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in loop_messages %}{% if loop.index0 == 0 %}{{'<|im_start|>system
' + system_message + '<|im_end|>
'}}{% endif %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}

Better Jinja Template, but does NOT fix it completely:

{{'<s>' }}{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% set system_message = 'You are a helpful assistant. Please give a long and detailed answer.' %}{% endif %}{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in loop_messages %}{% if loop.index0 == 0 %}{{'<|im_start|>system
' + system_message + '<|im_end|>
' }}{% endif %}{{'<|im_start|>' + message['role'] + '
' + message['content'] + '<|im_end|>' + '
' }}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant
' }}{% endif %}

The model responds with emptyness/nothingness:

Expected Behavior

Model works out of the box.

Your Environment

• GPT4All version: 3.5.1
• Operating System: Windows 10
• Chat model used (if applicable): https://huggingface.co/RichardErkhov/occiglot_-_occiglot-7b-de-en-instruct-gguf

Additional Information:

I recommend a diff tool like Meld to compare differences between default and better jinja template.