From 7b901310bd18328f69d145e0ecaf80ed60595ed0 Mon Sep 17 00:00:00 2001 From: Dustin Date: Sun, 17 Dec 2023 16:00:01 -0500 Subject: [PATCH] reverted quantization change, seemed to cause unwanted behaviors --- README.md | 4 ++-- ollamarama.py | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index da75153..3a1de68 100644 --- a/README.md +++ b/README.md @@ -14,9 +14,9 @@ You can install it with this command: curl https://ollama.ai/install.sh | sh ``` -Once it's all set up, you'll need to download the model. You can play with the available ones and see what works best for you, but for this bot, zephyr:7b-beta-q6_K seems to work best of the ones I've tested. To install: +Once it's all set up, you'll need to download the model. You can play with the available ones and see what works best for you, but for this bot, zephyr:7b-beta-q8_0 seems to work best of the ones I've tested. To install: ``` -ollama pull zephyr:7b-beta-q6_K +ollama pull zephyr:7b-beta-q8_0 ``` You'll also need to install matrix-nio and litellm diff --git a/ollamarama.py b/ollamarama.py index f369104..20230ca 100644 --- a/ollamarama.py +++ b/ollamarama.py @@ -30,7 +30,7 @@ class ollamarama: self.prompt = ("you are ", ". speak in the first person and never break character.") #set model, this one works best in my tests with the hardware i have, but you can try others - self.model = "ollama/zephyr:7b-beta-q6_K" + self.model = "ollama/zephyr:7b-beta-q8_0" # get the display name for a user