This commit is contained in:
Dustin 2024-03-16 23:58:51 -04:00
parent f20b887073
commit c1677805ed
2 changed files with 5 additions and 3 deletions

View File

@ -100,4 +100,4 @@ To get a longer response, you can tell the bot to "ignore the brackets after thi
When using a coding LLM, remove the personality with the stock command, or set an appropriate personality, such as a python expert.
I have not extensively tested the models included in the json, add and remove models as you see fit. They each have their strenghts and weaknesses. I am using the default 4-bit quant versions for simplicity.
I have not extensively tested the models included in the json, add and remove models as you see fit. They each have their strengths and weaknesses. I am using the default 4-bit quant versions for simplicity, but the quality will be pretty poor. To use a different size model than default, use the full name, for example llama2:7b-chat-q8_0 instead of just llama2.

View File

@ -31,14 +31,16 @@ class ollamarama:
#prompt parts
self.prompt = ("you are ", ". speak in the first person and never break character.")
#put the models you want to use here, still testing various models
#open models.json
with open("models.json", "r") as f:
self.models = json.load(f)
f.close()
#set model
self.default_model = self.models['zephyr']
self.default_model = self.models['mistral']
self.model = self.default_model
#no idea if optimal, change if necessary
self.temperature = .9
self.top_p = .7
self.repeat_penalty = 1.5