replaced dictionary with json
This commit is contained in:
Dustin
parent
090870d7b9
commit
4af6a43ffc
20
README.md
20
README.md
@ -1,7 +1,7 @@
|
||||
# ollamarama-matrix
|
||||
Ollamarama is an AI chatbot for the [Matrix](https://matrix.org/) chat protocol using LiteLLM and Ollama. It can roleplay as almost anything you can think of. You can set any default personality you would like. It can be changed at any time, and each user has their own separate chat history with their chosen personality setting. Users can interact with each others chat histories for collaboration if they would like, but otherwise, conversations are separated, per channel, per user.
|
||||
|
||||
This is based on my earlier project, [infinigpt-matrix](https://github.com/h1ddenpr0cess20/infinigpt-matrix), which uses OpenAI and costs money to use.
|
||||
This is based on my earlier project, [infinigpt-matrix](https://github.com/h1ddenpr0cess20/infinigpt-matrix), which uses OpenAI and costs money to use. (Now updated with OpenAI/Ollama model switching)
|
||||
|
||||
IRC version available at [ollamarama-irc](https://github.com/h1ddenpr0cess20/ollamarama-irc)
|
||||
|
||||
@ -11,13 +11,13 @@ Terminal-based version at [ollamarama](https://github.com/h1ddenpr0cess20/ollama
|
||||
|
||||
Install and familiarize yourself with [Ollama](https://ollama.ai/), make sure you can run offline LLMs, etc.
|
||||
|
||||
You can install it with this command:
|
||||
You can install and update it with this command:
|
||||
```
|
||||
curl https://ollama.ai/install.sh | sh
|
||||
```
|
||||
|
||||
|
||||
Once it's all set up, you'll need to [download the models](https://ollama.ai/library) you want to use. You can play with the available ones and see what works best for you. Add those to the self.models dictionary. If you want to use the ones I've included, just run the commands in the models.md file. You can skip this part, and they should download when the model is switched, but the response will be delayed until it finishes downloading.
|
||||
Once it's all set up, you'll need to [download the models](https://ollama.ai/library) you want to use. You can play with the available ones and see what works best for you. Add those to the models.json file. If you want to use the ones I've included, just run ollama pull modelname. You can skip this part, and they should download when the model is switched, but the response will be delayed until it finishes downloading.
|
||||
|
||||
|
||||
You'll also need to install matrix-nio and litellm
|
||||
@ -45,7 +45,7 @@ python3 launcher.py
|
||||
|
||||
**.persona _personality_**
|
||||
Changes the personality. It can be a character, personality type, object, idea.
|
||||
Don't use a custom prompt here.
|
||||
Don't use a custom system prompt here.
|
||||
|
||||
**.custom _prompt_**
|
||||
Allows use of a custom system prompt instead of the roleplaying prompt
|
||||
@ -90,4 +90,14 @@ python3 launcher.py
|
||||
Change global personality (bot owner only)
|
||||
|
||||
**.gpersona reset**
|
||||
Reset global personality (bot owner only)
|
||||
Reset global personality (bot owner only)
|
||||
|
||||
|
||||
|
||||
## Tips
|
||||
|
||||
To get a longer response, you can tell the bot to "ignore the brackets after this message".
|
||||
|
||||
When using a coding LLM, remove the personality with the stock command, or set an appropriate personality, such as a python expert.
|
||||
|
||||
I have not extensively tested the models included in the json, add and remove models as you see fit. They each have their strenghts and weaknesses. I am using the default 4-bit quant versions for simplicity.
|
||||
21
models.json
Normal file
21
models.json
Normal file
@ -0,0 +1,21 @@
|
||||
{
|
||||
"zephyr": "ollama/zephyr",
|
||||
"solar": "ollama/solar",
|
||||
"mistral": "ollama/mistral",
|
||||
"llama2": "ollama/llama2",
|
||||
"llama2-uncensored": "ollama/llama2-uncensored",
|
||||
"openchat": "ollama/openchat",
|
||||
"codellama": "ollama/codellama",
|
||||
"dolphin-mistral": "ollama/dolphin-mistral",
|
||||
"orca2": "ollama/orca2",
|
||||
"starling-lm": "ollama/starling-lm",
|
||||
"phi": "ollama/phi",
|
||||
"orca-mini": "ollama/orca-mini",
|
||||
"wizardcoder": "ollama/wizardcoder:python",
|
||||
"stablelm-zephyr": "ollama/stablelm-zephyr",
|
||||
"neural-chat": "ollama/neural-chat",
|
||||
"mistral-openorca": "ollama/mistral-openorca",
|
||||
"dolphin-phi": "ollama/dolphin-phi",
|
||||
"llama-pro": "ollama/llama-pro",
|
||||
"gemma": "ollama/gemma"
|
||||
}
|
||||
39
models.md
39
models.md
@ -1,39 +0,0 @@
|
||||
ollama pull zephyr:7b-beta-q8_0
|
||||
|
||||
ollama pull solar
|
||||
|
||||
ollama pull mistral
|
||||
|
||||
ollama pull llama2
|
||||
|
||||
ollama pull llama2-uncensored
|
||||
|
||||
ollama pull openchat
|
||||
|
||||
ollama pull codellama:13b-instruct-q4_0
|
||||
|
||||
ollama pull dolphin2.2-mistral:7b-q8_0
|
||||
|
||||
ollama pull deepseek-coder:6.7b
|
||||
|
||||
ollama pull orca2
|
||||
|
||||
ollama pull starling-lm
|
||||
|
||||
ollama pull vicuna:13b-q4_0
|
||||
|
||||
ollama pull phi
|
||||
|
||||
ollama pull orca-mini
|
||||
|
||||
ollama pull wizardcoder:python
|
||||
|
||||
ollama pull stablelm-zephyr
|
||||
|
||||
ollama pull neural-chat
|
||||
|
||||
ollama pull mistral-openorca
|
||||
|
||||
ollama pull deepseek-llm:7b-chat
|
||||
|
||||
ollama pull wizard-vicuna-uncensored
|
||||
@ -9,6 +9,7 @@ import asyncio
|
||||
from nio import AsyncClient, MatrixRoom, RoomMessageText
|
||||
import datetime
|
||||
from litellm import completion
|
||||
import json
|
||||
|
||||
class ollamarama:
|
||||
def __init__(self, server, username, password, channels, personality, admins):
|
||||
@ -31,29 +32,9 @@ class ollamarama:
|
||||
self.prompt = ("you are ", ". speak in the first person and never break character.")
|
||||
|
||||
#put the models you want to use here, still testing various models
|
||||
self.models = {
|
||||
'zephyr': 'ollama/zephyr:7b-beta-q8_0',
|
||||
'solar': 'ollama/solar',
|
||||
'mistral': 'ollama/mistral',
|
||||
'llama2': 'ollama/llama2',
|
||||
'llama2-uncensored': 'ollama/llama2-uncensored',
|
||||
'openchat': 'ollama/openchat',
|
||||
'codellama': 'ollama/codellama:13b-instruct-q4_0',
|
||||
'dolphin-mistral': 'ollama/dolphin2.2-mistral:7b-q8_0',
|
||||
'deepseek-coder': 'ollama/deepseek-coder:6.7b',
|
||||
'orca2': 'ollama/orca2',
|
||||
'starling-lm': 'ollama/starling-lm',
|
||||
'vicuna': 'ollama/vicuna:13b-q4_0',
|
||||
'phi': 'ollama/phi',
|
||||
'orca-mini': 'ollama/orca-mini',
|
||||
'wizardcoder': 'ollama/wizardcoder:python',
|
||||
'stablelm-zephyr': 'ollama/stablelm-zephyr',
|
||||
'neural-chat': 'ollama/neural-chat',
|
||||
'mistral-openorca': 'ollama/mistral-openorca',
|
||||
'deepseek-llm': 'ollama/deepseek-llm:7b-chat',
|
||||
'wizard-vicuna-uncensored': 'ollama/wizard-vicuna-uncensored'
|
||||
|
||||
}
|
||||
with open("models.json", "r") as f:
|
||||
self.models = json.load(f)
|
||||
|
||||
#set model
|
||||
self.default_model = self.models['zephyr']
|
||||
self.model = self.default_model
|
||||
|
||||
Loading…
Reference in New Issue
Block a user