How to properly deploy the Dobby AI model locally on Windows.
Let's move on to the guide:
1. Loading the model
Go to the HuggingFace website .
If you don't have an account , create one.
Go to the Sentient page.
Go to models:
Click on it:
Go to the "Files and versions" page.
Before doing so, create a folder on your PC dobby_modeland upload the selected model there.
2. Installing Ollama
Ollamais a tool (program) for running local models.
Download and install it from this link. Once installed, run the application — it will open in PowerShell. The application will automatically prompt you for the following command to install. If this does not happen, review the installation details.
3. Create a file, “ Modelfile “
Modelfileis a simple file that tells Ollama where to get your model from.
To do this, open any text editor (I use Notepad++ ), create a file Modelfilewithout the .txt extension , and save it in the same folder as your model.
Enter the following line into the file and save the changes:
FROM ./dobby-8b-unhinged-q4_k_m.gguf
- Model launch When the Modelfile is ready, use the console to navigate to the folder where the model and file are located, for example:
cd D:\dobby_model Enter the following command:
ollama create dobby-unhinged -f Modelfile dobby-unhinged– this is the name you give to the model (you can choose any, the main thing is to use it the same way later).
-f Modelfileindicates that we are taking instructions from your Modelfile.
After executing this command, Ollama will "build" its internal database about the model. This may take some time, depending on the file size and the power of your computer.
Now that the model is "created", you can run it:
ollama run dobby-unhinged dobby-unhinged– this is the same name you specified in the previous step.
After this command, Ollama will prepare itself and open a request-response interface in your terminal (it may look like a prompt >where you can type something).
x.com/amrit12005