From 516d88e75c9e768c0001a452dbad212494c586b3 Mon Sep 17 00:00:00 2001 From: Georgi Gerganov Date: Wed, 29 Mar 2023 19:37:20 +0300 Subject: [PATCH] readme : add GPT4All instructions (close #588) --- README.md | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 5675a92..c2323f4 100644 --- a/README.md +++ b/README.md @@ -10,9 +10,7 @@ Inference of [LLaMA](https://arxiv.org/abs/2302.13971) model in pure C/C++ **Hot topics:** - [Roadmap (short-term)](https://github.com/ggerganov/llama.cpp/discussions/457) -- New C-style API is now available: https://github.com/ggerganov/llama.cpp/pull/370 -- Cache input prompts for faster initialization: https://github.com/ggerganov/llama.cpp/issues/64 -- Create a `llama.cpp` logo: https://github.com/ggerganov/llama.cpp/issues/105 +- Support for [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all) ## Description @@ -37,6 +35,12 @@ Supported platforms: - [X] Windows (via CMake) - [X] Docker +Supported models: + +- [X] LLaMA +- [X] [Alpaca](https://github.com/ggerganov/llama.cpp#instruction-mode-with-alpaca) +- [X] [GPT4All](https://github.com/ggerganov/llama.cpp#using-gpt4all) + --- Here is a typical run using LLaMA-7B: @@ -222,6 +226,17 @@ cadaver, cauliflower, cabbage (vegetable), catalpa (tree) and Cailleach. > ``` +### Using [GPT4All](https://github.com/nomic-ai/gpt4all) + +- Obtain the `gpt4all-lora-quantized.bin` model +- It is distributed in the old `ggml` format which is not obsoleted. So you have to convert it to the new format using [./convert-gpt4all-to-ggml.py](./convert-gpt4all-to-ggml.py): + + ```bash + python3 convert-gpt4all-to-ggml.py models/gpt4all-7B/gpt4all-lora-quantized.bin ./models/tokenizer.model + ``` + +- You can now use the newly generated `gpt4all-lora-quantized.bin` model in exactly the same way as all other models. The original model is stored in the same folder with a suffix `.orig` + ### Obtaining and verifying the Facebook LLaMA original model and Stanford Alpaca model data - **Under no circumstances share IPFS, magnet links, or any other links to model downloads anywhere in this respository, including in issues, discussions or pull requests. They will be immediately deleted.**