2023-06-27 12:08:52 -04:00
|
|
|
# Ollama
|
2023-06-22 12:45:31 -04:00
|
|
|
|
2023-07-03 15:22:44 -04:00
|
|
|
A fast runtime for large language models, powered by [llama.cpp](https://github.com/ggerganov/llama.cpp).
|
2023-06-27 17:13:07 -04:00
|
|
|
|
2023-06-30 11:39:24 -04:00
|
|
|
> _Note: this project is a work in progress. Certain models that can be run with `ollama` are intended for research and/or non-commercial use only._
|
2023-06-28 09:57:36 -04:00
|
|
|
|
|
|
|
## Install
|
2023-06-22 12:45:31 -04:00
|
|
|
|
2023-06-30 12:39:25 -04:00
|
|
|
Using `pip`:
|
|
|
|
|
2023-06-22 12:45:31 -04:00
|
|
|
```
|
2023-06-27 12:08:52 -04:00
|
|
|
pip install ollama
|
2023-06-22 12:45:31 -04:00
|
|
|
```
|
|
|
|
|
2023-06-30 12:39:25 -04:00
|
|
|
Using `docker`:
|
|
|
|
|
2023-06-30 12:31:00 -04:00
|
|
|
```
|
|
|
|
docker run ollama/ollama
|
|
|
|
```
|
|
|
|
|
2023-06-28 09:57:36 -04:00
|
|
|
## Quickstart
|
2023-06-25 13:08:03 -04:00
|
|
|
|
2023-06-29 18:25:02 -04:00
|
|
|
To run a model, use `ollama run`:
|
|
|
|
|
|
|
|
```
|
|
|
|
ollama run orca-mini-3b
|
2023-06-25 13:08:03 -04:00
|
|
|
```
|
|
|
|
|
2023-06-29 18:25:02 -04:00
|
|
|
You can also run models from hugging face:
|
2023-06-27 12:08:52 -04:00
|
|
|
|
2023-06-29 18:25:02 -04:00
|
|
|
```
|
|
|
|
ollama run huggingface.co/TheBloke/orca_mini_3B-GGML
|
|
|
|
```
|
2023-06-28 09:57:36 -04:00
|
|
|
|
2023-06-29 18:25:02 -04:00
|
|
|
Or directly via downloaded model files:
|
|
|
|
|
|
|
|
```
|
|
|
|
ollama run ~/Downloads/orca-mini-13b.ggmlv3.q4_0.bin
|
2023-06-28 09:57:36 -04:00
|
|
|
```
|
|
|
|
|
2023-06-27 13:46:46 -04:00
|
|
|
## Documentation
|
|
|
|
|
|
|
|
- [Development](docs/development.md)
|
2023-07-01 17:54:29 -04:00
|
|
|
- [Python SDK](docs/python.md)
|