gered/ollama
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

Document response stream chunk delimiter. (#632)

Browse source 
Document response stream chunk delimiter.
This commit is contained in:
Jay Nakrani 2023-09-30 00:45:52 -04:00 committed by GitHub
parent a1b2d95f96
commit 1d0ebe67e8
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
2 changed files with 5 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
docs/api.md
View file

@ -23,6 +23,10 @@ Model names follow a `model:tag` format. Some examples are `orca-mini:3b-q4_1` a
All durations are returned in nanoseconds. All durations are returned in nanoseconds.
### Streaming responses
Certain endpoints stream responses as JSON objects delineated with the newline (`\n`) character.
## Generate a completion ## Generate a completion
```shell ```shell

1
server/routes.go
View file

@ -610,6 +610,7 @@ func streamResponse(c *gin.Context, ch chan any) {
return false return false
} }
// Delineate chunks with new-line delimiter
bts = append(bts, '\n') bts = append(bts, '\n')
if _, err := w.Write(bts); err != nil { if _, err := w.Write(bts); err != nil {
log.Printf("streamResponse: w.Write failed with %s", err) log.Printf("streamResponse: w.Write failed with %s", err)