Commit graph

40 commits

Author SHA1 Message Date
Michael Yang 9d3dcfd0ec fix logging 2024-01-26 11:04:27 -08:00
Michael Yang 6e0ea5ecc8
Merge pull request #1916 from ollama/mxyng/inactivity-monitor
download: add inactivity monitor
2024-01-26 10:56:00 -08:00
Daniel Hiltgen fedd705aea Mechanical switch from log to slog
A few obvious levels were adjusted, but generally everything mapped to "info" level.
2024-01-18 14:12:57 -08:00
Michael Yang 27331ae3a8 download: add inactivity monitor
if a download part is inactive for some time, restart it
2024-01-12 15:23:15 -08:00
Michael Yang 2bb2bdd5d4 fix lint 2024-01-09 09:36:58 -08:00
Michael Yang acfc376efd add .golangci.yaml 2024-01-09 09:36:58 -08:00
Jeffrey Morgan ac5076ce1e exponential backoff up to 30s 2023-11-19 14:32:19 -05:00
Michael Yang cb42589792 adjust download/upload parts 2023-11-19 14:32:19 -05:00
Jeffrey Morgan e1d7056496 update progress statuses 2023-11-19 09:21:13 -05:00
Michael Yang d6ecaa2cbf update progress responses 2023-11-17 10:06:19 -08:00
Daniel Hiltgen cc54a416c6 Resume chunk download on UnexpectedEOF errors
If the chunk download is interrupted, resume from where we left off
2023-11-10 08:29:42 -08:00
Michael Yang 434a6f9d46 return last error 2023-11-03 16:49:51 -07:00
Michael Yang d39709260f download with retry 2023-11-02 13:16:11 -07:00
Michael Yang 60bb3c03a1 use http.Method 2023-11-02 13:12:45 -07:00
Michael Yang 4e09aab8b9 concurrent uploads 2023-10-27 17:07:33 -07:00
Michael Yang 910816a532 fix(download): no retry when out of space 2023-10-26 11:34:07 -07:00
Michael Yang 257ffeb997 fix download 2023-10-12 12:52:43 -07:00
Michael Yang c413a55093 download: handle inner errors 2023-10-11 14:15:30 -07:00
Michael Yang 630bb75d2a dynamically size download parts based on file size 2023-10-11 14:10:25 -07:00
Michael Yang a2055a1e93 update download 2023-10-11 14:10:25 -07:00
Jeffrey Morgan 65dcd0ce35
always cleanup blob download (#747) 2023-10-10 13:12:29 -04:00
Michael Yang 0560b28a8d names 2023-10-06 12:56:56 -07:00
Michael Yang 10199c5987 replace done channel with file check 2023-10-06 12:56:56 -07:00
Michael Yang 288814d3e4 fix ref counts 2023-10-06 12:56:43 -07:00
Michael Yang 04733438da check head request response 2023-10-06 12:56:43 -07:00
Michael Yang 711e891f0f fix resumable downloads
glob returns files in lexical order which is not appropriate when
rebuilding the parts list
2023-10-06 12:56:43 -07:00
Michael Yang 090d08422b handle unexpected eofs 2023-10-06 12:56:43 -07:00
Michael Yang 5b84404c64 handle concurrent requests for the same blobs 2023-10-06 12:56:43 -07:00
Michael Yang 8544edca21 parallel chunked downloads 2023-10-06 12:56:43 -07:00
Michael Yang f40b3de758 use int64 consistently 2023-09-28 11:07:24 -07:00
Michael Yang b58d5d16b0 fix mkdir on windows 2023-09-19 09:41:13 -07:00
Michael Yang 246dc65417 loosen http status code checks 2023-08-28 18:34:53 -04:00
Michael Yang 2cc634689b use url.URL 2023-08-22 10:49:07 -07:00
Michael Yang 2c7f956b38 add version 2023-08-22 09:40:58 -07:00
Bruce MacDonald f0d7c2f5ea retry download on network errors 2023-08-15 15:07:19 -03:00
Bruce MacDonald f020e1d519 always remove from in progress map on download 2023-08-14 13:09:20 -03:00
Patrick Devine d9cf18e28d
add maximum retries when pushing (#334) 2023-08-11 15:41:55 -07:00
Bruce MacDonald 8228d166ce pr comments 2023-08-09 11:31:54 -04:00
Bruce MacDonald 907e6c56b3 unlock downloadu in case or requestDownload err 2023-08-09 11:31:54 -04:00
Bruce MacDonald 868e3b31c7 allow for concurrent pulls of the same files 2023-08-09 11:31:54 -04:00