Ran into an issue when running on a Raspberry Pi - occaisonally, the
int size for the files being provided from Backblaze (which is in
bytes) will be too large. This is fixed by upgrading the int to int64.
As a part of this debugging process, I upgraded the error handling when
fetching the Backblaze file listing to actually print the errors rather
than failing silently.
Implemented a much-needed Setup() function for providers to implement,
which (as demonstrated in the backblaze provider) allows for
authentication in advance of needing to make calls to remote locations.
This could also be used to create a directory or perform some other
sanity check required for the provider to work. So far, haven't noticed
any performance impacts from this approach, besides not needing to auth
each time we make a request.
Should offer some speed boost, copying the file contents rather than
reading it all to memory then writing it back out. Should theoretically
be able to handle larger files as well, although some benchmarks
are TBD around filesize limits.
Reworked the fileprovider to proxy data directly from the provider
rather than attempt to funkily redirect when needed, since it was overly
complex and wouldn't work well in the long run. Temporarily added "file"
as constant return for determining the filetype through Backblaze.
Added a Makefile to make life easier as well, and rewrote the README to
reflect the refactor/rewrite and new approach to providers.
Expanding this as we go, currently have POC Backblaze B2 support and
basic 'disk' provider as well. Still WIP, but functional for the most
part. Also moving to simplified YAML configuration.
Overall, simplifying things to be extensible down the line. Still work
to be done, but coming along nicely.
