Making some improvements to the future maintainability of the project,
namely in the form of tests. Also implemented a new "dist" command to
the Makefile that packs up the binaries, and will eventually be expanded
to do more legwork when prepping for a release.
Updated file providers to use io.Reader over an explicit multipart file,
and removed the handler in favour of a plain filename, since that was
it's primary purpose.
Frontend logic for routing was a bit broken when generating links for
directories, appending an extra / to the start. Resolved this to
properly link - interestingly, the file listing could still be served,
but the uploading was broken?
Also enforced regex for provider of directory listings and tidied up
some other string handling.
Removed common package, since it was largely useless and instead
integrated what I could into the router package. Should simplify things
going forward, but we can always split this out later. Also completely
removed authentication for the time being - want to consider other
options for this.
Addressing a bug where the configuration would try to interpret the
provider name from the entry in the configuration, rather than the
explicit `provider`. Also removed background uploading from Backblaze
(was worth a shot).
Implemented file upload progress to the frontend using XMLHttpRequests,
since fetch() doesn't offer this functionality right now. Bit of a
fragmented approach to requests in the codebase right now, but hope to
resolve this soon. Not styled at all either.
Also added some nifty commands to the Makefile for producing smaller
binaries using some compiler flags and upx, and requires upx to be
installed (make small and make small_pi).
Expanded the README with more up-to-date data and elaboration on
providers, which may have been a bit unclear before. Also added a few
configuration examples for each provider, with comments.
Implemented file uploading for both the disk and Backblaze providers.
Also implements a UI element and frontend logic for doing so from the
frontend.
Disk will write directly to disk, while the Backblaze provider will
attempt to stream the file from memory directly to the POST body. This
could introduce some problems down the line, so caching to disk then
uploading in the background may need to be implemented. It also
performs the final upload using a goroutine so the end client can
continue on it's merry way.
Backblaze proved a bit tricky to do, and considering switching to using
a library for managing Backblaze going forward.
Ran into an issue when running on a Raspberry Pi - occaisonally, the
int size for the files being provided from Backblaze (which is in
bytes) will be too large. This is fixed by upgrading the int to int64.
As a part of this debugging process, I upgraded the error handling when
fetching the Backblaze file listing to actually print the errors rather
than failing silently.
Started work on a new basic frontend, with some magic code from
@tomhodgins (Tom Hodgins) for templating and processing data, which was
the primary purpose for considering a full-fledged JavaScript
framework. Fairly simple right now, with minimal "bling", and seems
"functional enough". Might need to consider options for using less or
no JS down the line if there is demand.
As part of this frontend, actually implemented route logic for fetching
a sorted list of providers (might be worth investigating whether
storing this final result in memory is worth the performance to memory
trade-off).
Also fixed a bug where adding "/" to the end of a provider name without
a file path would result in a 404. This was addressed with some Regex
filtering on the path that should be able to handle that matching.
Implemented a much-needed Setup() function for providers to implement,
which (as demonstrated in the backblaze provider) allows for
authentication in advance of needing to make calls to remote locations.
This could also be used to create a directory or perform some other
sanity check required for the provider to work. So far, haven't noticed
any performance impacts from this approach, besides not needing to auth
each time we make a request.
Should offer some speed boost, copying the file contents rather than
reading it all to memory then writing it back out. Should theoretically
be able to handle larger files as well, although some benchmarks
are TBD around filesize limits.
Reworked the fileprovider to proxy data directly from the provider
rather than attempt to funkily redirect when needed, since it was overly
complex and wouldn't work well in the long run. Temporarily added "file"
as constant return for determining the filetype through Backblaze.
Added a Makefile to make life easier as well, and rewrote the README to
reflect the refactor/rewrite and new approach to providers.
Expanding this as we go, currently have POC Backblaze B2 support and
basic 'disk' provider as well. Still WIP, but functional for the most
part. Also moving to simplified YAML configuration.
Overall, simplifying things to be extensible down the line. Still work
to be done, but coming along nicely.
