From 636ab32eadeea71c588d8ef5de265fa7bf306a0a Mon Sep 17 00:00:00 2001 From: Gabriel Simmer Date: Sat, 29 May 2021 22:32:38 +0100 Subject: [PATCH] Tidy README, remove UPX requirement. --- .circleci/config.yml | 4 +- Makefile | 2 - README.md | 105 +++++++++++++++------------ assets/config_examples/providers.yml | 13 ++++ 4 files changed, 74 insertions(+), 50 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 8621a8f..23d11a3 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -1,6 +1,5 @@ version: 2.1 -orbs: - upx: circleci/upx@1.0.1 + jobs: build: docker: @@ -12,7 +11,6 @@ jobs: - go-mod-{{ checksum "go.sum" }}-v3 - go-mod-{{ checksum "go.sum" }} - go-mod - - upx/install - run: command: make dist - store_artifacts: diff --git a/Makefile b/Makefile index 4e24548..6071f6c 100644 --- a/Makefile +++ b/Makefile @@ -14,11 +14,9 @@ pi: make_build_dir small: make_build_dir go build -o build/bin/sliproad -ldflags="-s -w" - upx --brute build/bin/sliproad -9 --no-progress small_pi: make_build_dir env GOOS=linux GOARCH=arm GOARM=5 go build -o build/bin/sliproad-arm -ldflags="-s -w" - upx --brute build/bin/sliproad-arm -9 --no-progress run: go run webserver.go diff --git a/README.md b/README.md index b306d82..cf085d7 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,24 @@ -# sliproad -merging filesystems together +# Sliproad -## about +Merging filesystems together -this project aims to be an easy-to-user web API that allows the management of cloud storage, whether it be on -the host machine or part of a remote api. this is intended mostly for my own use, but i am documenting it in a way that -i hope allows others to pick it up and improve on it down the line. +## About -if something is unclear, feel free to open an issue :) +This project aims to be an easy-to-use web API and frontend that allows the +management of cloud storage, whether it be on the host machine or part of a +remote API, alongside local filesystems. While this is intended mostly for my +own use, I am documenting it in a way that I hope allows others to use it! -## configuration +## Configuration -unlike the initial version of this project, the current build uses _providers_ to determine how to handle various -functions related to files. currently, two are implemented, `disk` and `backblaze`, since they are the primary providers -i use myself. the providers you would like to use can be added to `providers.yml` alongside the binary. +Sliproad uses "Providers" to support various filesystems "types", whether it be +remote or local. Currently, three exist - `disk` for filesystems local to the +machine, `backblaze` to leverage Backblaze B2 file storage and `s3` for AWS S3 +(and other compatible providers). -for example, here is a sample configuration implementing both of them: +An example of leveraging all three, in various forms, can be found below. As +more are added, this example will be updated, and more examples can be found in +the `assets/config_examples` directory. ```yaml disk: @@ -24,53 +27,65 @@ disk: backblaze: provider: backblaze config: - appKeyId: APP_KEY_ID - appId: APP_ID - bucket: BUCKET_ID + bucket: some-bucket + applicationKeyId: application-key-id + applicationKey: application-key +s3: + provider: s3 + config: + region: eu-west-2 + bucket: some-bucket +# An example of an S3 compatible API, doesn't have to be Backblaze. +backblazes3: + provider: s3 + config: + bucket: some-bucket + region: us-west-000 + endpoint: s3.us-west-000.backblazeb2.com + keyid: key-id + keysecret: key-secret ``` -(read more here: [#providers](#providers)) +## Running -## running +After configuring the providers you would like to utilize, simply run +`./sliproad`. This will spin up the webserver at `127.0.0.1:3000`, listening on +all addresses. -after adding the providers you would like to use, the application can be run simply with `./nas`. it will attach to port -`:3000`. +## Building -## building - -this project uses go modules and a makefile, so building should be relatively straightforward. +This project leverages a Makefile to macro common commands for running, testing +and building this project. - `make` will build the project for your system's architecture. - `make run` will run the project with `go run` - - `make pi` will build the project with the `GOOS=linux GOARCH=arm GOARM=5 go` flags set for raspberry pis. - -## providers + - `make pi` will build the project with the `GOOS=linux GOARCH=arm GOARM=5 go` flags set for Raspberry Pi. + - `make dist` will build and package the binaries for distribution. -"providers" provide a handful of functions to interact nicely with a filesystem, whether it be on local disk or on a -remote server via an api. best-effort is done to keep these performant, up to date and minimal. +### Adding Providers -there are a few built-in providers, and more can be added by opening a pull request. +New file providers can be implemented by building off the +`FileProviderInterface` struct, as the existing providers demonstrate. You can +then instruct the [`TranslateProvider()`](https://github.com/gmemstr/sliproad/blob/master/files/fileutils.go#L8-L21) +that it exists and how to configure it. -|name|service|configuration example| -|----|-------|---------------------| -|disk|local filesystem|assets/config_examples/disk.yml| -|backblaze|backblaze b2|assets/config_examples/backblaze.yml| +## Authentication [!] -you can find a full configuration file under `assets/config_examples/providers.yml` +Authentication is a bit tricky and due to be reworked in the next iteration of +this project. Currently, support for [Keycloak](https://www.keycloak.org/) is +implemented, if a bit naively. You can turn this authentication requirement on +by adding `auth.yml` alongside your `providers.yml` file with the following: -### custom provider +```yaml +provider_url: "https://url-of-keycloak" +realm: "keycloak-realm" +redirect_base_url: "https://location-of-sliproad" +``` -custom file providers can be implemented by adding a new go file to the `files` module. it should -implement the `FileProviderInterface` interface. +Keycloak support is not currently actively supported, and is due to be removed +in the next major release of Sliproad. That said, if you encounter any major +bugs utilizing it before this, _please_ open an issue so I can dig in further. -## authentication - -basic authentication support utilizing [keycloak](https://keycloak.org/) has been implemented, but work -is being done to bring this more inline with the storage provider implementation. see `assets/config_examples/auth.yml` -for an example configuration - having this file alongside the binary will activate authentication on all -`/api/files` endpoints. note that this implementation is a work in progress, and a seperate branch will -contain further improvements. - -## icons +## Credits SVG Icons provided by Paweł Kuna: https://github.com/tabler/tabler-icons (see assets/web/icons/README.md) \ No newline at end of file diff --git a/assets/config_examples/providers.yml b/assets/config_examples/providers.yml index 5c19b86..780b7c3 100644 --- a/assets/config_examples/providers.yml +++ b/assets/config_examples/providers.yml @@ -18,3 +18,16 @@ backblaze: applicationKeyId: aaaaaaaaaaaa applicationKey: aaaaaaaaaaaa bucket: aaaaaaaaaaaa +s3: + provider: s3 + config: + region: eu-west-2 + bucket: some-bucket +backblazes3: + provider: s3 + config: + bucket: sliproad-testing + region: us-west-000 + endpoint: s3.us-west-000.backblazeb2.com + keyid: key-id + keysecret: key-secret \ No newline at end of file