Koel s3 uploader

Initially made for use with Yandex Cloud Object Storage and Koel. Tested on Yandex Cloud, Python 3.9 and Koel v5.1.8 with aws.php modified.

Pros

• Can automatically populate song tags based on filename (if tags are not present)
• Easy to extend to other s3 platforms

Cons

• Not tested with AWS currently
• Usage with non-AWS object storage requires modifying Koel's aws config (Koel v5.1.8)

Fast start

1. Download archive from GitHub as zip. Extract it and repack so only repo files exist in the archive (without parent folder) bash oneliner: d=$(unzip ./koel-s3-uploader-yandex.zip 2>&1 | grep creating | awk '{print $NF}') && zip -j s3.zip $d/*.py
2. Create private s3 bucket, lambda-like function, trigger and service user
   • Yandex
     1. Bucket
     2. Cloud function
     3. Trigger, User
   • AWS
     1. Official Koel docs for AWS
     2. Bucket
     3. Lambda
     4. Trigger
3. When creating cloud function, upload archive from step 1 into it.
4. Set up at least required environment variables in function settings. Checkout section below.
5. Select entrypoint for cloud function:
   • AWS: aws.handler
   • Yandex: yandex.handler
6. Modify aws.php (see section below)

Environment variable reference

Variable	Default or required	Example	Description
KOEL_HOST	Required	https://koel.dev	Public url of your Koel server.
KOEL_APP_KEY	Required	base64:abcdef=	Value from APP_KEY in .env
AWS_ACCESS_KEY_ID	Required	abc123abc...	Service User's secret key ID
AWS_SECRET_ACCESS_KEY	Required	AbC1a123...	Service User's secret key
LOGLEVEL	WARNING	DEBUG	Logging level upon function invocation
ASSUME_TAGS	False	True	Assuming: Try to create missing tags (see ref below)
ALBUMS_PATH	albums	artists	Assuming: Root folder of your discographies
ASSUME_ADD_ALBUM_YEAR	False	True	Assuming: If album folder contains year, add to album name
REMOVE_ALBUMARTIST_TAG	False	True	If True, removes albumartist from tags before upload (useless for now in Koel) Incompatible with ASSUME_COMPILATIONS
ASSUME_TAGS_FORCE	False	True	If True, assumed tags rewrites existing in a song
TELEGRAM_TOKEN	None	<token>	Enables processing error reporting to Telegram
TELEGRAM_CHAT	None	<chat_id>	Enables processing error reporting to Telegram
ASSUME_COMPILATIONS	False	True	Not supported by Koel: add tag with name of your compilation based on path
COMPILATIONS_PATH	compilations	my-awseme-lib	Not supported by Koel: root path of your compilations
ASSUME_COMPILATIONS_TAG	albumartist	compilation	Not supported by Koel: tag to assign compilation name to

Important: usage of non-AWS s3

As for Koel v5.1.8, to use Koel with non-AWS Object Storage, you have to modify config/aws.php in the project. Add 'endpoint' => env('AWS_ENDPOINT', 'https://s3.amazonaws.com'), to it and set up AWS_ENDPOINT at .env of your Koel installation. Info about entrypoint should be provided by your cloud provider. You might be required to use docker bind mount for this file.

Advanced features

Assuming

Assuming is pupulation of missing tags in records. It never rewrites any existing tags, but trying to assume if they miss. Currently supported tags are Title, Artist, Album, Track. To enable this feature, set ASSUME_TAGS environment variable to True.

This is how it works: create folder for your discographies (default is albums), and put everything into structure: albums/Artist/[anything]/Album/song.mp3

Examples:

Path	additional environment	tags
albums/Dope/Group Therapy/01. Falling Away.mp3		Title: Falling Away, Album: Group Therapy, Artist: Dope, Track: 01
albums/Dope/Group Therapy/01 - Falling Away.mp3		Title: Falling Away, Album: Group Therapy, Artist: Dope, Track: 01
my/Dope/albums/Group Therapy/Falling Away.mp3	ALBUMS_PATH=my	Title: Falling Away, Album: Group Therapy, Artist: Dope
albums/Dope/albums/2003 - Group Therapy/Falling Away.mp3		Title: Falling Away, Album: Group Therapy, Artist: Dope
albums/Dope/albums/2003 - Group Therapy/Falling Away.flac	ASSUME_ADD_ALBUM_YEAR=True	Title: Falling Away, Album: 2003 - Group Therapy, Artist: Dope
Dope/albums/2003 - Group Therapy/03. Falling Away.mp3		Title: Falling Away, Artist: No Artist
compilations/Rock/Dope - Falling Away.mp3		Title: Falling Away, Artist: Dope

However, this behaviour might be buggy and turned off by default.

Telegram notifications

You can send notifications about failed uploads to Telegram. To use it, first uncomment python-telegram-bot in requirements.txt before upload, create Telegram bot via BotFather and assign TELEGRAM_CHAT and TELEGRAM_TOKEN environment variables in lambda.

Sync

Call <provider>.sync(bucket, path_in_bucket) instead of handler in lambda or run locally. Be aware that syncing downloads songs, for huge music library syncing from local machine might be expensive. Consider it as not regular task. Example:

AWS_ACCESS_KEY_ID=<id> AWS_SECRET_ACCESS_KEY=<key> KOEL_HOST=https://koel.dev KOEL_APP_KEY=312 LOGLEVEL=INFO python3 -c 'import yandex, asyncio; asyncio.run(yandex.sync("my-music-lib", "compilations/test"))'

Development

Potentially this app can work with any s3 provider which supports s3 triggers and lambda-like functions. But we need to determine event structure passed by trigger. To understand it, you can call main.event function from inside lambda, it will dump raw event.

Then add file with clear platform definition, for example oracle.py and create function handler inside. Put the code needed for parsing and setting required data (object path, bucket name and s3 entrypoint) into handler, construct object of class main.S3Song and call await main.handler(song) at the end. Simple. For example see yandex.py.

PRs are welcome.