v21.0.1

• 2023/03/01 (v21.0.1):
  • Add warning to tensorboard start if the log information is missing
  • Fix issue with 8bitadam on older config file load
• 2023/02/27 (v21.0.0):
  • Add tensorboard start and stop support to the GUI

v20.8.2

• 2023/02/26 (v20.8.2):
  • Fix issue #231
  • Change default for seed to random
  • Add support for --share argument to kohya_gui.py and gui.ps1
  • Implement 8bit adam login to help with the legacy Use 8bit adam checkbox that is now superceided by the Optimizer dropdown selection. This field will be eventually removed. Kept for now for backward compatibility.

v20.8.1

• 2023/02/23 (v20.8.1):
  • Fix instability training issue in train_network.py.
    • fp16 training is probably not affected by this issue.
    • Training with float for SD2.x models will work now. Also training with bf16 might be improved.
    • This issue seems to have occurred in PR#190.
  • Add some metadata to LoRA model. Thanks to space-nuko!
  • Raise an error if optimizer options conflict (e.g. --optimizer_type and --use_8bit_adam.)
  • Support ControlNet in gen_img_diffusers.py (no documentation yet.)

v20.8.0

• 2023/02/22 (v20.8.0):
  • Add gui support for optimizers: AdamW, AdamW8bit, Lion, SGDNesterov, SGDNesterov8bit, DAdaptation, AdaFactor
  • Add gui support for --noise_offset
  • Refactor optmizer options. Thanks to mgz-dev!
    • Add --optimizer_type option for each training script. Please see help. Japanese documentation is here.
    • --use_8bit_adam and --use_lion_optimizer options also work and will override the options above for backward compatibility.
  • Add SGDNesterov and its 8bit.
  • Add D-Adaptation optimizer. Thanks to BootsofLagrangian and all!
    • Please install D-Adaptation optimizer with pip install dadaptation (it is not in requirements.txt currently.)
    • Please see kohya-ss/sd-scripts#181 for details.
  • Add AdaFactor optimizer. Thanks to Toshiaki!
  • Extra lr scheduler settings (num_cycles etc.) are working in training scripts other than train_network.py.
  • Add --max_grad_norm option for each training script for gradient clipping. 0.0 disables clipping.
  • Symbolic link can be loaded in each training script. Thanks to TkskKurumi!

v20.7.4

• 2023/02/19 (v20.7.4):
  • Add --use_lion_optimizer to each training script to use Lion optimizer.
    • Please install Lion optimizer with pip install lion-pytorch (it is not in requirements.txt currently.)
  • Add --lowram option to train_network.py. Load models to VRAM instead of VRAM (for machines which have bigger VRAM than RAM such as Colab and Kaggle). Thanks to Isotr0py!
    • Default behavior (without lowram) has reverted to the same as before 14 Feb.
  • Fixed git commit hash to be set correctly regardless of the working directory. Thanks to vladmandic!
• 2023/02/15 (v20.7.3):
  • Update upgrade.ps1 script
  • Integrate new kohya sd-script
  • Noise offset is recorded to the metadata. Thanks to space-nuko!
  • Show the moving average loss to prevent loss jumping in train_network.py and train_db.py. Thanks to shirayu!
  • Add support with multi-gpu trainining for train_network.py. Thanks to Isotr0py!
  • Add --verbose option for resize_lora.py. For details, see this PR. Thanks to mgz-dev!
  • Git commit hash is added to the metadata for LoRA. Thanks to space-nuko!
  • Add --noise_offset option for each training scripts.
    • Implementation of https://www.crosslabs.org//blog/diffusion-with-offset-noise
    • This option may improve ability to generate darker/lighter images. May work with LoRA.

v20.7.3

• 2023/02/16 (v20.7.3)
  • Noise offset is recorded to the metadata. Thanks to space-nuko!
  • Show the moving average loss to prevent loss jumping in train_network.py and train_db.py. Thanks to shirayu!

v20.7.2

• 2023/02/11 (v20.7.2):
  • lora_interrogator.py is added in networks folder. See python networks\lora_interrogator.py -h for usage.
    • For LoRAs where the activation word is unknown, this script compares the output of Text Encoder after applying LoRA to that of unapplied to find out which token is affected by LoRA. Hopefully you can figure out the activation word. LoRA trained with captions does not seem to be able to interrogate.
    • Batch size can be large (like 64 or 128).
  • train_textual_inversion.py now supports multiple init words.
  • Following feature is reverted to be the same as before. Sorry for confusion:

    Now the number of data in each batch is limited to the number of actual images (not duplicated). Because a certain bucket may contain smaller number of actual images, so the batch may contain same (duplicated) images.

  • Add new tool to sort, group and average crop image in a dataset

v20.7.1

2023/02/09 (v20.7.1)
- Caption dropout is supported in train_db.py, fine_tune.py and train_network.py. Thanks to forestsource!
- --caption_dropout_rate option specifies the dropout rate for captions (01.0, 0.1 means 10% chance for dropout). If dropout occurs, the image is trained with the empty caption. Default is 0 (no dropout).
- --caption_dropout_every_n_epochs option specifies how many epochs to drop captions. If 3 is specified, in epoch 3, 6, 9 ..., images are trained with all captions empty. Default is None (no dropout).
- --caption_tag_dropout_rate option specified the dropout rate for tags (comma separated tokens) (01.0, 0.1 means 10% chance for dropout). If dropout occurs, the tag is removed from the caption. If --keep_tokens option is set, these tokens (tags) are not dropped. Default is 0 (no droupout).
- The bulk image downsampling script is added. Documentation is here (in Jpanaese). Thanks to bmaltais!
- Typo check is added. Thanks to shirayu!
- Add option to autolaunch the GUI in a browser and set the server_port. USe either gui.ps1 --inbrowser --server_port 3456or gui.cmd -inbrowser -server_port 3456

v20.7.0

• 2023/02/06 (v20.7.0)
  • --bucket_reso_steps and --bucket_no_upscale options are added to training scripts (fine tuning, DreamBooth, LoRA and Textual Inversion) and prepare_buckets_latents.py.
  • --bucket_reso_steps takes the steps for buckets in aspect ratio bucketing. Default is 64, same as before.
    • Any value greater than or equal to 1 can be specified; 64 is highly recommended and a value divisible by 8 is recommended.
    • If less than 64 is specified, padding will occur within U-Net. The result is unknown.
    • If you specify a value that is not divisible by 8, it will be truncated to divisible by 8 inside VAE, because the size of the latent is 1/8 of the image size.
  • If --bucket_no_upscale option is specified, images smaller than the bucket size will be processed without upscaling.
    • Internally, a bucket smaller than the image size is created (for example, if the image is 300x300 and bucket_reso_steps=64, the bucket is 256x256). The image will be trimmed.
    • Implementation of #130.
    • Images with an area larger than the maximum size specified by --resolution are downsampled to the max bucket size.
  • Now the number of data in each batch is limited to the number of actual images (not duplicated). Because a certain bucket may contain smaller number of actual images, so the batch may contain same (duplicated) images.
  • --random_crop now also works with buckets enabled.
    • Instead of always cropping the center of the image, the image is shifted left, right, up, and down to be used as the training data. This is expected to train to the edges of the image.
    • Implementation of discussion #34.

v20.6.1

What's Changed

• Add support for LoRA resizing by @bmaltais in #107

Full Changelog: v20.6.0...v20.6.1