kohya-ss enhanced Stable Diffusion Trainer

Collection of PRs for kohya-ss


Validation loss

Track a separate validation dataset for loss to compare with. Helps detect overfitting of the training dataset.

PR: Add validation loss

Drop keys

Remove certain keys from being trained in the LoRA.

PR: Add drop_keys to drop certain keys from the LoRA network

Masked loss

Mask the loss to reduce training of the background.

PR: add masked loss implementation


Log and record momentum changes. In concert with CyclicLR and OneCycleLR PyTorch learning rates which modify the momentum.

PR: Momentum betas logs