kohya-ss enhanced Stable Diffusion Trainer
Collection of PRs for kohya-ss
Track a separate validation dataset for loss to compare with. Helps detect overfitting of the training dataset.
PR: Add validation loss
Remove certain keys from being trained in the LoRA.
PR: Add drop_keys to drop certain keys from the LoRA network
Mask the loss to reduce training of the background.
PR: add masked loss implementation
Log and record momentum changes. In concert with CyclicLR and OneCycleLR PyTorch learning rates which modify the momentum.
PR: Momentum betas logs