>#stablediffusion

I am sorry for editing this video and trimming a large portion of it, Please check the updated video in https://www.youtube.com/watch?v=vA2v2IugK6w which is better anyway.

#A1111 #AI #Lora #koyass #sd
#character #charactertraining

This video shows and presents the steps needed for a Perfect LoRA Model of a character that is flexible, able to adapt to new settings, and works for face and full body shots without the need for inpaint in stable diffusion.

steps and key notes used and some of the resources used in the process.

LoRA Training Guide: Training Steps
After installation of Koyass from https://github.com/bmaltais/kohya_ss
Prepare your data set (Good Data is essential)
Caption it
Set up folders
Set up training parameters
Train for few epochs often 5 – 10 epochs could be enough (or more)
select best epoch after comparison.
Test for each epoch: new set of clothes, different colors, different hair styles, to make sure your model is flexible and not overfitting
using lower Dimension such as 64 and lower alpha may make results even better, but takes more epochs to train … for smaller data sets always use 64 dimension and less and 8-32 alpha only, dont use 128.
Iterate, … (you can also get good results from previous model to augment next model dataset in some cases or change its features, such as turn a fat person to slim, or vice versa, but similiarity to origin drops slightly)
Number of Regularization images = Kohya ss will only use number of images *repeats number of reg images, all remaining images are ignored… for example if you have 40 repeats for each image, and you have 20 images only then Kohya will only use 40*20=800 images… all remaining images are ignored.

other useful info about how stable diffusion works in general and some tips can be seen at
https://youtu.be/1MkZB2YEiy4
Beginners guide to stable diffusion in Automatic1111 at:
https://youtu.be/RtjDswbSEEY

Note: LyCORIS may produce even better results too than standard LoRA, Dreambooth will produce best results, but affects the entire SD model which makes it a good option in some cases, and LoRA good in others.

Note: only (number of images * repeats are used from class images, extra images are ignored) about regularization, it is possible to train a subject without regularization, but it will increase risk of overfitting and artifacts, training becomes faster though, half the time, and converges faster…. also for single subject training, it is also possible not to include captions at all, but makes the more prompting slightly less effective and might get some unexpected results in some generations, but the output could actually be good in some cases, but mostly captioning is better!!!! so SD will only use the Class name (woman) and the Instance name (oliviacasta) for instance.

Note: it is possible to include cropped body images and train portraits with cropped body images, such as an upper body without head, or lower body only, SD may (and may not) stich them together later on, instead of diffusing your images with other images which can reduce resemblance if the other image has faces too for example. SD will seek features from your LoRA first based on your prompt but may also use features from the SD , thus images trained in your data will have higher weight…the number of bad images generated will be high, but you may be able to pick up some good samples from them without inpaint.

Koyass for lora training https://github.com/bmaltais/kohya_ss
Down pictures from Instagram https://chrome.google.com/webstore/detail/download-albums-for-insta/inkncgklbglecgdlcpfpajejocdpbpbd/
https://apps.microsoft.com/store/detail/microsoft-powertoys/XP89DCGQ3K6VLD Microsoft power toys for resizing images locally in blucks
https://www.birme.net/ Birme for bulk image resize online (can turn off network and resize too using the website because it runs locally)

sample regularization set that you can download… you can remove some of the images from it, add more body shots to it, adapt it to your needs, reg set ideally should resemble the training subject in terms of general features but other people, can include real people too, and images from other checkpoints
https://huggingface.co/datasets/AIHowto/Chilloutmix_woman_regset1/blob/main/1340chilloutmix_class_woman.zip
another more comprehensive set with full body, faces, upper body that maybe used in different checkpoitns
https://huggingface.co/datasets/AIHowto/woman_class_images_better_1832/blob/main/reg_images_general_fullbody_portraits_upperbody1832_images.zip

Regularization set: it is recommended if generated by SD in the same checkpoint of training and be of good quality or mix with real high quality images … filtering the data set is not necessary but can be done such as removing pictures with deformations, it may improve the results… using data from other Checkpoints is also acceptable … even using real pictures is good too