Artistic LoRAs.
My one most beloved
and interesting way
to create incredible datasets.
- Project: Soulstealer
I believe that all Art should be unique, in one way or another. Art is a fleeting inspiration, a certain random set of events spread in time that would inevitably lead to an interesting finale - and I wished to capture exactly that moment!
So I created an Low-Ranking Adaptation Art Randomizer, or, Random/Artistic LoRAs, winch I use in my every generation in some way, shape or form. Let's learn how to make and use them!
FAIR WARNING: This is NOT how you must create your average normal LoRAs! I intentionally side-stepped a lot mechanics to increase artistic value! If you wish to learn how to make good LoRAs - seek other resources!
Preparation
You need:
* Installed Python 3.10 interpreter.
* Copy of a Stable Diffusion Trainer
(you can take one here: https://github.com/Linaqruf/kohya-trainer)
In this example I will install trainer in C:\stable-diffusion-train path
* Supported videocard (I use RTX3090, but you may use anything, as long as it does have 8 GB of VRAM or more).
Replication
Step 1:
Download a python script that I wrote myself, for creating Artistic Dataset.
Be advised - my script uses 'requests' and 'rich' python modules, so you may want to install them beforehand using command like "pip install requests rich".
The script for creating Artistic Datasets is available here: https://pastebin.com/ZDW2h7Pn .
Do not forget to name it main.py
Step 2:
Run Artistic Dataset Generator script using command
python3 main.py --tag 'score:>100' --count 1000
This action will do one of the following actions:
* will download you either every single image from e621 imageboard, if there is less than a 1000 that falls under your tagging...
* or will download exactly 1000 of tag-specifical random pictures.
Whatever it downloaded, is your Random Artistic Dataset. It contains a vague mix of everything you can get from a publicly available resources.
You may also use general tags like "canine" if you wish to train for a specific creature type, and/or use negative prompts such as -poop to get dataset more refined.
Step 3:
Wait for script to finish it's job.
Rename output folder (that should appear near script file) to "Dataset" and copy it somewhere on your hard drive.
Remember the path.
For this particular example it will be "C:\Dataset".
Step 4:
Manually shuffle through all images in dataset.
Possibly, remove every single one that not ends with .png/.jpg
Also, remove .txt files. They do contain tags that you might not want to create your Art.
Step 5:
Download two more configuration files:
- a configuration file - https://pastebin.com/yAy6dgJT
- and a dataset descriptor - https://pastebin.com/L7fg9UFb
Name them config_file.toml and dataset_config.toml accordingly.
Place them into stable-diffusion-train folder.
Step 6:
If nessesary, manually edit both configuration files from Step 5, replacing paths to your models, vaes, and other things nessesary.
Step 7:
Finally, use command
accelerate launch --num_cpu_threads_per_process 1 train_db.py --dataset_config dataset_config.toml --config_file config_file.toml
to start the process of creating your own Artistic LoRA.
It may take a lot of time, especially at slower GPUs. Have patience.
Final:
After trainer finished it's work, there will be a lot of LoRA files, sorted by epochs. Try any of them to find what is best suited for you. Name favorite ones however you wish.
To actually use them, set token 'sks' anywhere you want in your prompt, following standart logic - the earlier sks appears in prompt, the stronger effects is. You may use it both as 'sks' creature or 'sks style' for an interesting and sometimes inspiringly unpredictable effects.
PS: If you, like me, are fan of Tibet Monks and their perception of Art (https://www.youtube.com/watch?v=bgoHUH-_yWo), then after you finish your masterpiece, remove any traces of your dataset and LoRA. The Art is fleeting.
