256x512 resolution anime generation

[science-knowledge]

I think it would be really useful if there was an option to make 256x512 resolution anime artworks, as they can be used in card games, visual novels, RPGs, and many other gaming and artistic mediums, removing a lot of the existing barriers for creativity in those fields.

Thanks again for making the future possible! This is one of the AI I'm most excited for its potential.

[nolan-dev]

Sorry for the delay. The unfortunate fact of the matter is that the 256x512 model produces NSFW images significantly more often than not (and occasional nudity), which is why I originally stuck to the cropped version. This is partly due to the nature of the dataset, and partly due to variance reduction strategies I used to filter data (turns out clothes are high variance). In the future, I might try to train one that handles variance better (or maybe someone else has trained one that can be converted), but retraining is expensive. You can try using the 'cloth' modifier to add more clothing, which isn't ideal but it's what I did for the images in the blog.
A few things to note:

• The tool automatically writes images do the 'portraits' directory, which may show up in your 'recent files' list, so if your machine is accessed by children or anyone who might be made uncomfortable by the image you should avoid using the 256x512 model unless you're careful to delete them or remove them from the recent files list.
• The 256x512 model should be able to load images from the default 'anime_faces.pb' model as they're the same model, just cropped.

To use the model, download the zip from here:

1. https://drive.google.com/file/d/1ibGbf46D8lHYHc4aUbeWcDMgX3IY8VLC/view?usp=sharing
2. Unzip it, and move the files (except readme.txt) to the directory for the v0.2.0 version of the tool
3. Run the tool and select the 'anime_512x256_WARNING_NUDITY_AND_ADULT_IMAGES.pb' model

Let me know if you have any issues.

[science-knowledge]

Great work, really useful tool.

I just have a minor problem that I can't seem to import(Load) custom PNG files even if they are the same resolution as the samples(256x512). I'm not very familiar with AI so I don't know if it's a limitation or if I'm just missing something.

Actually, if I open any of the samples, change even 1 pixel with editing software and save, they become unloadable for some reason: error.

I was hoping to maybe edit them a bit before animating them with spatial selections, but even if it isn't possible the job can still be done so that is just a minor problem really.

Thanks for taking the time to develop and share the 256x512 model, it will help a lot with game art!

[nolan-dev]

Unfortunately, it is a limitation with this type of model (GANs): it just converts from a latent vector to an image and not the other way around. There's no (simple) way to convert from an arbitrary image to a latent vector that the model can then load. What the tool actually does is append the latent code that was used to generate an image to the PNG for that image, so when you 'load' an image it doesn't actually read the pixels, just the latent code that was appended. Any PNG that wasn't generated by the tool can't be loaded, and if you edit a PNG and then save it, the software probably discards the appended latent vector (but even if it didn't, loading the image would not reflect the edits). Pretty much the tool uses a hack to make it look like you can load arbitrary images, but unfortunately it can't (I should have made that more clear in the error message)

There are other generative models that allow conversions from image to latent (https://openai.com/blog/glow/, VQ-VAE as well I believe), and while I believe they aren't quite as good as GANs yet given the same compute, I have high hopes that the kind of functionality you want will be simple in the future.

[mochaii]

Hi, thank you so much for your response to this issue, @nolan-dev ! I've been following it for a while and I'm very curious if there's a way to change the size of the pb on our own? Or if there's a way to get an even larger model? I agree with the original issue that it would help a lot with game art.

[nolan-dev]

Hi Mochaii, the original faces model could technically have been hacked to enlarge it to the 256x512 model as 256x512 is the original resolution the model was trained at (though now you can just download it from the link above). Unfortunately, it can't be increased beyond that without retraining. This can take several weeks with expensive GPUs, and increasing the resolution across the whole training set might reduce the number of available training samples. There is a good chance the model could be made quite a bit better/bigger though, by using stylegan2 instead of the original stylegan and the danbooru 2019 dataset (https://www.gwern.net/Danbooru2019) rather than 2018. Stylegan2 would need some modification to support rectangular images though. I've been pretty busy but I might try that if no one else gets around to it.

[mochaii]

Thank you so much for your reply! That makes sense, I'm new to learning about GAN's and training models, so you're helping me a lot with understanding. I'm excited to see if you try it in the future! It's alright if you're busy though, you've done so much already and I'm sure so many people like me appreciate all you've done on this project.

[science-knowledge]

I just wanted to post here to show how incredibly useful I'm finding this amazing tool.

I made a game prototype with 1000 playable characters using the batch generation(censored to make it SFW).

With some minor tweaking, I had a playable VN prototype with high quality "breathing" characters.

• I picked 1000 images from a domain of 3000 images batch generated with standard program parameters.

• Characters shown in the second link are completely random ones, there are much higher quality characters I managed to get.

I think this model is the most advanced and underrated GAN model for game character generation, and the batch generation is revolutionary ^^, perhaps someone will make a VN with 100% procedurally generated characters using this tool in the future, and combined with GPT-3, a 100% AI-generated game. What a time to be alive!

I will include here the full prototype video that contains nudity from the generated pics to better show my results for what I believe could become a procedural system of playable characters with some more tweaking, keep in mind NSFW: WARNING_NUDITY_AND_ADULT_IMAGES