10 Awesome Google Colab Notebooks on Stable Diffusion
The recently released image-generation model by Stability.AI, StableDiffusion, is showing groundbreaking results and promising new innovations after its open-source public release. Researchers and developers have been experimenting to figure out all the possible fields where Stable Diffusion can be applied. We have compiled a list of a few of these contributions available on Google Colab as notebooks. Deforum Stable Diffusion For generating animations with simple prompts, Deforum created a notebook that allows users to input prompts along with the number of frames, frames per second, zoom, angle, and other such metrics. Deforum Stable Diffusion v0.5– Weighted Prompts– Perspective 2D Flipping– Dynamic Video Masking– Custom MATH expressions– Waifu and Robo Diffusion Modelshttps://t.co/mWNkzWtPsK— deforum (@deforum_art) October 1, 2022 Recently, the team released a newer version, v05, that includes new options like perspective 2D flipping, importing custom settings file, custom maths expressions, dynamic video masking, auto download of models, and weighted prompts. Check out Deforum here. Video Killed the Radio Star…Diffusion Owing to its interesting name, this notebook can make an animated music video for you, using a YouTube video. The model uses OpenAI’s Whisper speech-to-text to create Stable Diffusion animation by prompts from the lyrics of the video. Deforum Stable Diffusion v0.5– Weighted Prompts– Perspective 2D Flipping– Dynamic Video Masking– Custom MATH expressions– Waifu and Robo Diffusion Modelshttps://t.co/mWNkzWtPsK— deforum (@deforum_art) October 1, 2022 First, an image is generated using a text prompt from the lyrics. Then, subsequent images are generated by making variations in the first image. Thereafter, the images are set in a sequence, reordered and organised to create a smooth animation. Click here to check out the notebook. Doohickey Diffusion With CLIP guidance, perceptual guidance, and Perlin initial noise, this Colab notebook contains all tools required for Stable Diffusion. the newer version linked on my huggingface has clip guidance properly donehttps://t.co/uHXWieOr2I https://t.co/1QjwSha2QQ— 469 (@cephaloform) October 14, 2022 The model uses textual-inversion from HuggingFace hub and also allows loading Midjourney style images. Click here to visit the Colab notebook. Prompt Parrot This is designed to train language models on prompts, customised prompts, and render images. It is a fine tuned GPT-2 model. https://twitter.com/KyrickYoung/status/1563962142633648129 The model was recently updated to V2.5 to integrate with Stable Diffusion for image generation. It also comes with 185 built-in signature Kyrick prompts to run it straight out of the box, and a streamlined training UX. To visit the notebook of Prompt Parrot, click here. Stable Diffusion Interpolation Two different prompts on Stable Diffusion can now be interpolated seamlessly with this notebook. The combined images can also be generated into a video after the new V2.2 update. The new update also supports multiple seeds and fixes the blurring issue present in previous releases. https://twitter.com/ygantigravity/status/1562870954132672515 To start interpolating images with Stable Diffusion, click here. Stable-DreamFusion This notebook has combined Stable Diffusion and Google’s DreamFusion to create 3D objects from simple text prompts. A implementation of text-to-3D dreamfusion, powered by stable diffusiongithub: https://t.co/1GGjtphz8K pic.twitter.com/FaxIBL5WxE— AK (@_akhaliq) October 6, 2022 Google’s DreamFusion uses NeRF to create 3D models from 2D images generated by text-prompts on Stable Diffusion. Click here to check out the training and testing process. Stable Diffusion DreamBooth Inference This notebook elaborates on teaching Stable Diffusion new concepts via Google’s Dreambooth. Using a set of 3-5 images, developers can input and personalise models. A implementation of text-to-3D dreamfusion, powered by stable diffusiongithub: https://t.co/1GGjtphz8K pic.twitter.com/FaxIBL5WxE— AK (@_akhaliq) October 6, 2022 Unlike Textual Inversion used in Dreambooth on Stable Diffusion, this approach trains the whole model, yielding better results. Check out the Colab notebook here. Stable Craiyon Combining Stable Diffusion and CraiyonAI, this notebook can interpret and improve on the images generated using Craiyon to further improve the quality of the output. Just released a Colab notebook that combines Craiyon+Stable Diffusion , to get the best of both worlds. (Works in Colab free tier)GitHub Repo : https://t.co/RL5FuSlQ8cColab :https://t.co/KKCGHt9O0G(Top image is Craiyon, bottom is stable diffusion's reinterpretation) pic.twitter.com/3jeUHvmoLu— Geeve George (@GeeveGeorge) September 6, 2022 The creator of the notebook also dropped a tutorial on how to use the notebook. Click here for the Colab notebook and here for the GitHub repository. Seamless Texture Inpainting Using Stable Diffusion, MetaSemantic released an inpainting tool to generate seamless textures that do not look like they have been repeated at all. Seamless textures with #stablediffusion notebook released! Different than one yesterday that uses convolutions (this cuts the outside, pastes it on the inside, inpaints the outside, and glues them back making a 100% perfect seam)https://t.co/qJHMmx58sn pic.twitter.com/9PFrK0czZZ— Travis Hoppe (@metasemantic) September 11, 2022 Though this uses tiles to generate images, users have been trying to generate symmetrical and also abstract images with this method. Click here for the Colab notebook of this interesting computer graphics application. Stable Worlds Apart from generation of 3D worlds and 2D images, Stable Diffusion can also be used to create panoramic immersive worlds. Stable Worlds generates images using Stable Diffusion then stitches them to create seamless panoramas. https://twitter.com/NaxAlpha/status/1578685845099290624 Click here to check out the code.
List of Google Colab Notebooks for learning and using Stable Diffusion
List of Google Colab Notebooks for learning and using Stable Diffusion
["text to image models"]
["Go", "TPU", "OpenAI", "AI", "text to image models", "ML", "Git", "Colab", "GPT", "GitHub", "R"]
["AI", "ML", "OpenAI", "Colab", "TPU", "R", "Go", "Git", "GitHub", "GPT"]
Mohit writes about AI in simple, explainable, and often funny words. He's especially passionate about chatting with those building AI for Bharat, with the occasional detour into AGI.