Also ultimate sd upscale is also a node if you dont have enough vram it tiles the image so that you dont run out of memory. Essentially, each conditioning was being rendered at the full canvas size, then the mask was just grabbing whatever latent noise existed within its bounds, rather than generating a latent of the size/dimensions of the mask. I want to replicate the "upscale" feature inside "extras" in A1111, where you can select a model and the final size of the image. This is because latent upscalers are kinda destructive in how they upscale images. Tried it, it is pretty low quality and you cannot really diverge from CFG1 (so, no negative prompt) otherwise the picture gets baked instantly, cannot either go higher than 512 up to 768 resolution (which is quite lower than 1024 + upscale), and when you ask for slightly less rough output (4steps) as in the paper's comparison, its gets slower. The latent representation of an image to be upscaled. LATENT. The workflow is kept very simple for this test; Load image Upscale Save image. To move multiple nodes at once, select them and hold down SHIFT before moving. Hello, For more consistent faces i sample an image using the ipadapter node (so that the sampled image has a similar face), then i latent upscale the image and use the reactor node to map the same face used in the ipadapter on the latent upscaled image. Upscale image using model to a certain size. Please share your tips, tricks, and workflows for using this… The hires script is overriding the ksamplers denoise so your actually using . Basic latent upscale, basic upscaling via model in pixel space, with tile controlnet, with sd ultimate upscale, with LDSR, with SUPIR and whatnot. 50) each time. （ Image processing example ）. this breaks the composition a little bit, because the Because latent upscale often causes significant image degradation, and latent downscale results in irreparable image damage, pixelspace must be retained for detailing regardless. (I didn't use AI, latent noise, and a prompt to generate it) - What nodes/workflow would you guys use to get the best results? As my test bed, i'll A few examples of my ComfyUI workflow to make very detailed 2K images of real people (cosplayers in my case) using LoRAs and with fast renders (10 minutes on a laptop RTX3060) LATENT Tricks - Amazing ways to use ComfyUI. I saw in some post that some people do it iterativelly or mixing many samplers but I don't understand much how to do that. For 'photorealistic' videos with lots of fine details it doesnt seem a great approach, the final Oh and if you upscale and you are using multi area conditioning, make sure you scale the conditioning along with it for better results. Even more crucially, since narrow ranges of faces cannot be represented by the latent space, using a detailer is essential. But I would like a bit more refinement (Hands still don't look even remotely good most of the time if they're further from the camera Welcome to the unofficial ComfyUI subreddit. Launch ComfyUI by running python main. That's because latent upscale turns the base image into noise (blur). It's supposed to be much better and faster than the default latent upscaling method. ERSGAN models like 4X-Animesharp also utilize latent space but they are generative adversarial networks (ERS GAN) rather than diffusion models. Settled on 2/5, or 12 steps of upscaling. 5+ denoise. As such ksampler 1 will stop denoising at step 12. 5x the latent would really start to shine more, which is not possible with the pixel upscale. The only approach I've seen so far is using a the Hires fix node, where its latent input comes from AI upscale > downscale image, nodes. COMBO[STRING] Specifies the method used for upscaling the latent image. 5 CFG, and 1. Final 1/5 are done in refiner. Reply reply More replies Input your batched latent and vae. eg: batch index 2, Length 2 would send image number 3 and 4 to preview img in this example. The image in the left (directly after generation) is blurry and lost some tiny details; the image on the right (after mask-compose node) retains the sharpness, but you can see clearly the bad composition line, with sharp transition. I think I have a reasonable workflow, that allows you to test your prompts and settings and then "flip a switch", put in the image numbers you want to upscale Bella donna Italiana - 8K image - ComfyUI + DreamshaperXL + TiledDiffusion + Kohya deep shrink - latent upscale + clipvision and my poor 4060ti If you want to use low denoising strength, you have to decode the latent, upscale the pixel space image, then encode the image back to latent space first. height. Before / after. ComfyUI workflow. Both these are of similar speed. samples. 2 / 4. But these things can be used with any upscale workflow. I'll still use them when a latent upscale just isn't working right for whatever reason. Latent of batch size 2 of 512x768 is 229 kb. Hello, I did some testing of KSampler schedulers used during an upscale pass in ComfyUI. - lots of pieces to combine with other workflows: 6. Ignore the LoRA node that makes the result look EXACTLY A little about my step math: Total steps need to be divisible by 5. Please keep posted images SFW. Hires fix 2x (two pass img) THE LAB EVOLVED is an intuitive, ALL-IN-ONE workflow. Which one is better will depend on ComfyUI, 1024x1024 latent to start. OnlyEconomist4. So latent upscaling gives really nice results but it is really slow on my 2060 super. This is done after the refined image is upscaled and encoded into a latent. For the best results diffuse again with a low denoise tiled or via ultimateupscale (without scaling!). So from VAE Decode you need a "Uplscale Image (using model)" under loaders. 0 upscale usually requires 0. It allows for the adjustment of the output image's dimensions and the method of upscaling, providing flexibility in enhancing the resolution of latent images. This is the image I created using ComfyUI, utilizing Dream ShaperXL 1. 5, which I always need to improve any image, especially while applying LoRAs. Upscale (I go for 1848x1848 since this somehow results from a 1. 5 denoising strength. After borrowing many ideas, and learning ComfyUI. 2x, upscale using a 4x model (e. to the corresponding Comfy folders, as discussed in ComfyUI manual installation. It is a 4GB upscaling model that works with all models and operates in the latent space before the vae so it's super fast with unmatched quality. ago. ) These images are zoomed-in views that I created to examine the details of the upscaling process, showing how much detail For the latent upscale, I use two nodes : Upscale Latent by, then a KSampler. If you do all in latent: Generate image -> upscale latent -> hires. An IPAdapter takes in the first image to condition a model that is fed into the mixing KSampler, guiding the mixing even more along the semantic lines of the first image. 0. Your main issue is the model: don't use base SD 1. Txt-to-img, img-to-img, Inpainting, Outpainting, Image Upcale, Latent Upscale, multiple characters at once, LoRAs, ControlNet, IP-Adapter, but also video generation, pixelization, 360 image generation, and even Live painting! 6- Latent upscaler, so, latent upscalers are useful because SD, LOVES the noise they make for adding detail to things, how the detail adding is done depends on which upscale method gets used. Reply. You could try to pp your denoise at the start of an iterative upscale at say . Welcome to the unofficial ComfyUI subreddit. The node can be found in "Add Node -> latent -> NNLatentUpscale". ultrasharp), then downscale. In this example you have 20 steps, so it will calculate the amount to de-noise in each step. I’ve also used pose and depth controlnets. Especially if the latent has leftover noise or is in a mid schedule state (with leftover noise canceled). Comfyui batch img2img Help r/StableDiffusion • I made a long guide called [Insights for Intermediates] - How to craft the images you want with A1111, on Civitai. I like to create images like that one: end result. Width. To find the downscale factor in the second part, calculate by: factor = desired total upscale / fixed upscale. The little grey dot on the upper left of the various nodes will minimize a node if clicked. Upscaling: Increasing the resolution and sharpness at the same time. For example, I can load an image, select a model (4xUltrasharp Welcome to the unofficial ComfyUI subreddit. Set up a new comfy instance, either locally or via network. 17K subscribers in the comfyui community. Jun 2, 2024 · Description. Is there a way to "pause the flow" to the latent upscale until a switch is flipped? Reply. I used 4 samplers, running it through a bilinear latent upscale (x1. (you can load and choose your desired image from batch with Load Latent -> Latent from Batch). You can use the UpscaleImageBy node to scale up and down, by using a scale factor < 1. I use latent since I'm only on 4GB GPU and can get a decent resolution I'm happy with in about 30 seconds. Each sampler goes from 75 > 50 > 35 > 30 steps, 1. For upscaling there are many options. Motion LoRAs w/ Latent Upscale: This workflow by Kosinkadink is a good example of Motion LoRAs in action: 7. •. Distorted. I upscaled it to a resolution of 10240x6144 px for us to examine the results. I managed to make a very good workflow with IP-Adapter with regional masks and ControlNet and it's just missing a good upscale. Upscale the refiner result or dont use the refiner. There is no upscaling method that can preserve the latent quality so far. 55. 5, it's dogshit. I've never had good luck with latent upscaling in the past, which is "Upscale Latent By" and then re-sampling. He's the whole reason I've switched to comfy. Class name: LatentUpscale; Category: latent; Output node: False; The LatentUpscale node is designed for upscaling latent representations of images. 5 for the diffusion after scaling. You will save time doing everything in latent, and the end result is good too. Works best with low weights and ending as early as possible, high steps, and a non-ancestral sampler, followed by latent high-res fix, then an img2img based upscale. To drag select multiple nodes, hold down CTRL and drag. I love to go with an SDXL model for the initial image and with a good 1. A proper node for sequential batch inputs, and a means to load separate loras in a composition. 5=1024). ComfyUI SDXL Upscale Tests. You either upscale in pixel space first and then do a low denoise 2nd pass or you upscale in latent space and do a high denoise 2nd pass. ZeonSeven. This repository includes a custom node for ComfyUI for upscaling the latents quickly using a small neural network without needing to decode and encode with VAE. In the second workflow you first configure the workflow which will be used in the remote node. There is a face detailer node. Input types ComfyUI - Ultimate SD Upscaler Tutorial. Wether or not to center-crop the image to maintain the aspect ratio of the original latent images. Connect the Load Upscale model with the Upscale Image (using model) to VAE Decode, then from that image to your preview/save image. It's based on the wonderful example from Sytan, but I un-collapsed it and removed upscaling to make it very simple to understand. upscale_method. Getting started. The node 'Load Upscale Model' doesn't seem to be of any help (only 1 node can be connected to it), please It uses the sampling method to determine the mathematics it will use in denoising each step. Also, if this is new and exciting to you, feel free to post From the ComfyUI_examples, there are two different 2-pass (Hires fix) methods, one is latent scaling, one is non-latent scaling. 5 depending on how much you are upscaling. it will add details to your workflow generally if your noise is set too high but it definitely won't blur and the sharpness would be dependent on your upscale model In the end, it was 30 steps using Heun and Karras that got the best results though. dr_lm. The mixer just helps a little more (hopefully). and Try NNLatentUpscale instead of the regular latent upscale node. Belittling their efforts will get you banned. ComfyUI Neural network latent upscale custom node. 5 X current size. how to upscale image that was created with clip area set using "PixelKSampleUpscalerProviderPipe" Then I have a nice result I do composition ( Image 2). View community ranking In the Top 10% of largest communities on Reddit latent upscale tile an already existing video? i made a bunch of cool animatediff animations but those are small (768x432) This. Txt/Img2Vid + Upscale/Interpolation: This is a very nicely refined workflow by Kaïros featuring upscaling, interpolation, etc. In the provided sample image from ComfyUI_Dave_CustomNode , the Empty Latent Image node Img2Img Upscale - Upscale a real photo? Trying to expand my knowledge, and one of the things I am curious about is upscaling a photo - lets say I have a backup image, but its not the best quality. ps. Through recommended youtube videos i learned that a good way to increase the size and quality of gens i can use iterative upscales first in latent and then iterative upscale for the itself image and also that you can generate pretty high resolutions images with kohyas deep shrink but It seems that Upscayl only uses a upscaling model, so there is no difussion involved and the result will depend only on the upscaling model. I use latent upscale a lot because most of the image upscale models affect the model output to some degree, changing style. And then upscale it by 2 to 1080 by 1920 tiktok/ig video resolution. I wanted to know what difference they make, and they do! Credit to Sytan's SDXL workflow, which I reverse engineered, mostly because I'm new to ComfyUI Latent of size 512x768 is 134 kb. Having compared the details on my 2x latent upscale (with model swap) and your mixed pixel upscale they are not really very different: But I think if I push it above a 2x to around 2. It includes literally everything possible with AI image generation. /r/StableDiffusion is back open after the Hey all, Pretty new to the whole comfyui thing with using 1. This way it replicates the sd upscale/ultimate upscale scripts from A1111. - latent upscale looks much more detailed, but gets rid of the detail of the original image. ComfyUI: An extremely powerful Stable Diffusion GUI with a graph/nodes interface for advanced users that gives you precise control over the diffusion process without coding anything now supports ControlNets View community ranking In the Top 1% of largest communities on Reddit anyone know how to set lowvram in comfyui colab setup? when i use SDXL Base + Refiner + Upscale Latent in colab notebook 1st 2 ksampler work fine then upscale latent process stop colab notebook anyone have solution please share Apr 16, 2024 · Generate image -> VAE decode the latent to image -> upscale the image with model -> VAE encode the image back into latent -> hires. You tell it de noising strength is . To duplicate parts of a workflow from one These comparisons are done using ComfyUI with default node settings and fixed seeds. The big key behind what makes latent highres special is that you're upscaling the latent space (and thus associated noise) and continuing to do diffusion on it. So if you want 2. 5 models but i need some advice on my workflow. Testing was done with that 1/5 of total steps being used in the upscaling. 4/5 of the total steps are done in the base. 1. Maybe all of this doesn't matter, but I like equations. Latent quality is better but the final image deviates significantly from the initial generation. A lot of people are just discovering this technology, and want to show off what they created. Using Comfyui, is there a good way to downscale a 4096x4096 (for example) sized image, sample it then re-upscale it for faster generations? I'm playing around with "Image Scale by Ratio" and "Upscale Latent" but unsure of a good strategy for this, or if this is even a good idea. The process is faster, because is less complex. 45 > 0. I don't get where the problem is, I have checked the comfyui examples and used one of their hires fix, but when I upscale the latent image I get a glitchy image (only the non masked part of the original I2I image) after the second pass, if I upscale the image out of the latent space then into latent again for the second pass the result is ok. 5 with 10 steps on the regular one is the same as setting 20 steps in the advanced sampler and starting at step 10. Curious if anyone knows the most modern, best ComfyUI solutions for these problems? Detailing/Refiner: Keeping same resolution but re-rendering it with a neural network to get a sharper, clearer image. Hands work too with it, but I prefer the MeshGraphormer Hand Refiner controlnet. I tried them all, most gives terrible artifacts with a denoise strength under 0. Ok, comparing the two images, they are certainly different, but not necessarily showing a quality improvement. It changes the image too much and often adds mutations. 0 > 2. It seems a proccess similar to the one we can find in the EXTRAS Menu in Automatic1111 or the upscaling nodes in ComfyUI. I'm trying to find a way of upscaling the SD video up from its 1024x576. • 6 mo. Share. I have a ComfyUI workflow that produces great results. 5 based model and 30 seconds using 30 steps/SD 2. Workflow Included. 56 denoise which is quite high and giving it just enough freedom to totally screw up your image. Super simple yet powerful upscaler node that delivers a detail added upscale to any image! If you use a 'latent upscale' you need to make your denoise on the following ksampler at anywhere from 0. I don't need to go to a really big resolution, I want to achieve a good 2048 x 2048. Then another node under loaders> "load upscale model" node. Many things wrong here. Since you have only 6GB VRAM i would choose Install the ComfyUI dependencies. 6 (60%). 40 > 0. This is solely for ComfyUi. My GPU only has 6GB VRAM, but it was enough before. Hello, A1111 user here, trying to make a transition to Comfyui, or at least to learn of ways to use both. 0 Alpha + SD XL Refiner 1. I don't understand. I spent some time fine-tuning it and really like it. Positive prompt: A galaxy trapped inside a gemstone. Adding extra intermediate frames. . This is not the case. 8x resize in my upscaler). Nearest Exact in this instance creates a checkerboard effect where the latent space between pixels is expanded. This will be the output image. Denoise of 0. I am curious both which nodes are the best for this, and which models. "Upscale Model Loader / Load Upscale Model" "ImageUpscaleWithModel / Upscale Image (using Model)" "Image Save / Image Save" or "SaveImage / Save Image" That will upscale with no latent invention/injection of creative bits, but still intelligently adds pixels per ESRGAN upscaler models. it is a simple way to compare these methods, it is a bit messy as I have no artistic cell in my body. Different methods can affect the quality and characteristics of the upscaled image. Jul 29, 2023 · I can see only 5 methods available : nearest-exact,bilinear,area,bicubic,bislerp. 40 denoise. The target height in pixels. The blurred latent mask does its best to prevent ugly seams. 3. BREAK, shot on Aaton LTR, sharp focus, professional For faces you can use Facedetailer. My understanding is by place this node right before your KSampler, you don't need to upscaling at all, just generate the image at the intended resolution in the first WorkFlow - Choose images from batch to upscale. The resized latents. I personally use the ultimate upscale node in a variety of workflows. Below is my XL Turbo workflow, which includes a lot of toggles and focuses on latent upscaling. crop. example¶ example usage text with workflow image Upscale Latent Documentation. Infinite Zoom: TopQuark67. There are only two things I feel I'm missing. • 5 mo. You usually need higher denoising when using latent upscaling. Overall: - image upscale is less detailed, but more faithful to the image you upscale. I am going to use To increase resolution you have to upscale image, you can do a latent upscale and increase size of image by whatever you want eg. This node is meant to be used in a workflow where the initial image Posted by u/Sqwall - 81 votes and 26 comments You just have to use the node "upscale by" using bicubic method and a fractional value (0. Look at this workflow : 5. Please share your tips, tricks, and workflows for using this software to create your AI art. Latent upscale it or use a model upscale then vae encode it again and then run it through the second sampler. Along with normal image preview other methods are: Latent Upscaled 2x. it can result in things like extra fingers and lines being 23K subscribers in the comfyui community. Better upscaling of the latents fixes that. Award. My nonscientific answer is that A1111 can do it around 60 seconds at 30 steps using a 1. attach to it a "latent_image" in this case it's "upscale latent" Welcome to the unofficial ComfyUI subreddit. If you have another Stable Diffusion UI you might be able to reuse the dependencies. Now there's also a `PatchModelAddDownscale` node. Hello Fellow ComfyUI users, this is my workflow for testing different methods to improve image resolution. No attempts to fix jpg artifacts, etc. I noticed that if I disable Upscale Latent by, the render is fast and clean. Upscale : load upscale model node > Ksampler > VAE decode > output. g. Is there a way to get tiled latent upscale? I looked up some upscaling but they look REALLY convoluted. Put this in your input folder. The target width in pixels. To disable/mute a node (or group of nodes) select them and press CTRL + m. Length defines the ammount of images after the target to send ahead. I'm trying to use a Latent Upscale node on ComfyUI and this is the results I'm getting: Is this normal? I remember using this in the past days (I'm…. This parameter is crucial for determining the starting point of the upscaling process. 00 > 0. I liked the ability in MJ, to choose an image from the batch and upscale just that image. VAE decoded png image of 512x768 is 500+kb, AND it takes like half a minute to convert! Image processing should include enhancing sharpness and texture, softening shadows, and eliminating unwanted artifacts. In your video, you mention that noise increases prompt accuracy, which is interesting - but I feel we need to see more data before substantiating that claim. To improve sharpness search for "was node suite comfyui workflow examples" on Google, should take you to a github page with various workflows, one of them I see is for running hipass for The latent images to be upscaled. For example, if you start with a 512x512 latent empty image, then apply a 4x model, apply "upscale by" 0. Tried the latent upscale on Auto1111 and Stable Swarm and both gave me the same result as the image. 1 but I resize with 4x-Ultrasharp set to x2 and in ComfyUI this workflow uses a nearest/exact latent upscale. The reason for the strange noise artifacts is actually poor latent upscaling between stages. OP • 10 mo. If those were both in I'd be so happy. sd-x2-latent-upscaler is a new latent upscaler trained by Katherine Crowson in collaboration with Stability AI. i still use a latent upscale in my upscale processes to add detail, whatever works really, do some comparisons. So I want to place the latent hiresfix upscale before the refiner, but the Advance KSamplers do not have denoise option and they require start Comfyui Ultimate SD Upscale speed. Now I'm having a blast with it. If you have two instances you connect the output latent from the second one in the "Select current instance" group to the Tiled IP Adapter node. 4, but use a relevent to your image control net so you don't lose to much of your original image, and combining that with the iterative upscaler and concat a secondary posative telling the model to add detail or improve detail. As a result, ① enhanced image ー ② loss of detail during upscaling =. This is the concept: Generate your usual 1024x1024 Image. 5 > 2. The issue I think people run into is that they think the latent upscale is the same as the Latent Upscale from Auto1111. py; Note: Remember to add your models, VAE, LoRAs etc. Thank Welcome to the unofficial ComfyUI subreddit. PS: If someone has access to Magnific AI, please can you upscale and post result for 256x384 (5 jpg quality) and 256x384 (0 jpg quality). And above all, BE NICE. 5 if you want to divide by 2) after upscaling by a model. 5 to get a 1024x1024 final image (512 *4*0. Always wanted to integrate one myself. SDXL CLIP size vs Latent image size. Sort by: Sometimes you'd get part of the subject, but the rest would just be cropped out, especially with smaller subjects. But this sort of research is what makes ComfyUI so awesome. (also may want to try an upscale model>latent upscale, but thats just my personal preference really) I think multiple workflows could accomplish your goal if there was an attention to these other four items. 5. I noticed the memory usage is higher. If you want to do latent upscale, you need at least 0. 3 to 0. outputs¶ LATENT. factor = 2. Try to upscale a latent and do VAE decoding to preview the latent image before and after upscaling. ago • Edited 10 mo. Load the upscaled image to the workflow, use ComfyShop to draw a mask and inpaint. Once you build this you can choose an output from it using static seeds to get specific images or you can split up larger batches of images to reduce Where is the "Denoise" Option in the KSampler BASE (Advanced)-Mode to control the strength of the imput image? Denoise is equivalent to setting the start step on the advanced sampler. ( I am unable to upload the full-sized image. But as soon as I enable Upscale Latent by, the result is very slow and terrible. ago • Edited 6 mo. The latent upscale in ComfyUI is crude as hell, basically just a "stretch this image" type of upscale. This guy's videos are amazing. 2. Right click on the Empty Latent Image Node and select “Convert width to input” and “Convert height to input”. ③ upscaling with optimal detail. New to Comfyui, so not an expert. You can generate a 12 fps video which if you generate 25 frames is ~2 seconds with SVD, and then interpolate it at 5x with the RIFE VFI node and you'll get 2s video at 60fps instead of 12 for little extra cost Jemnite. 0 = 0. That is using an actual SD model to do the upscaling that, afaik, doesn't yet exist in ComfyUI. 0 > 1. The method used for resizing. In Image 3 I compare pre-compose with post-compose results. xo tx ea xz rt xg vo zc il dk