by xinsir
Open source · 92k downloads · 1712 likes
ControlNet Union SDXL 1.0 is an advanced model that combines multiple image control and editing features into a single versatile tool. It enables the generation or modification of images from various conditions—such as outlines, poses, or masks—simultaneously without compromising quality or increasing technical parameters. Thanks to its optimized architecture, it supports over ten different control types while maintaining compatibility with other SDXL models and LoRAs, offering exceptional flexibility. Its editing capabilities include tools like blur reduction, stylistic variation, and super-resolution, allowing for precise enhancement or transformation of existing images. This model stands out for its ability to produce high-resolution images comparable to professional tools while remaining accessible and open source for the community.
We design a new architecture that can support 10+ control types in condition text-to-image generation and can generate high resolution images visually comparable with midjourney. The network is based on the original ControlNet architecture, we propose two new modules to: 1 Extend the original ControlNet to support different image conditions using the same network parameter. 2 Support multiple conditions input without increasing computation offload, which is especially important for designers who want to edit image in detail, different conditions use the same condition encoder, without adding extra computations or parameters. We do thoroughly experiments on SDXL and achieve superior performance both in control ability and aesthetic score. We release the method and the model to the open source community to make everyone can enjoy it.
Inference scripts and more details can found: https://github.com/xinsir6/ControlNetPlus/tree/main
If you find it useful, please give me a star, thank you very much
SDXL ProMax version has been released!!!，Enjoy it!!!
I am sorry that because of the project's revenue and expenditure are difficult to balance, the GPU resources are assigned to other projects that are more likely to be profitable, the SD3 trainging is stopped until I find enough GPU supprt, I will try my best to find GPUs to continue training. If this brings you inconvenience, I sincerely apologize for that. I want to thank everyone who likes this project, your support is what keeps me going
Note: we put the promax model with a promax suffix in the same huggingface model repo, detailed instructions will be added later.
Following example show from 1M resolution --> 9M resolution
