Earlier this month, Chinese college student Ni Hao (倪豪), who’s about to graduate this year, shared a new project named ControlNet for QR Code, on his official account. This project uses ControlNet for model training to generate a QR code image that retains the specific pattern contained in the input image and can be scanned and jump to a specified target.

The project is a continuation of qrbtf.com, a parametric QR code generator that NI Hao did during his sophomore year of college. Although the idea of "encoding hidden information in any image that is completely normal for naked eye observation" appeared very early, but like Ni said, "in the era of GAN, the ecology of machine learning was far less active than it is today, not to mention that there is no easy-to-use frameworks like Gradio Web UI, Diffusers, just configuring the environment is enough to dissuade me, so I put this idea on hold."

With the rapid development of Stable Diffusion and ControlNet, it was finally restarted by Ni again. According to Ni, "In the previous JAX Sprint, we were fortunate to use Google TPU v4 to complete the training of 3 million images very quickly. Unfortunately, after the event ended, we returned to the A6000 / 4090 in the lab and trained a version with 100,000 images, hoping "Sudden Convergence" comes as soon as possible."

After using ControlNet to train a large number of pictures and obtain a stable model, Ni and his team tried a variety of combinations of "Checkpoint + LoRA + QR Code ControlNet", and obtained a variety of artistic styles QR code images that are recognizable and functional.

Ni has shared several samples in various art style generated while testing, as long as the training process and model download address for whoever interested to give a try. Below are some cases Ni made during training, you may give a look or simply use your phone to scan the image and see if they work.

Model 01: Chinese Traditional Pattern

LoRA Training Process: aigc.ioclab.com/sd-showcase/chinese-ornament

LoRA Model Download: civitai.com/models/29858/chinese-traditional-pattern

Model 02: Ukiyo-e Style

LoRA Training Process: aigc.ioclab.com/sd-showcase/fuyue

LoRA Model Download: civitai.com/models/25222/ukiyo-e-fuyue-style-background-mix

Model 03: Anime Style

Model 03: Three-dimensional Style

So far, Ni only shared three styles of LoRA model (Chinese traditional pattern, Ukiyo-e and Genshin). More model releases and technical documents are still being updated. If you are interested, you can check his technical documents (aigc.ioclab.com) from time to time.

XRender | Fast · Affordable · Reliable