Utilizing the Newest Diffusers Monkey Patching operate to load LoRA produces precisely the identical outcome evaluate with A1111
Pull the newest code from Hugging Face’s Diffusers code repository, and located that the latest code up to date associated to LoRA loading is up to date and may do Monkey-Patching LoRA loading now.
To put in the newest Diffusers:
pip set up -U git+https://github.com/huggingface/diffusers.git@major
The LoRA loading operate was producing barely defective outcomes yesterday, in accordance with my take a look at. This text discusses find out how to use the newest LoRA loader from the Diffusers package deal.
Load LoRA and replace the Secure Diffusion mannequin weight
It has been some time since programmers utilizing Diffusers can’t have the LoRA loaded in a straightforward approach. To load LoRA to a checkpoint mannequin and output the identical outcome as A1111’s Secure Diffusion Webui did, we have to use further customized code to load the weights as I supplied on this article.
The answer supplied on this article works nicely and quick, whereas it requires further administration on the LoRA alpha weight, we have to create a variable to recollect the present LoRA weight α. As a result of the load LoRA code merely provides put the A and B matrix from LoRA collectively.
After which merge with the primary checkpoint mannequin weight W.
To take away the LoRA weights, we’ll want a detrimental -α to take away the LoRA weights, or recreate the pipeline.
The Monkey-Patching technique to load LoRA
One other approach to make use of LoRA is patching the code that executes the module ahead course of, and bringing the LoRA weights throughout the time of calculating textual content embedding and a spotlight rating.
And that is how Diffusers LoraLoaderMixin’s method to LoRA loading. The great a part of this method is that no mannequin weight is up to date, we are able to simply reset the LoRA and supply a brand new α to outline the LoRA weight.
