Issues with models from Civitai

I appreciate all the work you鈥檙e doing on this repository, I was hoping it to be my only requirement but it looks like I have to use automatic1111鈥檚 web ui backend or something. On the website https://civitai.com/ you can download SD models. Not a single model on that website whether it be .safetenors or .ckpt files I ever got working on the docker-diffusers template. Either the notebook crashes entirely, or, when I convert it to diffusers first on a space of the website of HuggingFace, the model is just broken. I have no idea why they do work on the web ui.

Thanks, man, appreciate it.

Is this all with recent docker-diffusers-api versions? Say :dev releases? I鈥檓 all the time updating the version of diffusers we use (and dependencies), and it often has fixes for this kind of thing.

Having said that, I鈥檝e never personally tried using any models from civitai. This is definitely something I鈥檇 like to support, so will be happy to look into this, however, I have a really long list of important things I鈥檓 slowly working through, so it鈥檚 hard to commit to a time yet. Would you like to spend some time on this before I start on the LoRA training stuff?

Some options if you have time to spend on this before I do:

  1. check with latest :dev release, if conversion works.
  2. open an issue on Issues 路 huggingface/diffusers 路 GitHub with full details including link to the civitai model, any errors you see, etc. they might want you to provide a reproduction in pure diffusers though, maybe.

Also, just wanted to double check that we鈥檙e talking about full SD models (as I understood from your message) and not just the LoRA weights. I don鈥檛 think there鈥檚 any way to currently use LoRA safetensor weights at least but there鈥檚 a PR on that started yesterday at Adding support for `safetensors` and LoRa. by Narsil 路 Pull Request #2448 路 huggingface/diffusers 路 GitHub.

Curious, how are you pointing to the CivitAI models? Are you pulling down locally and adding it to the container somewhere or at build time? I have only seen support for HF.

@gadicc Are there any examples of using custom ckpts? I can do the fine tuning locally but I don鈥檛 know how to get that into diffusers-api with the given docs?

@tehfailsafe

All you need to do is provide the URL to your .ckpt checkpoint in the dockerfile and it will automatically download it for you.
ARG CHECKPOINT_URL=鈥https://yoursite.com/models/yourmodel.ckpt

As the MODEL_ID, you can use any name but I highly recommend using the same name as checkpoint filename without the file extension. (so in this case 鈥榶ourmodel鈥)

When you call the API, simply provide that MODEL_ID as the model and it will work out of the box.

1 Like

Welcome, @tehfailsafe!

As the awesome @jochemstoel says, you just need to set CHECKPOINT_URL (and MODEL_ID to something unique).

Open to suggestions on how to make this clearer in the docs. It鈥檚 mentioned briefly in the readme and in Using custom checkpoint (.ckpt) files.

Thanks :slight_smile:

@gadicc When I download a picke tensor from CivitAI, pruned or unpruned, your image builds successfully but no matter what scheduler I use, I get these types of results:

Do you have any idea why this is? If you can tell me how to fix this, that鈥檇 be awesome.

Haven鈥檛 seen that before :confused: I understand it鈥檚 every model you鈥檝e tried from there? But either way send me a direct link so I can experiment locally and try figure out what鈥檚 going on. Weird! I had a quick look for any mention of civitai in diffusers issues and didn鈥檛 see anything specific. But let me play around a bit and see if I can figure anything out. In the logs is it running all the num_inference_steps ?