ControlNet pipelines

docker-diffusers-api
4

Hey @aroop. Thanks as usual for your great post. Re your questions:

  1. Happy to report that custom pipelines are already supported and working great. See e.g. ALL YOUR PIPELINES (are belong to us) and Lpw_stable_diffusion pipeline (longer prompts, prompt weights!) as an example.

  2. We’re on the same page. Maybe something like:

{
  callInputs: {
    controlnet: [
      "lllyasviel/sd-controlnet-openpose",
      "lllyasviel/sd-controlnet-canny",
    ],
    // and MODEL_ID, pipeline, etc.
  },
  modelInputs: {
    image: [
      "...base64encoded...",
      "...base64encoded...",
    ],
    controlnet_conditioning_scale: [1.0, 0.8],
  }
}

where the arrays values could also be a single string (non-array). But that’s just my thoughts from glancing through the blog post, let’s see what you encounter when actually implementing this.

  1. Sounds great! Thanks for thinking of this ahead of time.

Re version locking, yeah, you found the relevant posts, thanks for searching. You can use the latest version in your fork. I’ll get :dev up-to-date later this week and we can address any merge conflicts later. (I like to at least skim through upstream commit log when bumping versions to look out for anything that could affect us, and make sure all tests pass too of course).

Thanks again @aroop, looking forward to working together on this! :raised_hands: