[WIP] Upgrading from v0 to v1

[gadicc]

This guide is a work-in-progress (v1 isn’t officially released yet), but you can help improve it! DO NOT USE IN PRODUCTION (unless you’re kiri.art :)).

“How can I help?” See if you can get up and running using the instructions and guides linked below, and let us know what doesn’t work or isn’t clear, so we can fix it for everyone ahead of the official release (around end of Jan).

Breaking Changes

1. New “split” architecture. We now have pre-built docker images that are super easy to deploy and upgrade. See further instructions below, it’s unlikely you’ll want to clone / pull the main repo anymore.

2. Our own optimization with super fast inits. We no longer support banana’s optimization, but ours is faster anyways, and this frees us to explore a number of other optimization strategies for even faster inference moving forwards.

3. PRECISION build-arg was renamed MODEL_PRECISION to match call-arg.

4. MODEL_PRECISION and MODEL_REVISION must now both be explicitly set. This gives a lot more flexibility for Hugging Face repos that don’t follow the convention of them being equal, and, in anycase, diffusers are deprecating revision - see next point).

5. Upcoming: MODEL_VARIANT will replace the deprecated MODEL_REVISION by diffusers. “Revision” required a separate branch on the repo for the desired revision, which worked great when people did it, but mostly just made a mess. For this reason, diffusers team is instead suggesting for variants to be kept in the same repo branch with other files, but with a variation in the filename, and it will only download these files. Watch out for more info in diffusers when this is officially announced.

6. Upcoming: $timings.inference (and send()) will be renamed to $timings.handler to cover the entire handler (which can do training too, e.g. dreambooth), and a new .inference will be used to track just inference when inference occurs.

Automated testing w/ semver releases

I’ve spent a loooooot of time building out a new development flow. On every merge to main, in addition to unit testing (run on all branches), a suite of integration tests will be run to ensure all features work as expected in a real environment. If all tests pass, the commit history is analyzed to tag the new release with the appropriate semantic version number, and this is pushed to docker hub.

Deployment

There are a number of guides available for various providers (not all are up-to-date at time of writing… if they still mention cloning the original repo, please nudge us). But in general:

• Servers: In one line! Call docker run --gpus all -p 8000:8000 -e MYVAR1="value" -e MYVAR2="etc" gadicc/diffusers-api (possible environment variables may include your S3 or compatible credentials, for example). This uses the new “runtime downloads” feature by default, and will download models as needed. Optionally adapt -v ~/root-cache:/root/.cache for persistent storage, if relevant / desired.

• Serverless: See the “build-download” repo (banana, others) or “runpod” repo (for runpod.io) which use the above image but download and storage the model into your image at build time.

[DaveG2]

Hi there, I’ve figured out how to get depth2img working using v0 and banana, it required additional transformers package library. I tried to upgrade to v1 split, I have no idea about docker but is it possible to edit in docker-diffusers-api-build-download repo? I cannot find where I can add a line to install transformers package. Sorry noob here

[gadicc]

Hey, welcome! No worries at all, it’s a good question.

I’ll note firstly that docker-diffusers-api does already include transformers, and actually, v1.3.0 that I pushed on Friday (but haven’t announced yet) includes the latest version of it.

Nevertheless, it’s no problem to add additional packages in the -build-download repo… just add a line to Dockerfile, before the final CMD line (which should remain the last line), with e.g.

RUN pip install transformers

and then rebuild locally if developing locally or redeploy to say banana which will rebuild for you on each push. But again, you shouldn’t really need to do this as docker-diffusers-api v1.3.0 has the latest transformers version.

In any event, awesome work getting depth2img working I haven’t even had a chance to look into that yet And feel free to start a new topic sharing your experiences with it, I’m sure a lot of others would be very interested too.