This is a sub-topic of Running on other cloud providers - Setup - Kiri.Art Forums. See that post for why you might want to do this, and info and examples for other providers too.
Updated for docker-diffusers-api v1. Please let us know if anything doesn’t work.
1. Setup Docker
- Connect to your instance
$ ssh ubuntu@ip-address
output
The authenticity of host '129.154.239.57 (129.154.239.57)' can't be established.
ED25519 key fingerprint is SHA256:r4uuy5yDDnWEPxYAuYmSOUzHOHun++emCnnFPUsRO+0.
This key is not known by any other names.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '129.154.239.57' (ED25519) to the list of known hosts.
Welcome to Ubuntu 20.04.5 LTS (GNU/Linux 5.15.0-52-generic x86_64)
.============.
|| __ || _ _ _
|| \_\ || | | __ _ _ __ ___ | |__ __| | __ _
|| \_\ || | | / _` | '_ ` _ \| '_ \ / _` |/ _` |
|| /_λ_\ || | |__| (_| | | | | | | |_) | (_| | (_| |
|| /_/ \_\ || |_____\__,_|_| |_| |_|_.__/ \__,_|\__,_|
.============. GPU CLOUD
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Fri Dec 9 07:23:53 UTC 2022
System load: 0.02 Processes: 380
Usage of /: 3.9% of 496.03GB Users logged in: 0
Memory usage: 0% IPv4 address for docker0: 172.17.0.1
Swap usage: 0% IPv4 address for enp5s0: 10.19.101.99
* Super-optimized for small spaces - read how we shrank the memory
footprint of MicroK8s to make it the smallest full K8s around.
https://ubuntu.com/blog/microk8s-memory-optimisation
3 updates can be applied immediately.
3 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable
The list of available updates is more than a week old.
To check for new updates run: sudo apt update
Your Hardware Enablement Stack (HWE) is supported until April 2025.
/usr/bin/xauth: file /home/ubuntu/.Xauthority does not exist
- Add yourself to the docker group:
sudo usermod -aG docker ubuntu && newgrp docker
output with checks
$ whoami
dragon
$ id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),100(users),117(admin)
$ sudo usermod -aG docker ubuntu
$ newgrp ubuntu
$ id
uid=1000(ubuntu) gid=998(docker) groups=998(docker),100(users),117(admin),1000(ubuntu)
2. Clone and Build (Optional)
If you want to be able to make changes to the code, include these steps. If you just want to get docker-diffusers-api running asap, skip to Step 3: Running the Container in the next step, below.
- Clone the repository
$ git clone https://github.com/kiri-art/docker-diffusers-api
output
Cloning into 'docker-diffusers-api'...
remote: Enumerating objects: 817, done.
remote: Counting objects: 100% (325/325), done.
remote: Compressing objects: 100% (115/115), done.
remote: Total 817 (delta 238), reused 281 (delta 208), pack-reused 492
Receiving objects: 100% (817/817), 6.11 MiB | 5.43 MiB/s, done.
Resolving deltas: 100% (477/477), done.
-
cd docker-diffusers-api -
Building the Image
# Add any other build-args you need, don't miss the "." at the end
$ docker build -t gadic/diffusers-api --build-arg HF_AUTH_TOKEN="hf_XXX" .
output (on pastebin)
See also the included build script, which includes some common build args taken from local environment variables, and uses docker’s new(ish) BUILDKIT architecture for much faster rebuilds.
3. Running the Container
If you skipped Step 2 above, this single command will automatically download the latest version from docker hub for you. Otherwise, it will run the image you just built (and can rebuild again, with the same build command, if you make any changes to the code).
# Replace -it with -d if you want to run in the background,
# otherwise Ctrl-C to exit
$ docker run -it -p 8000:8000 --gpus all gadicc/diffusers-api
output
/opt/conda/envs/xformers/lib/python3.10/site-packages/transformers/image_utils.py:239: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
def resize(self, image, size, resample=PIL.Image.BILINEAR, default_to_square=True, max_size=None):
/opt/conda/envs/xformers/lib/python3.10/site-packages/transformers/image_utils.py:396: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.
def rotate(self, image, angle, resample=PIL.Image.NEAREST, expand=0, center=None, translate=None, fillcolor=None):
/opt/conda/envs/xformers/lib/python3.10/site-packages/transformers/models/clip/feature_extraction_clip.py:67: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.
resample=Image.BICUBIC,
environ({'CONDA_SHLVL': '2', 'LD_LIBRARY_PATH': '/usr/local/nvidia/lib:/usr/local/nvidia/lib64', 'REQUESTS_CA_BUNDLE': '', 'CONDA_EXE': '/opt/conda/bin/conda', '_': '/opt/conda/envs/xformers/bin/python3', 'MODEL_URL': '', 'HOSTNAME': '0518531ad9dd', 'PRECISION': 'fp16', 'HF_AUTH_TOKEN': 'hf_XXX', 'AWS_SECRET_ACCESS_KEY': '', 'PIPELINE': 'ALL', 'USE_DREAMBOOTH': '0', 'CONDA_PREFIX': '/opt/conda/envs/xformers', 'AWS_S3_ENDPOINT_URL': '', 'NVIDIA_VISIBLE_DEVICES': 'all', 'AWS_DEFAULT_REGION': '', '_CE_M': '', 'SIGN_KEY': '', 'CONDA_PREFIX_1': '/opt/conda', 'PWD': '/api', 'HOME': '/root', 'CONDA_PYTHON_EXE': '/opt/conda/bin/python', 'LC_CTYPE': 'C.UTF-8', 'CHECKPOINT_CONFIG_URL': '', 'PYTORCH_VERSION': 'v1.12.1-rc5', '_CONVERT_SPECIAL': '', 'https_proxy': '', 'DEBIAN_FRONTEND': 'noninteractive', 'http_proxy': '', '_CE_CONDA': '', 'MODEL_ID': 'stabilityai/stable-diffusion-2', 'CONDA_PROMPT_MODIFIER': '(xformers) ', 'TERM': 'xterm', 'NVIDIA_DRIVER_CAPABILITIES': 'compute,utility', 'CONDA_ROOT': '/opt/conda', 'AWS_ACCESS_KEY_ID': '', 'SHLVL': '2', 'CHECKPOINT_URL': '', 'SEND_URL': '', 'PATH': '/opt/conda/envs/xformers/bin:/opt/conda/condabin:/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin', 'CONDA_DEFAULT_ENV': 'xformers'})
2022-12-09 07:53:55.318327 {'type': 'init', 'status': 'start', 'container_id': '0518531ad9dd2e3b0efb686d9b5c92911bf51e5c08b283b09f55ad63e10bc96a', 'time': 1670572435318, 't': 0, 'tsl': 197, 'payload': {'device': 'NVIDIA A100-SXM4-40GB', 'hostname': '0518531ad9dd', 'model_id': 'stabilityai/stable-diffusion-2', 'diffusers': '0.9.0'}, 'init': True}
Loading model: stabilityai/stable-diffusion-2
Initializing DPMSolverMultistepScheduler for stabilityai/stable-diffusion-2...
Initialized DPMSolverMultistepScheduler for stabilityai/stable-diffusion-2 in 2ms
<frozen importlib._bootstrap>:283: DeprecationWarning: the load_module() method is deprecated and slated for removal in Python 3.12; use exec_module() instead
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion.StableDiffusionPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion_img2img.StableDiffusionImg2ImgPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion_inpaint.StableDiffusionInpaintPipeline'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
You have disabled the safety checker for <class 'diffusers.pipelines.stable_diffusion.pipeline_stable_diffusion_inpaint_legacy.StableDiffusionInpaintPipelineLegacy'> by passing `safety_checker=None`. Ensure that you abide to the conditions of the Stable Diffusion license and do not expose unfiltered results in services or applications open to the public. Both the diffusers team and Hugging Face strongly recommend to keep the safety filter enabled in all public facing circumstances, disabling it only for use-cases that involve analyzing network behavior or auditing its results. For more information, please have a look at https://github.com/huggingface/diffusers/pull/254 .
2022-12-09 07:53:57.984130 {'type': 'init', 'status': 'done', 'container_id': '0518531ad9dd2e3b0efb686d9b5c92911bf51e5c08b283b09f55ad63e10bc96a', 'time': 1670572437984, 't': 2666, 'tsl': 2666, 'payload': {}}
[2022-12-09 07:53:57 +0000] [25] [INFO]
┌────────────────────────────────────────────────────────────────────────┐
│ Sanic v22.6.2 │
│ Goin' Fast @ http://0.0.0.0:8000 │
├───────────────────────┬────────────────────────────────────────────────┤
│ │ mode: production, single worker │
│ ▄███ █████ ██ │ server: sanic, HTTP/1.1 │
│ ██ │ python: 3.10.8 │
│ ▀███████ ███▄ │ platform: Linux-5.15.0-52-generic-x86_64-with- │
│ ██ │ glibc2.27 │
│ ████ ████████▀ │ packages: sanic-routing==22.3.0 │
│ │ │
│ Build Fast. Run Fast. │ │
└───────────────────────┴────────────────────────────────────────────────┘
[2022-12-09 07:53:57 +0000] [25] [WARNING] Sanic is running in PRODUCTION mode. Consider using '--debug' or '--dev' while actively developing your application.
[2022-12-09 07:53:58 +0000] [25] [INFO] Starting worker [25]
See also the included run.sh script for common environmental variables, persistent volumes, etc.
4. Testing
The server listens on port 8000 (or whatever you set it to in the previous command), and can be tested in a variety of ways. Here’s one simple way, assuming you cloned the repo in Step 2:
- Open a second ssh terminal
cd docker-diffusers-apipython test.py txt2img
Test (command) output
unning test: txt2img
Request took 12.8s (init: 2.7s, inference: 12.8s)
Saved /home/ubuntu/docker-diffusers-api/tests/output/txt2img.png
Server output
{
"modelInputs": {
"prompt": "realistic field of grass"
},
"callInputs": {
"MODEL_ID": "runwayml/stable-diffusion-v1-5",
"PIPELINE": "StableDiffusionPipeline",
"SCHEDULER": "LMSDiscreteScheduler"
}
}
[2022-12-09 07:57:59 +0000] - (sanic.access)[INFO][172.17.0.1:35864]: POST http://localhost:8000/ 200 262
{
"modelInputs": {
"prompt": "realistic field of grass"
},
"callInputs": {
"MODEL_ID": "stabilityai/stable-diffusion-2",
"PIPELINE": "StableDiffusionPipeline",
"SCHEDULER": "LMSDiscreteScheduler"
}
}
Initializing LMSDiscreteScheduler for stabilityai/stable-diffusion-2...
Initialized LMSDiscreteScheduler for stabilityai/stable-diffusion-2 in 1ms
2022-12-09 07:58:22.694172 {'type': 'inference', 'status': 'start', 'container_id': '0518531ad9dd2e3b0efb686d9b5c92911bf51e5c08b283b09f55ad63e10bc96a', 'time': 1670572702694, 't': 0, 'tsl': 264710, 'payload': {'startRequestId': None}, 'init': True}
pipeline.enable_xformers_memory_efficient_attention()
100%|███████████████████████████████████████████| 50/50 [00:11<00:00, 4.20it/s]
2022-12-09 07:58:35.452958 {'type': 'inference', 'status': 'done', 'container_id': '0518531ad9dd2e3b0efb686d9b5c92911bf51e5c08b283b09f55ad63e10bc96a', 'time': 1670572715453, 't': 12759, 'tsl': 12759, 'payload': {'startRequestId': None}}
[2022-12-09 07:58:35 +0000] - (sanic.access)[INFO][172.17.0.1:41650]: POST http://localhost:8000/ 200 1083700
That’s it!
Let us know of any issues or hacks you came up with below, and happy diffusing! 