cxllm-vendors/stable-diffusion-webui-forge
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Gradio 4 + WebUI 1.10

Browse Source
This commit is contained in:
layerdiffusion
2024-07-26 08:51:34 -07:00
parent e95333c556
commit e26abf87ec
201 changed files with 7562 additions and 4834 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.eslintrc.js
+2
View File
@@ -78,6 +78,8 @@ module.exports = {
//extraNetworks.js
requestGet: "readonly",
popup: "readonly",
// profilerVisualization.js
createVisualizationTable: "readonly",
// from python
localization: "readonly",
// progrssbar.js
.github/ISSUE_TEMPLATE/bug_report.yml
-105
View File
@@ -1,105 +0,0 @@
name: Bug Report
description: You think something is broken in the UI
title: "[Bug]: "
labels: ["bug-report"]
body:
- type: markdown
attributes:
value: |
> The title of the bug report should be short and descriptive.
> Use relevant keywords for searchability.
> Do not leave it blank, but also do not put an entire error log in it.
- type: checkboxes
attributes:
label: Checklist
description: |
Please perform basic debugging to see if extensions or configuration is the cause of the issue.
Basic debug procedure
　1. Disable all third-party extensions - check if extension is the cause
　2. Update extensions and webui - sometimes things just need to be updated
　3. Backup and remove your config.json and ui-config.json - check if the issue is caused by bad configuration
　4. Delete venv with third-party extensions disabled - sometimes extensions might cause wrong libraries to be installed
　5. Try a fresh installation webui in a different directory - see if a clean installation solves the issue
Before making a issue report please, check that the issue hasn't been reported recently.
options:
- label: The issue exists after disabling all extensions
- label: The issue exists on a clean installation of webui
- label: The issue is caused by an extension, but I believe it is caused by a bug in the webui
- label: The issue exists in the current version of the webui
- label: The issue has not been reported before recently
- label: The issue has been reported before but has not been fixed yet
- type: markdown
attributes:
value: |
> Please fill this form with as much information as possible. Don't forget to "Upload Sysinfo" and "What browsers" and provide screenshots if possible
- type: textarea
id: what-did
attributes:
label: What happened?
description: Tell us what happened in a very clear and simple way
placeholder: |
txt2img is not working as intended.
validations:
required: true
- type: textarea
id: steps
attributes:
label: Steps to reproduce the problem
description: Please provide us with precise step by step instructions on how to reproduce the bug
placeholder: |
1. Go to ...
2. Press ...
3. ...
validations:
required: true
- type: textarea
id: what-should
attributes:
label: What should have happened?
description: Tell us what you think the normal behavior should be
placeholder: |
WebUI should ...
validations:
required: true
- type: dropdown
id: browsers
attributes:
label: What browsers do you use to access the UI ?
multiple: true
options:
- Mozilla Firefox
- Google Chrome
- Brave
- Apple Safari
- Microsoft Edge
- Android
- iOS
- Other
- type: textarea
id: sysinfo
attributes:
label: Sysinfo
description: System info file, generated by WebUI. You can generate it in settings, on the Sysinfo page. Drag the file into the field to upload it. If you submit your report without including the sysinfo file, the report will be closed. If needed, review the report to make sure it includes no personal information you don't want to share. If you can't start WebUI, you can use --dump-sysinfo commandline argument to generate the file.
placeholder: |
1. Go to WebUI Settings -> Sysinfo -> Download system info.
If WebUI fails to launch, use --dump-sysinfo commandline argument to generate the file
2. Upload the Sysinfo as a attached file, Do NOT paste it in as plain text.
validations:
required: true
- type: textarea
id: logs
attributes:
label: Console logs
description: Please provide **full** cmd/terminal logs from the moment you started UI to the end of it, after the bug occured. If it's very long, provide a link to pastebin or similar service.
render: Shell
validations:
required: true
- type: textarea
id: misc
attributes:
label: Additional information
description: |
Please provide us with any relevant additional info or context.
Examples:
　I have updated my GPU driver recently.
.github/ISSUE_TEMPLATE/config.yml
-5
View File
@@ -1,5 +0,0 @@
blank_issues_enabled: false
contact_links:
- name: WebUI Community Support
url: https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions
about: Please ask and answer questions here.
.github/ISSUE_TEMPLATE/feature_request.yml
-40
View File
@@ -1,40 +0,0 @@
name: Feature request
description: Suggest an idea for this project
title: "[Feature Request]: "
labels: ["enhancement"]
body:
- type: checkboxes
attributes:
label: Is there an existing issue for this?
description: Please search to see if an issue already exists for the feature you want, and that it's not implemented in a recent build/commit.
options:
- label: I have searched the existing issues and checked the recent builds/commits
required: true
- type: markdown
attributes:
value: |
*Please fill this form with as much information as possible, provide screenshots and/or illustrations of the feature if possible*
- type: textarea
id: feature
attributes:
label: What would your feature do ?
description: Tell us about your feature in a very clear and simple way, and what problem it would solve
validations:
required: true
- type: textarea
id: workflow
attributes:
label: Proposed workflow
description: Please provide us with step by step information on how you'd like the feature to be accessed and used
value: |
1. Go to ....
2. Press ....
3. ...
validations:
required: true
- type: textarea
id: misc
attributes:
label: Additional information
description: Add any other context or screenshots about the feature request here.
.github/pull_request_template.md
-15
View File
@@ -1,15 +0,0 @@
## Description
* a simple description of what you're trying to accomplish
* a summary of changes in code
* which issues it fixes, if any
## Screenshots/videos:
## Checklist:
- [ ] I have read [contributing wiki page](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Contributing)
- [ ] I have performed a self-review of my own code
- [ ] My code follows the [style guidelines](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Contributing#code-style)
- [ ] My code passes [tests](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Tests)
.github/workflows/on_pull_request.yaml
-38
View File
@@ -1,38 +0,0 @@
name: Linter
on:
- push
- pull_request
jobs:
lint-python:
name: ruff
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
steps:
- name: Checkout Code
uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: 3.11
# NB: there's no cache: pip here since we're not installing anything
# from the requirements.txt file(s) in the repository; it's faster
# not to have GHA download an (at the time of writing) 4 GB cache
# of PyTorch and other dependencies.
- name: Install Ruff
run: pip install ruff==0.1.6
- name: Run Ruff
run: ruff .
lint-js:
name: eslint
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
steps:
- name: Checkout Code
uses: actions/checkout@v3
- name: Install Node.js
uses: actions/setup-node@v3
with:
node-version: 18
- run: npm i --ci
- run: npm run lint
.github/workflows/run_tests.yaml
-107
View File
@@ -1,107 +0,0 @@
name: Tests
on:
- push
- pull_request
env:
FORGE_CQ_TEST: "True"
jobs:
test:
name: tests on CPU
runs-on: ubuntu-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
steps:
- name: Checkout Code
uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v4
with:
python-version: 3.10.6
cache: pip
cache-dependency-path: |
**/requirements*txt
launch.py
- name: Cache models
id: cache-models
uses: actions/cache@v3
with:
path: models
key: "2023-12-30"
- name: Install test dependencies
run: pip install wait-for-it -r requirements-test.txt
env:
PIP_DISABLE_PIP_VERSION_CHECK: "1"
PIP_PROGRESS_BAR: "off"
- name: Setup environment
run: python launch.py --skip-torch-cuda-test --exit
env:
PIP_DISABLE_PIP_VERSION_CHECK: "1"
PIP_PROGRESS_BAR: "off"
TORCH_INDEX_URL: https://download.pytorch.org/whl/cpu
WEBUI_LAUNCH_LIVE_OUTPUT: "1"
PYTHONUNBUFFERED: "1"
- name: Print installed packages
run: pip freeze
- name: Download models
run: |
declare -a urls=(
"https://huggingface.co/lllyasviel/fav_models/resolve/main/fav/realisticVisionV51_v51VAE.safetensors"
)
for url in "${urls[@]}"; do
filename="models/Stable-diffusion/${url##*/}" # Extracts the last part of the URL
if [ ! -f "$filename" ]; then
curl -Lo "$filename" "$url"
fi
done
# - name: Download ControlNet models
# run: |
# declare -a urls=(
# "https://huggingface.co/lllyasviel/ControlNet-v1-1/resolve/main/control_v11p_sd15_canny.pth"
# )
# for url in "${urls[@]}"; do
# filename="models/ControlNet/${url##*/}" # Extracts the last part of the URL
# if [ ! -f "$filename" ]; then
# curl -Lo "$filename" "$url"
# fi
# done
- name: Start test server
run: >
python -m coverage run
--data-file=.coverage.server
launch.py
--skip-prepare-environment
--skip-torch-cuda-test
--test-server
--do-not-download-clip
--no-half
--disable-opt-split-attention
--always-cpu
--api-server-stop
--ckpt models/Stable-diffusion/realisticVisionV51_v51VAE.safetensors
2>&1 | tee output.txt &
- name: Run tests
run: |
wait-for-it --service 127.0.0.1:7860 -t 20
python -m pytest -vv --junitxml=test/results.xml --cov . --cov-report=xml --verify-base-url test
# TODO(huchenlei): Enable ControlNet tests. Currently it is too slow to run these tests on CPU with
# real SD model. We need to find a way to load empty SD model.
# - name: Run ControlNet tests
# run: >
# python -m pytest
# --junitxml=test/results.xml
# --cov ./extensions-builtin/sd_forge_controlnet
# --cov-report=xml
# --verify-base-url
# ./extensions-builtin/sd_forge_controlnet/tests
- name: Kill test server
if: always()
run: curl -vv -XPOST http://127.0.0.1:7860/sdapi/v1/server-stop && sleep 10
- name: Upload main app output
uses: actions/upload-artifact@v3
if: always()
with:
name: output
path: output.txt
.github/workflows/warns_merge_master.yml
-19
View File
@@ -1,19 +0,0 @@
name: Pull requests can't target master branch
"on":
pull_request:
types:
- opened
- synchronize
- reopened
branches:
- master
jobs:
check:
runs-on: ubuntu-latest
steps:
- name: Warning marge into master
run: |
echo -e "::warning::This pull request directly merge into \"master\" branch, normally development happens on \"dev\" branch."
exit 1
.gitignore
+7 -1
View File
@@ -2,6 +2,9 @@ __pycache__
*.ckpt
*.safetensors
*.pth
*.dev.js
.DS_Store
/output/
/ESRGAN/*
/SwinIR/*
/repositories
@@ -39,6 +42,9 @@ notification.mp3
/package-lock.json
/.coverage*
/test/test_outputs
/cache
trace.json
/sysinfo-????-??-??-??-??.json
/test/results.xml
coverage.xml
**/tests/**/expectations
**/tests/**/expectations
CHANGELOG.md
+411 -6
View File
@@ -1,3 +1,407 @@
## 1.10.0
### Features:
* A lot of performance improvements (see below in Performance section)
* Stable Diffusion 3 support ([#16030](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16030), [#16164](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16164), [#16212](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16212))
* Recommended Euler sampler; DDIM and other timestamp samplers currently not supported
* T5 text model is disabled by default, enable it in settings
* New schedulers:
* Align Your Steps ([#15751](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15751))
* KL Optimal ([#15608](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15608))
* Normal ([#16149](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16149))
* DDIM ([#16149](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16149))
* Simple ([#16142](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16142))
* Beta ([#16235](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16235))
* New sampler: DDIM CFG++ ([#16035](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16035))
### Minor:
* Option to skip CFG on early steps ([#15607](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15607))
* Add --models-dir option ([#15742](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15742))
* Allow mobile users to open context menu by using two fingers press ([#15682](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15682))
* Infotext: add Lora name as TI hashes for bundled Textual Inversion ([#15679](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15679))
* Check model's hash after downloading it to prevent corruped downloads ([#15602](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15602))
* More extension tag filtering options ([#15627](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15627))
* When saving AVIF, use JPEG's quality setting ([#15610](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15610))
* Add filename pattern: `[basename]` ([#15978](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15978))
* Add option to enable clip skip for clip L on SDXL ([#15992](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15992))
* Option to prevent screen sleep during generation ([#16001](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16001))
* ToggleLivePriview button in image viewer ([#16065](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16065))
* Remove ui flashing on reloading and fast scrollong ([#16153](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16153))
* option to disable save button log.csv ([#16242](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16242))
### Extensions and API:
* Add process_before_every_sampling hook ([#15984](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15984))
* Return HTTP 400 instead of 404 on invalid sampler error ([#16140](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16140))
### Performance:
* [Performance 1/6] use_checkpoint = False ([#15803](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15803))
* [Performance 2/6] Replace einops.rearrange with torch native ops ([#15804](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15804))
* [Performance 4/6] Precompute is_sdxl_inpaint flag ([#15806](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15806))
* [Performance 5/6] Prevent unnecessary extra networks bias backup ([#15816](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15816))
* [Performance 6/6] Add --precision half option to avoid casting during inference ([#15820](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15820))
* [Performance] LDM optimization patches ([#15824](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15824))
* [Performance] Keep sigmas on CPU ([#15823](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15823))
* Check for nans in unet only once, after all steps have been completed
* Added pption to run torch profiler for image generation
### Bug Fixes:
* Fix for grids without comprehensive infotexts ([#15958](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15958))
* feat: lora partial update precede full update ([#15943](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15943))
* Fix bug where file extension had an extra '.' under some circumstances ([#15893](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15893))
* Fix corrupt model initial load loop ([#15600](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15600))
* Allow old sampler names in API ([#15656](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15656))
* more old sampler scheduler compatibility ([#15681](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15681))
* Fix Hypertile xyz ([#15831](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15831))
* XYZ CSV skipinitialspace ([#15832](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15832))
* fix soft inpainting on mps and xpu, torch_utils.float64 ([#15815](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15815))
* fix extention update when not on main branch ([#15797](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15797))
* update pickle safe filenames
* use relative path for webui-assets css ([#15757](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15757))
* When creating a virtual environment, upgrade pip in webui.bat/webui.sh ([#15750](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15750))
* Fix AttributeError ([#15738](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15738))
* use script_path for webui root in launch_utils ([#15705](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15705))
* fix extra batch mode P Transparency ([#15664](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15664))
* use gradio theme colors in css ([#15680](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15680))
* Fix dragging text within prompt input ([#15657](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15657))
* Add correct mimetype for .mjs files ([#15654](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15654))
* QOL Items - handle metadata issues more cleanly for SD models, Loras and embeddings ([#15632](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15632))
* replace wsl-open with wslpath and explorer.exe ([#15968](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15968))
* Fix SDXL Inpaint ([#15976](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15976))
* multi size grid ([#15988](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15988))
* fix Replace preview ([#16118](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16118))
* Possible fix of wrong scale in weight decomposition ([#16151](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16151))
* Ensure use of python from venv on Mac and Linux ([#16116](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16116))
* Prioritize python3.10 over python3 if both are available on Linux and Mac (with fallback) ([#16092](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16092))
* stoping generation extras ([#16085](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16085))
* Fix SD2 loading ([#16078](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16078), [#16079](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16079))
* fix infotext Lora hashes for hires fix different lora ([#16062](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16062))
* Fix sampler scheduler autocorrection warning ([#16054](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16054))
* fix ui flashing on reloading and fast scrollong ([#16153](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16153))
* fix upscale logic ([#16239](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16239))
* [bug] do not break progressbar on non-job actions (add wrap_gradio_call_no_job) ([#16202](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16202))
* fix OSError: cannot write mode P as JPEG ([#16194](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16194))
### Other:
* fix changelog #15883 -> #15882 ([#15907](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15907))
* ReloadUI backgroundColor --background-fill-primary ([#15864](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15864))
* Use different torch versions for Intel and ARM Macs ([#15851](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15851))
* XYZ override rework ([#15836](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15836))
* scroll extensions table on overflow ([#15830](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15830))
* img2img batch upload method ([#15817](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15817))
* chore: sync v1.8.0 packages according to changelog ([#15783](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15783))
* Add AVIF MIME type support to mimetype definitions ([#15739](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15739))
* Update imageviewer.js ([#15730](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15730))
* no-referrer ([#15641](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15641))
* .gitignore trace.json ([#15980](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15980))
* Bump spandrel to 0.3.4 ([#16144](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16144))
* Defunct --max-batch-count ([#16119](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16119))
* docs: update bug_report.yml ([#16102](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16102))
* Maintaining Project Compatibility for Python 3.9 Users Without Upgrade Requirements. ([#16088](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16088), [#16169](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16169), [#16192](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16192))
* Update torch for ARM Macs to 2.3.1 ([#16059](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16059))
* remove deprecated setting dont_fix_second_order_samplers_schedule ([#16061](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16061))
* chore: fix typos ([#16060](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16060))
* shlex.join launch args in console log ([#16170](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16170))
* activate venv .bat ([#16231](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16231))
* add ids to the resize tabs in img2img ([#16218](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16218))
* update installation guide linux ([#16178](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16178))
* Robust sysinfo ([#16173](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16173))
* do not send image size on paste inpaint ([#16180](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16180))
* Fix noisy DS_Store files for MacOS ([#16166](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/16166))
## 1.9.4
### Bug Fixes:
* pin setuptools version to fix the startup error ([#15882](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15882))
## 1.9.3
### Bug Fixes:
* fix get_crop_region_v2 ([#15594](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15594))
## 1.9.2
### Extensions and API:
* restore 1.8.0-style naming of scripts
## 1.9.1
### Minor:
* Add avif support ([#15582](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15582))
* Add filename patterns: `[sampler_scheduler]` and `[scheduler]` ([#15581](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15581))
### Extensions and API:
* undo adding scripts to sys.modules
* Add schedulers API endpoint ([#15577](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15577))
* Remove API upscaling factor limits ([#15560](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15560))
### Bug Fixes:
* Fix images do not match / Coordinate 'right' is less than 'left' ([#15534](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15534))
* fix: remove_callbacks_for_function should also remove from the ordered map ([#15533](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15533))
* fix x1 upscalers ([#15555](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15555))
* Fix cls.__module__ value in extension script ([#15532](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15532))
* fix typo in function call (eror -> error) ([#15531](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15531))
### Other:
* Hide 'No Image data blocks found.' message ([#15567](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15567))
* Allow webui.sh to be runnable from arbitrary directories containing a .git file ([#15561](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15561))
* Compatibility with Debian 11, Fedora 34+ and openSUSE 15.4+ ([#15544](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15544))
* numpy DeprecationWarning product -> prod ([#15547](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15547))
* get_crop_region_v2 ([#15583](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15583), [#15587](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15587))
## 1.9.0
### Features:
* Make refiner switchover based on model timesteps instead of sampling steps ([#14978](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14978))
* add an option to have old-style directory view instead of tree view; stylistic changes for extra network sorting/search controls
* add UI for reordering callbacks, support for specifying callback order in extension metadata ([#15205](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15205))
* Sgm uniform scheduler for SDXL-Lightning models ([#15325](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15325))
* Scheduler selection in main UI ([#15333](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15333), [#15361](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15361), [#15394](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15394))
### Minor:
* "open images directory" button now opens the actual dir ([#14947](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14947))
* Support inference with LyCORIS BOFT networks ([#14871](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14871), [#14973](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14973))
* make extra network card description plaintext by default, with an option to re-enable HTML as it was
* resize handle for extra networks ([#15041](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15041))
* cmd args: `--unix-filenames-sanitization` and `--filenames-max-length` ([#15031](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15031))
* show extra networks parameters in HTML table rather than raw JSON ([#15131](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15131))
* Add DoRA (weight-decompose) support for LoRA/LoHa/LoKr ([#15160](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15160), [#15283](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15283))
* Add '--no-prompt-history' cmd args for disable last generation prompt history ([#15189](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15189))
* update preview on Replace Preview ([#15201](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15201))
* only fetch updates for extensions' active git branches ([#15233](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15233))
* put upscale postprocessing UI into an accordion ([#15223](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15223))
* Support dragdrop for URLs to read infotext ([#15262](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15262))
* use diskcache library for caching ([#15287](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15287), [#15299](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15299))
* Allow PNG-RGBA for Extras Tab ([#15334](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15334))
* Support cover images embedded in safetensors metadata ([#15319](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15319))
* faster interrupt when using NN upscale ([#15380](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15380))
* Extras upscaler: an input field to limit maximul side length for the output image ([#15293](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15293), [#15415](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15415), [#15417](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15417), [#15425](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15425))
* add an option to hide postprocessing options in Extras tab
### Extensions and API:
* ResizeHandleRow - allow overriden column scale parametr ([#15004](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15004))
* call script_callbacks.ui_settings_callback earlier; fix extra-options-section built-in extension killing the ui if using a setting that doesn't exist
* make it possible to use zoom.js outside webui context ([#15286](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15286), [#15288](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15288))
* allow variants for extension name in metadata.ini ([#15290](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15290))
* make reloading UI scripts optional when doing Reload UI, and off by default
* put request: gr.Request at start of img2img function similar to txt2img
* open_folder as util ([#15442](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15442))
* make it possible to import extensions' script files as `import scripts.<filename>` ([#15423](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15423))
### Performance:
* performance optimization for extra networks HTML pages
* optimization for extra networks filtering
* optimization for extra networks sorting
### Bug Fixes:
* prevent escape button causing an interrupt when no generation has been made yet
* [bug] avoid doble upscaling in inpaint ([#14966](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14966))
* possible fix for reload button not appearing in some cases for extra networks.
* fix: the `split_threshold` parameter does not work when running Split oversized images ([#15006](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15006))
* Fix resize-handle visability for vertical layout (mobile) ([#15010](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15010))
* register_tmp_file also for mtime ([#15012](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15012))
* Protect alphas_cumprod during refiner switchover ([#14979](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14979))
* Fix EXIF orientation in API image loading ([#15062](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15062))
* Only override emphasis if actually used in prompt ([#15141](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15141))
* Fix emphasis infotext missing from `params.txt` ([#15142](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15142))
* fix extract_style_text_from_prompt #15132 ([#15135](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15135))
* Fix Soft Inpaint for AnimateDiff ([#15148](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15148))
* edit-attention: deselect surrounding whitespace ([#15178](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15178))
* chore: fix font not loaded ([#15183](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15183))
* use natural sort in extra networks when ordering by path
* Fix built-in lora system bugs caused by torch.nn.MultiheadAttention ([#15190](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15190))
* Avoid error from None in get_learned_conditioning ([#15191](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15191))
* Add entry to MassFileLister after writing metadata ([#15199](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15199))
* fix issue with Styles when Hires prompt is used ([#15269](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15269), [#15276](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15276))
* Strip comments from hires fix prompt ([#15263](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15263))
* Make imageviewer event listeners browser consistent ([#15261](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15261))
* Fix AttributeError in OFT when trying to get MultiheadAttention weight ([#15260](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15260))
* Add missing .mean() back ([#15239](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15239))
* fix "Restore progress" button ([#15221](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15221))
* fix ui-config for InputAccordion [custom_script_source] ([#15231](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15231))
* handle 0 wheel deltaY ([#15268](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15268))
* prevent alt menu for firefox ([#15267](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15267))
* fix: fix syntax errors ([#15179](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15179))
* restore outputs path ([#15307](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15307))
* Escape btn_copy_path filename ([#15316](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15316))
* Fix extra networks buttons when filename contains an apostrophe ([#15331](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15331))
* escape brackets in lora random prompt generator ([#15343](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15343))
* fix: Python version check for PyTorch installation compatibility ([#15390](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15390))
* fix typo in call_queue.py ([#15386](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15386))
* fix: when find already_loaded model, remove loaded by array index ([#15382](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15382))
* minor bug fix of sd model memory management ([#15350](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15350))
* Fix CodeFormer weight ([#15414](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15414))
* Fix: Remove script callbacks in ordered_callbacks_map ([#15428](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15428))
* fix limited file write (thanks, Sylwia)
* Fix extra-single-image API not doing upscale failed ([#15465](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15465))
* error handling paste_field callables ([#15470](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15470))
### Hardware:
* Add training support and change lspci for Ascend NPU ([#14981](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14981))
* Update to ROCm5.7 and PyTorch ([#14820](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14820))
* Better workaround for Navi1, removing --pre for Navi3 ([#15224](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15224))
* Ascend NPU wiki page ([#15228](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15228))
### Other:
* Update comment for Pad prompt/negative prompt v0 to add a warning about truncation, make it override the v1 implementation
* support resizable columns for touch (tablets) ([#15002](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15002))
* Fix #14591 using translated content to do categories mapping ([#14995](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14995))
* Use `absolute` path for normalized filepath ([#15035](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15035))
* resizeHandle handle double tap ([#15065](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15065))
* --dat-models-path cmd flag ([#15039](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15039))
* Add a direct link to the binary release ([#15059](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15059))
* upscaler_utils: Reduce logging ([#15084](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15084))
* Fix various typos with crate-ci/typos ([#15116](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15116))
* fix_jpeg_live_preview ([#15102](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15102))
* [alternative fix] can't load webui if selected wrong extra option in ui ([#15121](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15121))
* Error handling for unsupported transparency ([#14958](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14958))
* Add model description to searched terms ([#15198](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15198))
* bump action version ([#15272](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15272))
* PEP 604 annotations ([#15259](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15259))
* Automatically Set the Scale by value when user selects an Upscale Model ([#15244](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15244))
* move postprocessing-for-training into builtin extensions ([#15222](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15222))
* type hinting in shared.py ([#15211](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15211))
* update ruff to 0.3.3
* Update pytorch lightning utilities ([#15310](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15310))
* Add Size as an XYZ Grid option ([#15354](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15354))
* Use HF_ENDPOINT variable for HuggingFace domain with default ([#15443](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15443))
* re-add update_file_entry ([#15446](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15446))
* create_infotext allow index and callable, re-work Hires prompt infotext ([#15460](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15460))
* update restricted_opts to include more options for --hide-ui-dir-config ([#15492](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15492))
## 1.8.0
### Features:
* Update torch to version 2.1.2
* Soft Inpainting ([#14208](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14208))
* FP8 support ([#14031](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14031), [#14327](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14327))
* Support for SDXL-Inpaint Model ([#14390](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14390))
* Use Spandrel for upscaling and face restoration architectures ([#14425](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14425), [#14467](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14467), [#14473](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14473), [#14474](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14474), [#14477](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14477), [#14476](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14476), [#14484](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14484), [#14500](https://github.com/AUTOMATIC1111/stable-difusion-webui/pull/14500), [#14501](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14501), [#14504](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14504), [#14524](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14524), [#14809](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14809))
* Automatic backwards version compatibility (when loading infotexts from old images with program version specified, will add compatibility settings)
* Implement zero terminal SNR noise schedule option (**[SEED BREAKING CHANGE](https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Seed-breaking-changes#180-dev-170-225-2024-01-01---zero-terminal-snr-noise-schedule-option)**, [#14145](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14145), [#14979](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14979))
* Add a [✨] button to run hires fix on selected image in the gallery (with help from [#14598](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14598), [#14626](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14626), [#14728](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14728))
* [Separate assets repository](https://github.com/AUTOMATIC1111/stable-diffusion-webui-assets); serve fonts locally rather than from google's servers
* Official LCM Sampler Support ([#14583](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14583))
* Add support for DAT upscaler models ([#14690](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14690), [#15039](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15039))
* Extra Networks Tree View ([#14588](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14588), [#14900](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14900))
* NPU Support ([#14801](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14801))
* Prompt comments support
### Minor:
* Allow pasting in WIDTHxHEIGHT strings into the width/height fields ([#14296](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14296))
* add option: Live preview in full page image viewer ([#14230](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14230), [#14307](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14307))
* Add keyboard shortcuts for generate/skip/interrupt ([#14269](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14269))
* Better TCMALLOC support on different platforms ([#14227](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14227), [#14883](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14883), [#14910](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14910))
* Lora not found warning ([#14464](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14464))
* Adding negative prompts to Loras in extra networks ([#14475](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14475))
* xyz_grid: allow varying the seed along an axis separate from axis options ([#12180](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12180))
* option to convert VAE to bfloat16 (implementation of [#9295](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/9295))
* Better IPEX support ([#14229](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14229), [#14353](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14353), [#14559](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14559), [#14562](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14562), [#14597](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14597))
* Option to interrupt after current generation rather than immediately ([#13653](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/13653), [#14659](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14659))
* Fullscreen Preview control fading/disable ([#14291](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14291))
* Finer settings freezing control ([#13789](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/13789))
* Increase Upscaler Limits ([#14589](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14589))
* Adjust brush size with hotkeys ([#14638](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14638))
* Add checkpoint info to csv log file when saving images ([#14663](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14663))
* Make more columns resizable ([#14740](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14740), [#14884](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14884))
* Add an option to not overlay original image for inpainting for #14727
* Add Pad conds v0 option to support same generation with DDIM as before 1.6.0
* Add "Interrupting..." placeholder.
* Button for refresh extensions list ([#14857](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14857))
* Add an option to disable normalization after calculating emphasis. ([#14874](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14874))
* When counting tokens, also include enabled styles (can be disabled in settings to revert to previous behavior)
* Configuration for the [📂] button for image gallery ([#14947](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14947))
* Support inference with LyCORIS BOFT networks ([#14871](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14871), [#14973](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14973))
* support resizable columns for touch (tablets) ([#15002](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15002))
### Extensions and API:
* Removed packages from requirements: basicsr, gfpgan, realesrgan; as well as their dependencies: absl-py, addict, beautifulsoup4, future, gdown, grpcio, importlib-metadata, lmdb, lpips, Markdown, platformdirs, PySocks, soupsieve, tb-nightly, tensorboard-data-server, tomli, Werkzeug, yapf, zipp, soupsieve
* Enable task ids for API ([#14314](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14314))
* add override_settings support for infotext API
* rename generation_parameters_copypaste module to infotext_utils
* prevent crash due to Script __init__ exception ([#14407](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14407))
* Bump numpy to 1.26.2 ([#14471](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14471))
* Add utility to inspect a model's dtype/device ([#14478](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14478))
* Implement general forward method for all method in built-in lora ext ([#14547](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14547))
* Execute model_loaded_callback after moving to target device ([#14563](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14563))
* Add self to CFGDenoiserParams ([#14573](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14573))
* Allow TLS with API only mode (--nowebui) ([#14593](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14593))
* New callback: postprocess_image_after_composite ([#14657](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14657))
* modules/api/api.py: add api endpoint to refresh embeddings list ([#14715](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14715))
* set_named_arg ([#14773](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14773))
* add before_token_counter callback and use it for prompt comments
* ResizeHandleRow - allow overridden column scale parameter ([#15004](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15004))
### Performance:
* Massive performance improvement for extra networks directories with a huge number of files in them in an attempt to tackle #14507 ([#14528](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14528))
* Reduce unnecessary re-indexing extra networks directory ([#14512](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14512))
* Avoid unnecessary `isfile`/`exists` calls ([#14527](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14527))
### Bug Fixes:
* fix multiple bugs related to styles multi-file support ([#14203](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14203), [#14276](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14276), [#14707](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14707))
* Lora fixes ([#14300](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14300), [#14237](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14237), [#14546](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14546), [#14726](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14726))
* Re-add setting lost as part of e294e46 ([#14266](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14266))
* fix extras caption BLIP ([#14330](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14330))
* include infotext into saved init image for img2img ([#14452](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14452))
* xyz grid handle axis_type is None ([#14394](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14394))
* Update Added (Fixed) IPV6 Functionality When there is No Webui Argument Passed webui.py ([#14354](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14354))
* fix API thread safe issues of txt2img and img2img ([#14421](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14421))
* handle selectable script_index is None ([#14487](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14487))
* handle config.json failed to load ([#14525](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14525), [#14767](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14767))
* paste infotext cast int as float ([#14523](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14523))
* Ensure GRADIO_ANALYTICS_ENABLED is set early enough ([#14537](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14537))
* Fix logging configuration again ([#14538](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14538))
* Handle CondFunc exception when resolving attributes ([#14560](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14560))
* Fix extras big batch crashes ([#14699](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14699))
* Fix using wrong model caused by alias ([#14655](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14655))
* Add # to the invalid_filename_chars list ([#14640](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14640))
* Fix extension check for requirements ([#14639](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14639))
* Fix tab indexes are reset after restart UI ([#14637](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14637))
* Fix nested manual cast ([#14689](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14689))
* Keep postprocessing upscale selected tab after restart ([#14702](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14702))
* XYZ grid: filter out blank vals when axis is int or float type (like int axis seed) ([#14754](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14754))
* fix CLIP Interrogator topN regex ([#14775](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14775))
* Fix dtype error in MHA layer/change dtype checking mechanism for manual cast ([#14791](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14791))
* catch load style.csv error ([#14814](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14814))
* fix error when editing extra networks card
* fix extra networks metadata failing to work properly when you create the .json file with metadata for the first time.
* util.walk_files extensions case insensitive ([#14879](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14879))
* if extensions page not loaded, prevent apply ([#14873](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14873))
* call the right function for token counter in img2img
* Fix the bugs that search/reload will disappear when using other ExtraNetworks extensions ([#14939](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14939))
* Gracefully handle mtime read exception from cache ([#14933](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14933))
* Only trigger interrupt on `Esc` when interrupt button visible ([#14932](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14932))
* Disable prompt token counters option actually disables token counting rather than just hiding results.
* avoid double upscaling in inpaint ([#14966](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14966))
* Fix #14591 using translated content to do categories mapping ([#14995](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14995))
* fix: the `split_threshold` parameter does not work when running Split oversized images ([#15006](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15006))
* Fix resize-handle for mobile ([#15010](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15010), [#15065](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15065))
### Other:
* Assign id for "extra_options". Replace numeric field with slider. ([#14270](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14270))
* change state dict comparison to ref compare ([#14216](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14216))
* Bump torch-rocm to 5.6/5.7 ([#14293](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14293))
* Base output path off data path ([#14446](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14446))
* reorder training preprocessing modules in extras tab ([#14367](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14367))
* Remove `cleanup_models` code ([#14472](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14472))
* only rewrite ui-config when there is change ([#14352](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14352))
* Fix lint issue from 501993eb ([#14495](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14495))
* Update README.md ([#14548](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14548))
* hires button, fix seeds ()
* Logging: set formatter correctly for fallback logger too ([#14618](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14618))
* Read generation info from infotexts rather than json for internal needs (save, extract seed from generated pic) ([#14645](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14645))
* improve get_crop_region ([#14709](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14709))
* Bump safetensors' version to 0.4.2 ([#14782](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14782))
* add tooltip create_submit_box ([#14803](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14803))
* extensions tab table row hover highlight ([#14885](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14885))
* Always add timestamp to displayed image ([#14890](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14890))
* Added core.filemode=false so doesn't track changes in file permission… ([#14930](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14930))
* Normalize command-line argument paths ([#14934](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14934), [#15035](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15035))
* Use original App Title in progress bar ([#14916](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14916))
* register_tmp_file also for mtime ([#15012](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/15012))
## 1.7.0
### Features:
@@ -40,7 +444,8 @@
* infotext updates: add option to disregard certain infotext fields, add option to not include VAE in infotext, add explanation to infotext settings page, move some options to infotext settings page
* add FP32 fallback support on sd_vae_approx ([#14046](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14046))
* support XYZ scripts / split hires path from unet ([#14126](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14126))
* allow use of mutiple styles csv files ([#14125](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14125))
* allow use of multiple styles csv files ([#14125](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/14125))
* make extra network card description plaintext by default, with an option (Treat card description as HTML) to re-enable HTML as it was (originally by [#13241](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/13241))
### Extensions and API:
* update gradio to 3.41.2
@@ -176,7 +581,7 @@
* new samplers: Restart, DPM++ 2M SDE Exponential, DPM++ 2M SDE Heun, DPM++ 2M SDE Heun Karras, DPM++ 2M SDE Heun Exponential, DPM++ 3M SDE, DPM++ 3M SDE Karras, DPM++ 3M SDE Exponential ([#12300](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12300), [#12519](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12519), [#12542](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12542))
* rework DDIM, PLMS, UniPC to use CFG denoiser same as in k-diffusion samplers:
* makes all of them work with img2img
* makes prompt composition posssible (AND)
* makes prompt composition possible (AND)
* makes them available for SDXL
* always show extra networks tabs in the UI ([#11808](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/11808))
* use less RAM when creating models ([#11958](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/11958), [#12599](https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/12599))
@@ -352,7 +757,7 @@
* user metadata system for custom networks
* extended Lora metadata editor: set activation text, default weight, view tags, training info
* Lora extension rework to include other types of networks (all that were previously handled by LyCORIS extension)
* show github stars for extenstions
* show github stars for extensions
* img2img batch mode can read extra stuff from png info
* img2img batch works with subdirectories
* hotkeys to move prompt elements: alt+left/right
@@ -571,7 +976,7 @@
* do not wait for Stable Diffusion model to load at startup
* add filename patterns: `[denoising]`
* directory hiding for extra networks: dirs starting with `.` will hide their cards on extra network tabs unless specifically searched for
* LoRA: for the `<...>` text in prompt, use name of LoRA that is in the metdata of the file, if present, instead of filename (both can be used to activate LoRA)
* LoRA: for the `<...>` text in prompt, use name of LoRA that is in the metadata of the file, if present, instead of filename (both can be used to activate LoRA)
* LoRA: read infotext params from kohya-ss's extension parameters if they are present and if his extension is not active
* LoRA: fix some LoRAs not working (ones that have 3x3 convolution layer)
* LoRA: add an option to use old method of applying LoRAs (producing same results as with kohya-ss)
@@ -601,7 +1006,7 @@
* fix gamepad navigation
* make the lightbox fullscreen image function properly
* fix squished thumbnails in extras tab
* keep "search" filter for extra networks when user refreshes the tab (previously it showed everthing after you refreshed)
* keep "search" filter for extra networks when user refreshes the tab (previously it showed everything after you refreshed)
* fix webui showing the same image if you configure the generation to always save results into same file
* fix bug with upscalers not working properly
* fix MPS on PyTorch 2.0.1, Intel Macs
@@ -619,7 +1024,7 @@
* switch to PyTorch 2.0.0 (except for AMD GPUs)
* visual improvements to custom code scripts
* add filename patterns: `[clip_skip]`, `[hasprompt<>]`, `[batch_number]`, `[generation_number]`
* add support for saving init images in img2img, and record their hashes in infotext for reproducability
* add support for saving init images in img2img, and record their hashes in infotext for reproducibility
* automatically select current word when adjusting weight with ctrl+up/down
* add dropdowns for X/Y/Z plot
* add setting: Stable Diffusion/Random number generator source: makes it possible to make images generated from a given manual seed consistent across different GPUs
README.md
+666 -5
View File
@@ -1,13 +1,11 @@
# Under Construction
**Oops we are upgrading the repo now ... Please come back several hours later ...**
# Stable Diffusion WebUI Forge
Stable Diffusion WebUI Forge is a platform on top of [Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) (based on [Gradio](https://www.gradio.app/)) to make development easier, optimize resource management, speed up inference, and study experimental features.
The name "Forge" is inspired from "Minecraft Forge". This project is aimed at becoming SD WebUI's Forge.
This repo will undergo major change very recently. See also the [Announcement](https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/801).
# Installing Forge
If you are proficient in Git and you want to install Forge as another branch of SD-WebUI, please see [here](https://github.com/continue-revolution/sd-webui-animatediff/blob/forge/master/docs/how-to-use.md#you-have-a1111-and-you-know-git). In this way, you can reuse all SD checkpoints and all extensions you installed previously in your OG SD-WebUI, but you should know what you are doing.
@@ -24,6 +22,669 @@ Note that running `update.bat` is important, otherwise you may be using a previo
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/c49bd60d-82bd-4086-9859-88d472582b94)
### Previous Versions
## Previous Versions
You can download previous versions [here](https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/849).
# Screenshots of Comparison
I tested with several devices, and this is a typical result from 8GB VRAM (3070ti laptop) with SDXL.
**This is original WebUI:**
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/16893937-9ed9-4f8e-b960-70cd5d1e288f)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/7bbc16fe-64ef-49e2-a595-d91bb658bd94)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/de1747fd-47bc-482d-a5c6-0728dd475943)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/96e5e171-2d74-41ba-9dcc-11bf68be7e16)
(average about 7.4GB/8GB, peak at about 7.9GB/8GB)
**This is WebUI Forge:**
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/ca5e05ed-bd86-4ced-8662-f41034648e8c)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/3629ee36-4a99-4d9b-b371-12efb260a283)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/6d13ebb7-c30d-4aa8-9242-c0b5a1af8c95)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/c4f723c3-6ea7-4539-980b-0708ed2a69aa)
(average and peak are all 6.3GB/8GB)
You can see that Forge does not change WebUI results. Installing Forge is not a seed breaking change.
Forge can perfectly keep WebUI unchanged even for most complicated prompts like `fantasy landscape with a [mountain:lake:0.25] and [an oak:a christmas tree:0.75][ in foreground::0.6][ in background:0.25] [shoddy:masterful:0.5]`.
All your previous works still work in Forge!
# Forge Backend
Forge backend removes all WebUI's codes related to resource management and reworked everything. All previous CMD flags like `medvram, lowvram, medvram-sdxl, precision full, no half, no half vae, attention_xxx, upcast unet`, ... are all **REMOVED**. Adding these flags will not cause error but they will not do anything now. **We highly encourage Forge users to remove all cmd flags and let Forge to decide how to load models.**
Without any cmd flag, Forge can run SDXL with 4GB vram and SD1.5 with 2GB vram.
**Some flags that you may still pay attention to:**
1. `--always-offload-from-vram` (This flag will make things **slower** but less risky). This option will let Forge always unload models from VRAM. This can be useful if you use multiple software together and want Forge to use less VRAM and give some VRAM to other software, or when you are using some old extensions that will compete vram with Forge, or (very rarely) when you get OOM.
2. `--cuda-malloc` (This flag will make things **faster** but more risky). This will ask pytorch to use *cudaMallocAsync* for tensor malloc. On some profilers I can observe performance gain at millisecond level, but the real speed up on most my devices are often unnoticed (about or less than 0.1 second per image). This cannot be set as default because many users reported issues that the async malloc will crash the program. Users need to enable this cmd flag at their own risk.
3. `--cuda-stream` (This flag will make things **faster** but more risky). This will use pytorch CUDA streams (a special type of thread on GPU) to move models and compute tensors simultaneously. This can almost eliminate all model moving time, and speed up SDXL on 30XX/40XX devices with small VRAM (eg, RTX 4050 6GB, RTX 3060 Laptop 6GB, etc) by about 15\% to 25\%. However, this unfortunately cannot be set as default because I observe higher possibility of pure black images (Nan outputs) on 2060, and higher chance of OOM on 1080 and 2060. When the resolution is large, there is a chance that the computation time of one single attention layer is longer than the time for moving entire model to GPU. When that happens, the next attention layer will OOM since the GPU is filled with the entire model, and no remaining space is available for computing another attention layer. Most overhead detecting methods are not robust enough to be reliable on old devices (in my tests). Users need to enable this cmd flag at their own risk.
4. `--pin-shared-memory` (This flag will make things **faster** but more risky). Effective only when used together with `--cuda-stream`. This will offload modules to Shared GPU Memory instead of system RAM when offloading models. On some 30XX/40XX devices with small VRAM (eg, RTX 4050 6GB, RTX 3060 Laptop 6GB, etc), I can observe significant (at least 20\%) speed-up for SDXL. However, this unfortunately cannot be set as default because the OOM of Shared GPU Memory is a much more severe problem than common GPU memory OOM. Pytorch does not provide any robust method to unload or detect Shared GPU Memory. Once the Shared GPU Memory OOM, the entire program will crash (observed with SDXL on GTX 1060/1050/1066), and there is no dynamic method to prevent or recover from the crash. Users need to enable this cmd flag at their own risk.
If you really want to play with cmd flags, you can additionally control the GPU with:
(extreme VRAM cases)
--always-gpu
--always-cpu
(rare attention cases)
--attention-split
--attention-quad
--attention-pytorch
--disable-xformers
--disable-attention-upcast
(float point type)
--all-in-fp32
--all-in-fp16
--unet-in-bf16
--unet-in-fp16
--unet-in-fp8-e4m3fn
--unet-in-fp8-e5m2
--vae-in-fp16
--vae-in-fp32
--vae-in-bf16
--clip-in-fp8-e4m3fn
--clip-in-fp8-e5m2
--clip-in-fp16
--clip-in-fp32
(rare platforms)
--directml
--disable-ipex-hijack
--pytorch-deterministic
Again, Forge do not recommend users to use any cmd flags unless you are very sure that you really need these.
# UNet Patcher
Note that [Forge does not use any other software as backend](https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/169). The full name of the backend is `Stable Diffusion WebUI with Forge backend`, or for simplicity, the `Forge backend`. The API and python symbols are made similar to previous software only for reducing the learning cost of developers.
Now developing an extension is super simple. We finally have a patchable UNet.
Below is using one single file with 80 lines of codes to support FreeU:
`extensions-builtin/sd_forge_freeu/scripts/forge_freeu.py`
```python
import torch
import gradio as gr
from modules import scripts
def Fourier_filter(x, threshold, scale):
x_freq = torch.fft.fftn(x.float(), dim=(-2, -1))
x_freq = torch.fft.fftshift(x_freq, dim=(-2, -1))
B, C, H, W = x_freq.shape
mask = torch.ones((B, C, H, W), device=x.device)
crow, ccol = H // 2, W //2
mask[..., crow - threshold:crow + threshold, ccol - threshold:ccol + threshold] = scale
x_freq = x_freq * mask
x_freq = torch.fft.ifftshift(x_freq, dim=(-2, -1))
x_filtered = torch.fft.ifftn(x_freq, dim=(-2, -1)).real
return x_filtered.to(x.dtype)
def set_freeu_v2_patch(model, b1, b2, s1, s2):
model_channels = model.model.model_config.unet_config["model_channels"]
scale_dict = {model_channels * 4: (b1, s1), model_channels * 2: (b2, s2)}
def output_block_patch(h, hsp, *args, **kwargs):
scale = scale_dict.get(h.shape[1], None)
if scale is not None:
hidden_mean = h.mean(1).unsqueeze(1)
B = hidden_mean.shape[0]
hidden_max, _ = torch.max(hidden_mean.view(B, -1), dim=-1, keepdim=True)
hidden_min, _ = torch.min(hidden_mean.view(B, -1), dim=-1, keepdim=True)
hidden_mean = (hidden_mean - hidden_min.unsqueeze(2).unsqueeze(3)) / \
(hidden_max - hidden_min).unsqueeze(2).unsqueeze(3)
h[:, :h.shape[1] // 2] = h[:, :h.shape[1] // 2] * ((scale[0] - 1) * hidden_mean + 1)
hsp = Fourier_filter(hsp, threshold=1, scale=scale[1])
return h, hsp
m = model.clone()
m.set_model_output_block_patch(output_block_patch)
return m
class FreeUForForge(scripts.Script):
def title(self):
return "FreeU Integrated"
def show(self, is_img2img):
# make this extension visible in both txt2img and img2img tab.
return scripts.AlwaysVisible
def ui(self, *args, **kwargs):
with gr.Accordion(open=False, label=self.title()):
freeu_enabled = gr.Checkbox(label='Enabled', value=False)
freeu_b1 = gr.Slider(label='B1', minimum=0, maximum=2, step=0.01, value=1.01)
freeu_b2 = gr.Slider(label='B2', minimum=0, maximum=2, step=0.01, value=1.02)
freeu_s1 = gr.Slider(label='S1', minimum=0, maximum=4, step=0.01, value=0.99)
freeu_s2 = gr.Slider(label='S2', minimum=0, maximum=4, step=0.01, value=0.95)
return freeu_enabled, freeu_b1, freeu_b2, freeu_s1, freeu_s2
def process_before_every_sampling(self, p, *script_args, **kwargs):
# This will be called before every sampling.
# If you use highres fix, this will be called twice.
freeu_enabled, freeu_b1, freeu_b2, freeu_s1, freeu_s2 = script_args
if not freeu_enabled:
return
unet = p.sd_model.forge_objects.unet
unet = set_freeu_v2_patch(unet, freeu_b1, freeu_b2, freeu_s1, freeu_s2)
p.sd_model.forge_objects.unet = unet
# Below codes will add some logs to the texts below the image outputs on UI.
# The extra_generation_params does not influence results.
p.extra_generation_params.update(dict(
freeu_enabled=freeu_enabled,
freeu_b1=freeu_b1,
freeu_b2=freeu_b2,
freeu_s1=freeu_s1,
freeu_s2=freeu_s2,
))
return
```
It looks like this:
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/277bac6e-5ea7-4bff-b71a-e55a60cfc03c)
Similar components like HyperTile, KohyaHighResFix, SAG, can all be implemented within 100 lines of codes (see also the codes).
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/06472b03-b833-4816-ab47-70712ac024d3)
ControlNets can finally be called by different extensions.
Implementing Stable Video Diffusion and Zero123 are also super simple now (see also the codes).
*Stable Video Diffusion:*
`extensions-builtin/sd_forge_svd/scripts/forge_svd.py`
```python
import torch
import gradio as gr
import os
import pathlib
from modules import script_callbacks
from modules.paths import models_path
from modules.ui_common import ToolButton, refresh_symbol
from modules import shared
from modules_forge.forge_util import numpy_to_pytorch, pytorch_to_numpy
from ldm_patched.modules.sd import load_checkpoint_guess_config
from ldm_patched.contrib.external_video_model import VideoLinearCFGGuidance, SVD_img2vid_Conditioning
from ldm_patched.contrib.external import KSampler, VAEDecode
opVideoLinearCFGGuidance = VideoLinearCFGGuidance()
opSVD_img2vid_Conditioning = SVD_img2vid_Conditioning()
opKSampler = KSampler()
opVAEDecode = VAEDecode()
svd_root = os.path.join(models_path, 'svd')
os.makedirs(svd_root, exist_ok=True)
svd_filenames = []
def update_svd_filenames():
global svd_filenames
svd_filenames = [
pathlib.Path(x).name for x in
shared.walk_files(svd_root, allowed_extensions=[".pt", ".ckpt", ".safetensors"])
]
return svd_filenames
@torch.inference_mode()
@torch.no_grad()
def predict(filename, width, height, video_frames, motion_bucket_id, fps, augmentation_level,
sampling_seed, sampling_steps, sampling_cfg, sampling_sampler_name, sampling_scheduler,
sampling_denoise, guidance_min_cfg, input_image):
filename = os.path.join(svd_root, filename)
model_raw, _, vae, clip_vision = \
load_checkpoint_guess_config(filename, output_vae=True, output_clip=False, output_clipvision=True)
model = opVideoLinearCFGGuidance.patch(model_raw, guidance_min_cfg)[0]
init_image = numpy_to_pytorch(input_image)
positive, negative, latent_image = opSVD_img2vid_Conditioning.encode(
clip_vision, init_image, vae, width, height, video_frames, motion_bucket_id, fps, augmentation_level)
output_latent = opKSampler.sample(model, sampling_seed, sampling_steps, sampling_cfg,
sampling_sampler_name, sampling_scheduler, positive,
negative, latent_image, sampling_denoise)[0]
output_pixels = opVAEDecode.decode(vae, output_latent)[0]
outputs = pytorch_to_numpy(output_pixels)
return outputs
def on_ui_tabs():
with gr.Blocks() as svd_block:
with gr.Row():
with gr.Column():
input_image = gr.Image(label='Input Image', source='upload', type='numpy', height=400)
with gr.Row():
filename = gr.Dropdown(label="SVD Checkpoint Filename",
choices=svd_filenames,
value=svd_filenames[0] if len(svd_filenames) > 0 else None)
refresh_button = ToolButton(value=refresh_symbol, tooltip="Refresh")
refresh_button.click(
fn=lambda: gr.update(choices=update_svd_filenames),
inputs=[], outputs=filename)
width = gr.Slider(label='Width', minimum=16, maximum=8192, step=8, value=1024)
height = gr.Slider(label='Height', minimum=16, maximum=8192, step=8, value=576)
video_frames = gr.Slider(label='Video Frames', minimum=1, maximum=4096, step=1, value=14)
motion_bucket_id = gr.Slider(label='Motion Bucket Id', minimum=1, maximum=1023, step=1, value=127)
fps = gr.Slider(label='Fps', minimum=1, maximum=1024, step=1, value=6)
augmentation_level = gr.Slider(label='Augmentation Level', minimum=0.0, maximum=10.0, step=0.01,
value=0.0)
sampling_steps = gr.Slider(label='Sampling Steps', minimum=1, maximum=200, step=1, value=20)
sampling_cfg = gr.Slider(label='CFG Scale', minimum=0.0, maximum=50.0, step=0.1, value=2.5)
sampling_denoise = gr.Slider(label='Sampling Denoise', minimum=0.0, maximum=1.0, step=0.01, value=1.0)
guidance_min_cfg = gr.Slider(label='Guidance Min Cfg', minimum=0.0, maximum=100.0, step=0.5, value=1.0)
sampling_sampler_name = gr.Radio(label='Sampler Name',
choices=['euler', 'euler_ancestral', 'heun', 'heunpp2', 'dpm_2',
'dpm_2_ancestral', 'lms', 'dpm_fast', 'dpm_adaptive',
'dpmpp_2s_ancestral', 'dpmpp_sde', 'dpmpp_sde_gpu',
'dpmpp_2m', 'dpmpp_2m_sde', 'dpmpp_2m_sde_gpu',
'dpmpp_3m_sde', 'dpmpp_3m_sde_gpu', 'ddpm', 'lcm', 'ddim',
'uni_pc', 'uni_pc_bh2'], value='euler')
sampling_scheduler = gr.Radio(label='Scheduler',
choices=['normal', 'karras', 'exponential', 'sgm_uniform', 'simple',
'ddim_uniform'], value='karras')
sampling_seed = gr.Number(label='Seed', value=12345, precision=0)
generate_button = gr.Button(value="Generate")
ctrls = [filename, width, height, video_frames, motion_bucket_id, fps, augmentation_level,
sampling_seed, sampling_steps, sampling_cfg, sampling_sampler_name, sampling_scheduler,
sampling_denoise, guidance_min_cfg, input_image]
with gr.Column():
output_gallery = gr.Gallery(label='Gallery', show_label=False, object_fit='contain',
visible=True, height=1024, columns=4)
generate_button.click(predict, inputs=ctrls, outputs=[output_gallery])
return [(svd_block, "SVD", "svd")]
update_svd_filenames()
script_callbacks.on_ui_tabs(on_ui_tabs)
```
Note that although the above codes look like independent codes, they actually will automatically offload/unload any other models. For example, below is me opening webui, load SDXL, generated an image, then go to SVD, then generated image frames. You can see that the GPU memory is perfectly managed and the SDXL is moved to RAM then SVD is moved to GPU.
Note that this management is fully automatic. This makes writing extensions super simple.
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/de1a2d05-344a-44d7-bab8-9ecc0a58a8d3)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/14bcefcf-599f-42c3-bce9-3fd5e428dd91)
Similarly, Zero123:
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/7685019c-7239-47fb-9cb5-2b7b33943285)
### Write a simple ControlNet:
Below is a simple extension to have a completely independent pass of ControlNet that never conflicts any other extensions:
`extensions-builtin/sd_forge_controlnet_example/scripts/sd_forge_controlnet_example.py`
Note that this extension is hidden because it is only for developers. To see it in UI, use `--show-controlnet-example`.
The memory optimization in this example is fully automatic. You do not need to care about memory and inference speed, but you may want to cache objects if you wish.
```python
# Use --show-controlnet-example to see this extension.
import cv2
import gradio as gr
import torch
from modules import scripts
from modules.shared_cmd_options import cmd_opts
from modules_forge.shared import supported_preprocessors
from modules.modelloader import load_file_from_url
from ldm_patched.modules.controlnet import load_controlnet
from modules_forge.controlnet import apply_controlnet_advanced
from modules_forge.forge_util import numpy_to_pytorch
from modules_forge.shared import controlnet_dir
class ControlNetExampleForge(scripts.Script):
model = None
def title(self):
return "ControlNet Example for Developers"
def show(self, is_img2img):
# make this extension visible in both txt2img and img2img tab.
return scripts.AlwaysVisible
def ui(self, *args, **kwargs):
with gr.Accordion(open=False, label=self.title()):
gr.HTML('This is an example controlnet extension for developers.')
gr.HTML('You see this extension because you used --show-controlnet-example')
input_image = gr.Image(source='upload', type='numpy')
funny_slider = gr.Slider(label='This slider does nothing. It just shows you how to transfer parameters.',
minimum=0.0, maximum=1.0, value=0.5)
return input_image, funny_slider
def process(self, p, *script_args, **kwargs):
input_image, funny_slider = script_args
# This slider does nothing. It just shows you how to transfer parameters.
del funny_slider
if input_image is None:
return
# controlnet_canny_path = load_file_from_url(
# url='https://huggingface.co/lllyasviel/sd_control_collection/resolve/main/sai_xl_canny_256lora.safetensors',
# model_dir=model_dir,
# file_name='sai_xl_canny_256lora.safetensors'
# )
controlnet_canny_path = load_file_from_url(
url='https://huggingface.co/lllyasviel/fav_models/resolve/main/fav/control_v11p_sd15_canny_fp16.safetensors',
model_dir=controlnet_dir,
file_name='control_v11p_sd15_canny_fp16.safetensors'
)
print('The model [control_v11p_sd15_canny_fp16.safetensors] download finished.')
self.model = load_controlnet(controlnet_canny_path)
print('Controlnet loaded.')
return
def process_before_every_sampling(self, p, *script_args, **kwargs):
# This will be called before every sampling.
# If you use highres fix, this will be called twice.
input_image, funny_slider = script_args
if input_image is None or self.model is None:
return
B, C, H, W = kwargs['noise'].shape # latent_shape
height = H * 8
width = W * 8
batch_size = p.batch_size
preprocessor = supported_preprocessors['canny']
# detect control at certain resolution
control_image = preprocessor(
input_image, resolution=512, slider_1=100, slider_2=200, slider_3=None)
# here we just use nearest neighbour to align input shape.
# You may want crop and resize, or crop and fill, or others.
control_image = cv2.resize(
control_image, (width, height), interpolation=cv2.INTER_NEAREST)
# Output preprocessor result. Now called every sampling. Cache in your own way.
p.extra_result_images.append(control_image)
print('Preprocessor Canny finished.')
control_image_bchw = numpy_to_pytorch(control_image).movedim(-1, 1)
unet = p.sd_model.forge_objects.unet
# Unet has input, middle, output blocks, and we can give different weights
# to each layers in all blocks.
# Below is an example for stronger control in middle block.
# This is helpful for some high-res fix passes. (p.is_hr_pass)
positive_advanced_weighting = {
'input': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2],
'middle': [1.0],
'output': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2]
}
negative_advanced_weighting = {
'input': [0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95, 1.05, 1.15, 1.25],
'middle': [1.05],
'output': [0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95, 1.05, 1.15, 1.25]
}
# The advanced_frame_weighting is a weight applied to each image in a batch.
# The length of this list must be same with batch size
# For example, if batch size is 5, the below list is [0.2, 0.4, 0.6, 0.8, 1.0]
# If you view the 5 images as 5 frames in a video, this will lead to
# progressively stronger control over time.
advanced_frame_weighting = [float(i + 1) / float(batch_size) for i in range(batch_size)]
# The advanced_sigma_weighting allows you to dynamically compute control
# weights given diffusion timestep (sigma).
# For example below code can softly make beginning steps stronger than ending steps.
sigma_max = unet.model.model_sampling.sigma_max
sigma_min = unet.model.model_sampling.sigma_min
advanced_sigma_weighting = lambda s: (s - sigma_min) / (sigma_max - sigma_min)
# You can even input a tensor to mask all control injections
# The mask will be automatically resized during inference in UNet.
# The size should be B 1 H W and the H and W are not important
# because they will be resized automatically
advanced_mask_weighting = torch.ones(size=(1, 1, 512, 512))
# But in this simple example we do not use them
positive_advanced_weighting = None
negative_advanced_weighting = None
advanced_frame_weighting = None
advanced_sigma_weighting = None
advanced_mask_weighting = None
unet = apply_controlnet_advanced(unet=unet, controlnet=self.model, image_bchw=control_image_bchw,
strength=0.6, start_percent=0.0, end_percent=0.8,
positive_advanced_weighting=positive_advanced_weighting,
negative_advanced_weighting=negative_advanced_weighting,
advanced_frame_weighting=advanced_frame_weighting,
advanced_sigma_weighting=advanced_sigma_weighting,
advanced_mask_weighting=advanced_mask_weighting)
p.sd_model.forge_objects.unet = unet
# Below codes will add some logs to the texts below the image outputs on UI.
# The extra_generation_params does not influence results.
p.extra_generation_params.update(dict(
controlnet_info='You should see these texts below output images!',
))
return
# Use --show-controlnet-example to see this extension.
if not cmd_opts.show_controlnet_example:
del ControlNetExampleForge
```
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/822fa2fc-c9f4-4f58-8669-4b6680b91063)
### Add a preprocessor
Below is the full codes to add a normalbae preprocessor with perfect memory managements.
You can use arbitrary independent extensions to add a preprocessor.
Your preprocessor will be read by all other extensions using `modules_forge.shared.preprocessors`
Below codes are in `extensions-builtin\forge_preprocessor_normalbae\scripts\preprocessor_normalbae.py`
```python
from modules_forge.supported_preprocessor import Preprocessor, PreprocessorParameter
from modules_forge.shared import preprocessor_dir, add_supported_preprocessor
from modules_forge.forge_util import resize_image_with_pad
from modules.modelloader import load_file_from_url
import types
import torch
import numpy as np
from einops import rearrange
from annotator.normalbae.models.NNET import NNET
from annotator.normalbae import load_checkpoint
from torchvision import transforms
class PreprocessorNormalBae(Preprocessor):
def __init__(self):
super().__init__()
self.name = 'normalbae'
self.tags = ['NormalMap']
self.model_filename_filters = ['normal']
self.slider_resolution = PreprocessorParameter(
label='Resolution', minimum=128, maximum=2048, value=512, step=8, visible=True)
self.slider_1 = PreprocessorParameter(visible=False)
self.slider_2 = PreprocessorParameter(visible=False)
self.slider_3 = PreprocessorParameter(visible=False)
self.show_control_mode = True
self.do_not_need_model = False
self.sorting_priority = 100 # higher goes to top in the list
def load_model(self):
if self.model_patcher is not None:
return
model_path = load_file_from_url(
"https://huggingface.co/lllyasviel/Annotators/resolve/main/scannet.pt",
model_dir=preprocessor_dir)
args = types.SimpleNamespace()
args.mode = 'client'
args.architecture = 'BN'
args.pretrained = 'scannet'
args.sampling_ratio = 0.4
args.importance_ratio = 0.7
model = NNET(args)
model = load_checkpoint(model_path, model)
self.norm = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
self.model_patcher = self.setup_model_patcher(model)
def __call__(self, input_image, resolution, slider_1=None, slider_2=None, slider_3=None, **kwargs):
input_image, remove_pad = resize_image_with_pad(input_image, resolution)
self.load_model()
self.move_all_model_patchers_to_gpu()
assert input_image.ndim == 3
image_normal = input_image
with torch.no_grad():
image_normal = self.send_tensor_to_model_device(torch.from_numpy(image_normal))
image_normal = image_normal / 255.0
image_normal = rearrange(image_normal, 'h w c -> 1 c h w')
image_normal = self.norm(image_normal)
normal = self.model_patcher.model(image_normal)
normal = normal[0][-1][:, :3]
normal = ((normal + 1) * 0.5).clip(0, 1)
normal = rearrange(normal[0], 'c h w -> h w c').cpu().numpy()
normal_image = (normal * 255.0).clip(0, 255).astype(np.uint8)
return remove_pad(normal_image)
add_supported_preprocessor(PreprocessorNormalBae())
```
# New features (that are not available in original WebUI)
Thanks to Unet Patcher, many new things are possible now and supported in Forge, including SVD, Z123, masked Ip-adapter, masked controlnet, photomaker, etc.
Masked Ip-Adapter
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/d26630f9-922d-4483-8bf9-f364dca5fd50)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/03580ef7-235c-4b03-9ca6-a27677a5a175)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/d9ed4a01-70d4-45b4-a6a7-2f765f158fae)
Masked ControlNet
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/872d4785-60e4-4431-85c7-665c781dddaa)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/335a3b33-1ef8-46ff-a462-9f1b4f2c49fc)
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/b3684a15-8895-414e-8188-487269dfcada)
PhotoMaker
(Note that photomaker is a special control that need you to add the trigger word "photomaker". Your prompt should be like "a photo of photomaker")
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/07b0b626-05b5-473b-9d69-3657624d59be)
Marigold Depth
![image](https://github.com/lllyasviel/stable-diffusion-webui-forge/assets/19834515/bdf54148-892d-410d-8ed9-70b4b121b6e7)
# New Samplers (that are not in origin)
DDPM
DDPM Karras
DPM++ 2M Turbo
DPM++ 2M SDE Turbo
LCM Karras
Euler A Turbo
# About Extensions
ControlNet and TiledVAE are integrated, and you should uninstall these two extensions:
sd-webui-controlnet
multidiffusion-upscaler-for-automatic1111
Note that **AnimateDiff** is under construction by [continue-revolution](https://github.com/continue-revolution) at [sd-webui-animatediff forge/master branch](https://github.com/continue-revolution/sd-webui-animatediff/tree/forge/master) and [sd-forge-animatediff](https://github.com/continue-revolution/sd-forge-animatediff) (they are in sync). (continue-revolution original words: prompt travel, inf t2v, controlnet v2v have been proven to work well; motion lora, i2i batch still under construction and may be finished in a week")
Other extensions should work without problems, like:
canvas-zoom
translations/localizations
Dynamic Prompts
Adetailer
Ultimate SD Upscale
Reactor
However, if newer extensions use Forge, their codes can be much shorter.
Usually if an old extension rework using Forge's unet patcher, 80% codes can be removed, especially when they need to call controlnet.
# Contribution
Forge uses a bot to get commits and codes from https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/dev every afternoon (if merge is automatically successful by a git bot, or by my compiler, or by my ChatGPT bot) or mid-night (if my compiler and my ChatGPT bot both failed to merge and I review it manually).
All PRs that can be implemented in https://github.com/AUTOMATIC1111/stable-diffusion-webui/tree/dev should submit PRs there.
Feel free to submit PRs related to the functionality of Forge here.
_typos.toml
+5
View File
@@ -0,0 +1,5 @@
[default.extend-words]
# Part of "RGBa" (Pillow's pre-multiplied alpha RGB mode)
Ba = "Ba"
# HSA is something AMD uses for their GPUs
HSA = "HSA"
configs/alt-diffusion-inference.yaml
+1 -1
View File
@@ -40,7 +40,7 @@ model:
use_spatial_transformer: True
transformer_depth: 1
context_dim: 768
use_checkpoint: True
use_checkpoint: False
legacy: False
first_stage_config:
configs/alt-diffusion-m18-inference.yaml
+1 -1
View File
@@ -41,7 +41,7 @@ model:
use_linear_in_transformer: True
transformer_depth: 1
context_dim: 1024
use_checkpoint: True
use_checkpoint: False
legacy: False
first_stage_config:
configs/instruct-pix2pix.yaml
+1 -1
View File
@@ -45,7 +45,7 @@ model:
use_spatial_transformer: True
transformer_depth: 1
context_dim: 768
use_checkpoint: True
use_checkpoint: False
legacy: False
first_stage_config:
configs/sd3-inference.yaml
+5
View File
@@ -0,0 +1,5 @@
model:
target: modules.models.sd3.sd3_model.SD3Inferencer
params:
shift: 3
state_dict: null
configs/sd_xl_inpaint.yaml
+1 -1
View File
@@ -21,7 +21,7 @@ model:
params:
adm_in_channels: 2816
num_classes: sequential
use_checkpoint: True
use_checkpoint: False
in_channels: 9
out_channels: 4
model_channels: 320
configs/v1-inference.yaml
+1 -1
View File
@@ -40,7 +40,7 @@ model:
use_spatial_transformer: True
transformer_depth: 1
context_dim: 768
use_checkpoint: True
use_checkpoint: False
legacy: False
first_stage_config:
configs/v1-inpainting-inference.yaml
+1 -1
View File
@@ -40,7 +40,7 @@ model:
use_spatial_transformer: True
transformer_depth: 1
context_dim: 768
use_checkpoint: True
use_checkpoint: False
legacy: False
first_stage_config:
extensions-builtin/LDSR/sd_hijack_ddpm_v1.py
+5 -5
View File
@@ -301,7 +301,7 @@ class DDPMV1(pl.LightningModule):
elif self.parameterization == "x0":
target = x_start
else:
raise NotImplementedError(f"Paramterization {self.parameterization} not yet supported")
raise NotImplementedError(f"Parameterization {self.parameterization} not yet supported")
loss = self.get_loss(model_out, target, mean=False).mean(dim=[1, 2, 3])
@@ -572,7 +572,7 @@ class LatentDiffusionV1(DDPMV1):
:param h: height
:param w: width
:return: normalized distance to image border,
wtith min distance = 0 at border and max dist = 0.5 at image center
with min distance = 0 at border and max dist = 0.5 at image center
"""
lower_right_corner = torch.tensor([h - 1, w - 1]).view(1, 1, 2)
arr = self.meshgrid(h, w) / lower_right_corner
@@ -880,7 +880,7 @@ class LatentDiffusionV1(DDPMV1):
def apply_model(self, x_noisy, t, cond, return_ids=False):
if isinstance(cond, dict):
# hybrid case, cond is exptected to be a dict
# hybrid case, cond is expected to be a dict
pass
else:
if not isinstance(cond, list):
@@ -916,7 +916,7 @@ class LatentDiffusionV1(DDPMV1):
cond_list = [{c_key: [c[:, :, :, :, i]]} for i in range(c.shape[-1])]
elif self.cond_stage_key == 'coordinates_bbox':
assert 'original_image_size' in self.split_input_params, 'BoudingBoxRescaling is missing original_image_size'
assert 'original_image_size' in self.split_input_params, 'BoundingBoxRescaling is missing original_image_size'
# assuming padding of unfold is always 0 and its dilation is always 1
n_patches_per_row = int((w - ks[0]) / stride[0] + 1)
@@ -926,7 +926,7 @@ class LatentDiffusionV1(DDPMV1):
num_downs = self.first_stage_model.encoder.num_resolutions - 1
rescale_latent = 2 ** (num_downs)
# get top left postions of patches as conforming for the bbbox tokenizer, therefore we
# get top left positions of patches as conforming for the bbbox tokenizer, therefore we
# need to rescale the tl patch coordinates to be in between (0,1)
tl_patch_coordinates = [(rescale_latent * stride[0] * (patch_nr % n_patches_per_row) / full_img_w,
rescale_latent * stride[1] * (patch_nr // n_patches_per_row) / full_img_h)
extensions-builtin/Lora/extra_networks_lora.py
+9 -14
View File
@@ -9,6 +9,8 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
self.errors = {}
"""mapping of network names to the number of errors the network had during operation"""
remove_symbols = str.maketrans('', '', ":,")
def activate(self, p, params_list):
additional = shared.opts.sd_lora
@@ -43,22 +45,15 @@ class ExtraNetworkLora(extra_networks.ExtraNetwork):
networks.load_networks(names, te_multipliers, unet_multipliers, dyn_dims)
if shared.opts.lora_add_hashes_to_infotext:
network_hashes = []
if not getattr(p, "is_hr_pass", False) or not hasattr(p, "lora_hashes"):
p.lora_hashes = {}
for item in networks.loaded_networks:
shorthash = item.network_on_disk.shorthash
if not shorthash:
continue
if item.network_on_disk.shorthash and item.mentioned_name:
p.lora_hashes[item.mentioned_name.translate(self.remove_symbols)] = item.network_on_disk.shorthash
alias = item.mentioned_name
if not alias:
continue
alias = alias.replace(":", "").replace(",", "")
network_hashes.append(f"{alias}: {shorthash}")
if network_hashes:
p.extra_generation_params["Lora hashes"] = ", ".join(network_hashes)
if p.lora_hashes:
p.extra_generation_params["Lora hashes"] = ', '.join(f'{k}: {v}' for k, v in p.lora_hashes.items())
def deactivate(self, p):
if self.errors:
extensions-builtin/Lora/network.py
+41 -3
View File
@@ -7,6 +7,7 @@ import torch.nn as nn
import torch.nn.functional as F
from modules import sd_models, cache, errors, hashes, shared
import modules.models.sd3.mmdit
NetworkWeights = namedtuple('NetworkWeights', ['network_key', 'sd_key', 'w', 'sd_module'])
@@ -29,7 +30,6 @@ class NetworkOnDisk:
def read_metadata():
metadata = sd_models.read_metadata_from_safetensors(filename)
metadata.pop('ssmd_cover_images', None) # those are cover images, and they are too big to display in UI as text
return metadata
@@ -115,8 +115,17 @@ class NetworkModule:
self.sd_key = weights.sd_key
self.sd_module = weights.sd_module
if hasattr(self.sd_module, 'weight'):
if isinstance(self.sd_module, modules.models.sd3.mmdit.QkvLinear):
s = self.sd_module.weight.shape
self.shape = (s[0] // 3, s[1])
elif hasattr(self.sd_module, 'weight'):
self.shape = self.sd_module.weight.shape
elif isinstance(self.sd_module, nn.MultiheadAttention):
# For now, only self-attn use Pytorch's MHA
# So assume all qkvo proj have same shape
self.shape = self.sd_module.out_proj.weight.shape
else:
self.shape = None
self.ops = None
self.extra_kwargs = {}
@@ -146,6 +155,9 @@ class NetworkModule:
self.alpha = weights.w["alpha"].item() if "alpha" in weights.w else None
self.scale = weights.w["scale"].item() if "scale" in weights.w else None
self.dora_scale = weights.w.get("dora_scale", None)
self.dora_norm_dims = len(self.shape) - 1
def multiplier(self):
if 'transformer' in self.sd_key[:20]:
return self.network.te_multiplier
@@ -160,6 +172,27 @@ class NetworkModule:
return 1.0
def apply_weight_decompose(self, updown, orig_weight):
# Match the device/dtype
orig_weight = orig_weight.to(updown.dtype)
dora_scale = self.dora_scale.to(device=orig_weight.device, dtype=updown.dtype)
updown = updown.to(orig_weight.device)
merged_scale1 = updown + orig_weight
merged_scale1_norm = (
merged_scale1.transpose(0, 1)
.reshape(merged_scale1.shape[1], -1)
.norm(dim=1, keepdim=True)
.reshape(merged_scale1.shape[1], *[1] * self.dora_norm_dims)
.transpose(0, 1)
)
dora_merged = (
merged_scale1 * (dora_scale / merged_scale1_norm)
)
final_updown = dora_merged - orig_weight
return final_updown
def finalize_updown(self, updown, orig_weight, output_shape, ex_bias=None):
if self.bias is not None:
updown = updown.reshape(self.bias.shape)
@@ -175,7 +208,12 @@ class NetworkModule:
if ex_bias is not None:
ex_bias = ex_bias * self.multiplier()
return updown * self.calc_scale() * self.multiplier(), ex_bias
updown = updown * self.calc_scale()
if self.dora_scale is not None:
updown = self.apply_weight_decompose(updown, orig_weight)
return updown * self.multiplier(), ex_bias
def calc_updown(self, target):
raise NotImplementedError()
extensions-builtin/Lora/networks.py
+63 -10
View File
@@ -1,3 +1,6 @@
from __future__ import annotations
import gradio as gr
import logging
import os
import re
@@ -26,6 +29,14 @@ def assign_network_names_to_compvis_modules(sd_model):
pass
class BundledTIHash(str):
def __init__(self, hash_str):
self.hash = hash_str
def __str__(self):
return self.hash if shared.opts.lora_bundled_ti_to_infotext else ''
def load_network(name, network_on_disk):
net = network.Network(name, network_on_disk)
net.mtime = os.path.getmtime(network_on_disk.filename)
@@ -46,6 +57,16 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
loaded_networks.clear()
unavailable_networks = []
for name in names:
if name.lower() in forbidden_network_aliases and available_networks.get(name) is None:
unavailable_networks.append(name)
elif available_network_aliases.get(name) is None:
unavailable_networks.append(name)
if unavailable_networks:
update_available_networks_by_names(unavailable_networks)
networks_on_disk = [available_networks.get(name, None) if name.lower() in forbidden_network_aliases else available_network_aliases.get(name, None) for name in names]
if any(x is None for x in networks_on_disk):
list_available_networks()
@@ -84,6 +105,28 @@ def load_networks(names, te_multipliers=None, unet_multipliers=None, dyn_dims=No
return
def allowed_layer_without_weight(layer):
if isinstance(layer, torch.nn.LayerNorm) and not layer.elementwise_affine:
return True
return False
def store_weights_backup(weight):
if weight is None:
return None
return weight.to(devices.cpu, copy=True)
def restore_weights_backup(obj, field, weight):
if weight is None:
setattr(obj, field, None)
return
getattr(obj, field).copy_(weight)
def network_restore_weights_from_backup(self: Union[torch.nn.Conv2d, torch.nn.Linear, torch.nn.GroupNorm, torch.nn.LayerNorm, torch.nn.MultiheadAttention]):
pass
@@ -140,21 +183,15 @@ def network_MultiheadAttention_load_state_dict(self, *args, **kwargs):
pass
def list_available_networks():
available_networks.clear()
available_network_aliases.clear()
forbidden_network_aliases.clear()
available_network_hash_lookup.clear()
forbidden_network_aliases.update({"none": 1, "Addams": 1})
os.makedirs(shared.cmd_opts.lora_dir, exist_ok=True)
def process_network_files(names: list[str] | None = None):
candidates = list(shared.walk_files(shared.cmd_opts.lora_dir, allowed_extensions=[".pt", ".ckpt", ".safetensors"]))
for filename in candidates:
if os.path.isdir(filename):
continue
name = os.path.splitext(os.path.basename(filename))[0]
# if names is provided, only load networks with names in the list
if names and name not in names:
continue
try:
entry = network.NetworkOnDisk(name, filename)
except OSError: # should catch FileNotFoundError and PermissionError etc.
@@ -170,6 +207,22 @@ def list_available_networks():
available_network_aliases[entry.alias] = entry
def update_available_networks_by_names(names: list[str]):
process_network_files(names)
def list_available_networks():
available_networks.clear()
available_network_aliases.clear()
forbidden_network_aliases.clear()
available_network_hash_lookup.clear()
forbidden_network_aliases.update({"none": 1, "Addams": 1})
os.makedirs(shared.cmd_opts.lora_dir, exist_ok=True)
process_network_files()
re_network_name = re.compile(r"(.*)\s*\([0-9a-fA-F]+\)")
extensions-builtin/Lora/scripts/lora_script.py
+2 -1
View File
@@ -35,7 +35,8 @@ shared.options_templates.update(shared.options_section(('extra_networks', "Extra
"sd_lora": shared.OptionInfo("None", "Add network to prompt", gr.Dropdown, lambda: {"choices": ["None", *networks.available_networks]}, refresh=networks.list_available_networks),
"lora_preferred_name": shared.OptionInfo("Alias from file", "When adding to prompt, refer to Lora by", gr.Radio, {"choices": ["Alias from file", "Filename"]}),
"lora_add_hashes_to_infotext": shared.OptionInfo(True, "Add Lora hashes to infotext"),
"lora_show_all": shared.OptionInfo(False, "Always show all networks on the Lora page").info("otherwise, those detected as for incompatible version of Stable Diffusion will be hidden"),
"lora_bundled_ti_to_infotext": shared.OptionInfo(True, "Add Lora name as TI hashes for bundled Textual Inversion").info('"Add Textual Inversion hashes to infotext" needs to be enabled'),
"lora_filter_disabled": shared.OptionInfo(True, "Always show all networks on the Lora page").info("otherwise, those detected as for incompatible version of Stable Diffusion will be hidden"),
"lora_hide_unknown_for_versions": shared.OptionInfo([], "Hide networks of unknown versions for model versions", gr.CheckboxGroup, {"choices": ["SD1", "SD2", "SDXL"]}),
"lora_in_memory_limit": shared.OptionInfo(0, "Number of Lora networks to keep cached in memory", gr.Number, {"precision": 0}),
"lora_not_found_warning_console": shared.OptionInfo(False, "Lora not found warning in console"),
extensions-builtin/Lora/ui_edit_user_metadata.py
+8 -4
View File
@@ -21,10 +21,12 @@ re_comma = re.compile(r" *, *")
def build_tags(metadata):
tags = {}
for _, tags_dict in metadata.get("ss_tag_frequency", {}).items():
for tag, tag_count in tags_dict.items():
tag = tag.strip()
tags[tag] = tags.get(tag, 0) + int(tag_count)
ss_tag_frequency = metadata.get("ss_tag_frequency", {})
if ss_tag_frequency is not None and hasattr(ss_tag_frequency, 'items'):
for _, tags_dict in ss_tag_frequency.items():
for tag, tag_count in tags_dict.items():
tag = tag.strip()
tags[tag] = tags.get(tag, 0) + int(tag_count)
if tags and is_non_comma_tagset(tags):
new_tags = {}
@@ -149,6 +151,8 @@ class LoraUserMetadataEditor(ui_extra_networks_user_metadata.UserMetadataEditor)
v = random.random() * max_count
if count > v:
for x in "({[]})":
tag = tag.replace(x, '\\' + x)
res.append(tag)
return ", ".join(sorted(res))
extensions-builtin/Lora/ui_extra_networks_lora.py
+2 -2
View File
@@ -31,7 +31,7 @@ class ExtraNetworksPageLora(ui_extra_networks.ExtraNetworksPage):
"name": name,
"filename": lora_on_disk.filename,
"shorthash": lora_on_disk.shorthash,
"preview": self.find_preview(path),
"preview": self.find_preview(path) or self.find_embedded_preview(path, name, lora_on_disk.metadata),
"description": self.find_description(path),
"search_terms": search_terms,
"local_preview": f"{path}.{shared.opts.samples_format}",
@@ -60,7 +60,7 @@ class ExtraNetworksPageLora(ui_extra_networks.ExtraNetworksPage):
else:
sd_version = lora_on_disk.sd_version
if shared.opts.lora_show_all or not enable_filter:
if shared.opts.lora_filter_disabled or not enable_filter or not shared.sd_model:
pass
elif sd_version == network.SdVersion.Unknown:
model_version = network.SdVersion.SDXL if shared.sd_model.is_sdxl else network.SdVersion.SD2 if shared.sd_model.is_sd2 else network.SdVersion.SD1
extensions-builtin/canvas-zoom-and-pan/javascript/zoom.js
-968
View File
@@ -1,968 +0,0 @@
onUiLoaded(async() => {
const elementIDs = {
img2imgTabs: "#mode_img2img .tab-nav",
inpaint: "#img2maskimg",
inpaintSketch: "#inpaint_sketch",
rangeGroup: "#img2img_column_size",
sketch: "#img2img_sketch"
};
const tabNameToElementId = {
"Inpaint sketch": elementIDs.inpaintSketch,
"Inpaint": elementIDs.inpaint,
"Sketch": elementIDs.sketch
};
// Helper functions
// Get active tab
/**
* Waits for an element to be present in the DOM.
*/
const waitForElement = (id) => new Promise(resolve => {
const checkForElement = () => {
const element = document.querySelector(id);
if (element) return resolve(element);
setTimeout(checkForElement, 100);
};
checkForElement();
});
function getActiveTab(elements, all = false) {
const tabs = elements.img2imgTabs.querySelectorAll("button");
if (all) return tabs;
for (let tab of tabs) {
if (tab.classList.contains("selected")) {
return tab;
}
}
}
// Get tab ID
function getTabId(elements) {
const activeTab = getActiveTab(elements);
return tabNameToElementId[activeTab.innerText];
}
// Wait until opts loaded
async function waitForOpts() {
for (; ;) {
if (window.opts && Object.keys(window.opts).length) {
return window.opts;
}
await new Promise(resolve => setTimeout(resolve, 100));
}
}
// Detect whether the element has a horizontal scroll bar
function hasHorizontalScrollbar(element) {
return element.scrollWidth > element.clientWidth;
}
// Function for defining the "Ctrl", "Shift" and "Alt" keys
function isModifierKey(event, key) {
switch (key) {
case "Ctrl":
return event.ctrlKey;
case "Shift":
return event.shiftKey;
case "Alt":
return event.altKey;
default:
return false;
}
}
// Check if hotkey is valid
function isValidHotkey(value) {
const specialKeys = ["Ctrl", "Alt", "Shift", "Disable"];
return (
(typeof value === "string" &&
value.length === 1 &&
/[a-z]/i.test(value)) ||
specialKeys.includes(value)
);
}
// Normalize hotkey
function normalizeHotkey(hotkey) {
return hotkey.length === 1 ? "Key" + hotkey.toUpperCase() : hotkey;
}
// Format hotkey for display
function formatHotkeyForDisplay(hotkey) {
return hotkey.startsWith("Key") ? hotkey.slice(3) : hotkey;
}
// Create hotkey configuration with the provided options
function createHotkeyConfig(defaultHotkeysConfig, hotkeysConfigOpts) {
const result = {}; // Resulting hotkey configuration
const usedKeys = new Set(); // Set of used hotkeys
// Iterate through defaultHotkeysConfig keys
for (const key in defaultHotkeysConfig) {
const userValue = hotkeysConfigOpts[key]; // User-provided hotkey value
const defaultValue = defaultHotkeysConfig[key]; // Default hotkey value
// Apply appropriate value for undefined, boolean, or object userValue
if (
userValue === undefined ||
typeof userValue === "boolean" ||
typeof userValue === "object" ||
userValue === "disable"
) {
result[key] =
userValue === undefined ? defaultValue : userValue;
} else if (isValidHotkey(userValue)) {
const normalizedUserValue = normalizeHotkey(userValue);
// Check for conflicting hotkeys
if (!usedKeys.has(normalizedUserValue)) {
usedKeys.add(normalizedUserValue);
result[key] = normalizedUserValue;
} else {
console.error(
`Hotkey: ${formatHotkeyForDisplay(
userValue
)} for ${key} is repeated and conflicts with another hotkey. The default hotkey is used: ${formatHotkeyForDisplay(
defaultValue
)}`
);
result[key] = defaultValue;
}
} else {
console.error(
`Hotkey: ${formatHotkeyForDisplay(
userValue
)} for ${key} is not valid. The default hotkey is used: ${formatHotkeyForDisplay(
defaultValue
)}`
);
result[key] = defaultValue;
}
}
return result;
}
// Disables functions in the config object based on the provided list of function names
function disableFunctions(config, disabledFunctions) {
// Bind the hasOwnProperty method to the functionMap object to avoid errors
const hasOwnProperty =
Object.prototype.hasOwnProperty.bind(functionMap);
// Loop through the disabledFunctions array and disable the corresponding functions in the config object
disabledFunctions.forEach(funcName => {
if (hasOwnProperty(funcName)) {
const key = functionMap[funcName];
config[key] = "disable";
}
});
// Return the updated config object
return config;
}
/**
* The restoreImgRedMask function displays a red mask around an image to indicate the aspect ratio.
* If the image display property is set to 'none', the mask breaks. To fix this, the function
* temporarily sets the display property to 'block' and then hides the mask again after 300 milliseconds
* to avoid breaking the canvas. Additionally, the function adjusts the mask to work correctly on
* very long images.
*/
function restoreImgRedMask(elements) {
const mainTabId = getTabId(elements);
if (!mainTabId) return;
const mainTab = gradioApp().querySelector(mainTabId);
const img = mainTab.querySelector("img");
const imageARPreview = gradioApp().querySelector("#imageARPreview");
if (!img || !imageARPreview) return;
imageARPreview.style.transform = "";
if (parseFloat(mainTab.style.width) > 865) {
const transformString = mainTab.style.transform;
const scaleMatch = transformString.match(
/scale\(([-+]?[0-9]*\.?[0-9]+)\)/
);
let zoom = 1; // default zoom
if (scaleMatch && scaleMatch[1]) {
zoom = Number(scaleMatch[1]);
}
imageARPreview.style.transformOrigin = "0 0";
imageARPreview.style.transform = `scale(${zoom})`;
}
if (img.style.display !== "none") return;
img.style.display = "block";
setTimeout(() => {
img.style.display = "none";
}, 400);
}
const hotkeysConfigOpts = await waitForOpts();
// Default config
const defaultHotkeysConfig = {
canvas_hotkey_zoom: "Alt",
canvas_hotkey_adjust: "Ctrl",
canvas_hotkey_reset: "KeyR",
canvas_hotkey_fullscreen: "KeyS",
canvas_hotkey_move: "KeyF",
canvas_hotkey_overlap: "KeyO",
canvas_hotkey_shrink_brush: "KeyQ",
canvas_hotkey_grow_brush: "KeyW",
canvas_disabled_functions: [],
canvas_show_tooltip: true,
canvas_auto_expand: true,
canvas_blur_prompt: false,
};
const functionMap = {
"Zoom": "canvas_hotkey_zoom",
"Adjust brush size": "canvas_hotkey_adjust",
"Hotkey shrink brush": "canvas_hotkey_shrink_brush",
"Hotkey enlarge brush": "canvas_hotkey_grow_brush",
"Moving canvas": "canvas_hotkey_move",
"Fullscreen": "canvas_hotkey_fullscreen",
"Reset Zoom": "canvas_hotkey_reset",
"Overlap": "canvas_hotkey_overlap"
};
// Loading the configuration from opts
const preHotkeysConfig = createHotkeyConfig(
defaultHotkeysConfig,
hotkeysConfigOpts
);
// Disable functions that are not needed by the user
const hotkeysConfig = disableFunctions(
preHotkeysConfig,
preHotkeysConfig.canvas_disabled_functions
);
let isMoving = false;
let mouseX, mouseY;
let activeElement;
const elements = Object.fromEntries(
Object.keys(elementIDs).map(id => [
id,
gradioApp().querySelector(elementIDs[id])
])
);
const elemData = {};
// Apply functionality to the range inputs. Restore redmask and correct for long images.
const rangeInputs = elements.rangeGroup ?
Array.from(elements.rangeGroup.querySelectorAll("input")) :
[
gradioApp().querySelector("#img2img_width input[type='range']"),
gradioApp().querySelector("#img2img_height input[type='range']")
];
for (const input of rangeInputs) {
input?.addEventListener("input", () => restoreImgRedMask(elements));
}
function applyZoomAndPan(elemId, isExtension = true) {
const targetElement = gradioApp().querySelector(elemId);
if (!targetElement) {
console.log("Element not found");
return;
}
targetElement.style.transformOrigin = "0 0";
elemData[elemId] = {
zoom: 1,
panX: 0,
panY: 0
};
let fullScreenMode = false;
// Create tooltip
function createTooltip() {
const toolTipElemnt =
targetElement.querySelector(".image-container");
const tooltip = document.createElement("div");
tooltip.className = "canvas-tooltip";
// Creating an item of information
const info = document.createElement("i");
info.className = "canvas-tooltip-info";
info.textContent = "";
// Create a container for the contents of the tooltip
const tooltipContent = document.createElement("div");
tooltipContent.className = "canvas-tooltip-content";
// Define an array with hotkey information and their actions
const hotkeysInfo = [
{
configKey: "canvas_hotkey_zoom",
action: "Zoom canvas",
keySuffix: " + wheel"
},
{
configKey: "canvas_hotkey_adjust",
action: "Adjust brush size",
keySuffix: " + wheel"
},
{configKey: "canvas_hotkey_reset", action: "Reset zoom"},
{
configKey: "canvas_hotkey_fullscreen",
action: "Fullscreen mode"
},
{configKey: "canvas_hotkey_move", action: "Move canvas"},
{configKey: "canvas_hotkey_overlap", action: "Overlap"}
];
// Create hotkeys array with disabled property based on the config values
const hotkeys = hotkeysInfo.map(info => {
const configValue = hotkeysConfig[info.configKey];
const key = info.keySuffix ?
`${configValue}${info.keySuffix}` :
configValue.charAt(configValue.length - 1);
return {
key,
action: info.action,
disabled: configValue === "disable"
};
});
for (const hotkey of hotkeys) {
if (hotkey.disabled) {
continue;
}
const p = document.createElement("p");
p.innerHTML = `<b>${hotkey.key}</b> - ${hotkey.action}`;
tooltipContent.appendChild(p);
}
// Add information and content elements to the tooltip element
tooltip.appendChild(info);
tooltip.appendChild(tooltipContent);
// Add a hint element to the target element
toolTipElemnt.appendChild(tooltip);
}
//Show tool tip if setting enable
if (hotkeysConfig.canvas_show_tooltip) {
createTooltip();
}
// In the course of research, it was found that the tag img is very harmful when zooming and creates white canvases. This hack allows you to almost never think about this problem, it has no effect on webui.
function fixCanvas() {
const activeTab = getActiveTab(elements).textContent.trim();
if (activeTab !== "img2img") {
const img = targetElement.querySelector(`${elemId} img`);
if (img && img.style.display !== "none") {
img.style.display = "none";
img.style.visibility = "hidden";
}
}
}
// Reset the zoom level and pan position of the target element to their initial values
function resetZoom() {
elemData[elemId] = {
zoomLevel: 1,
panX: 0,
panY: 0
};
if (isExtension) {
targetElement.style.overflow = "hidden";
}
targetElement.isZoomed = false;
fixCanvas();
targetElement.style.transform = `scale(${elemData[elemId].zoomLevel}) translate(${elemData[elemId].panX}px, ${elemData[elemId].panY}px)`;
const canvas = gradioApp().querySelector(
`${elemId} canvas[key="interface"]`
);
toggleOverlap("off");
fullScreenMode = false;
const closeBtn = targetElement.querySelector("button[aria-label='Remove Image']");
if (closeBtn) {
closeBtn.addEventListener("click", resetZoom);
}
if (canvas && isExtension) {
const parentElement = targetElement.closest('[id^="component-"]');
if (
canvas &&
parseFloat(canvas.style.width) > parentElement.offsetWidth &&
parseFloat(targetElement.style.width) > parentElement.offsetWidth
) {
fitToElement();
return;
}
}
if (
canvas &&
!isExtension &&
parseFloat(canvas.style.width) > 865 &&
parseFloat(targetElement.style.width) > 865
) {
fitToElement();
return;
}
targetElement.style.width = "";
}
// Toggle the zIndex of the target element between two values, allowing it to overlap or be overlapped by other elements
function toggleOverlap(forced = "") {
const zIndex1 = "0";
const zIndex2 = "998";
targetElement.style.zIndex =
targetElement.style.zIndex !== zIndex2 ? zIndex2 : zIndex1;
if (forced === "off") {
targetElement.style.zIndex = zIndex1;
} else if (forced === "on") {
targetElement.style.zIndex = zIndex2;
}
}
// Adjust the brush size based on the deltaY value from a mouse wheel event
function adjustBrushSize(
elemId,
deltaY,
withoutValue = false,
percentage = 5
) {
const input =
gradioApp().querySelector(
`${elemId} input[aria-label='Brush radius']`
) ||
gradioApp().querySelector(
`${elemId} button[aria-label="Use brush"]`
);
if (input) {
input.click();
if (!withoutValue) {
const maxValue =
parseFloat(input.getAttribute("max")) || 100;
const changeAmount = maxValue * (percentage / 100);
const newValue =
parseFloat(input.value) +
(deltaY > 0 ? -changeAmount : changeAmount);
input.value = Math.min(Math.max(newValue, 0), maxValue);
input.dispatchEvent(new Event("change"));
}
}
}
// Reset zoom when uploading a new image
const fileInput = gradioApp().querySelector(
`${elemId} input[type="file"][accept="image/*"].svelte-116rqfv`
);
fileInput.addEventListener("click", resetZoom);
// Update the zoom level and pan position of the target element based on the values of the zoomLevel, panX and panY variables
function updateZoom(newZoomLevel, mouseX, mouseY) {
newZoomLevel = Math.max(0.1, Math.min(newZoomLevel, 15));
elemData[elemId].panX +=
mouseX - (mouseX * newZoomLevel) / elemData[elemId].zoomLevel;
elemData[elemId].panY +=
mouseY - (mouseY * newZoomLevel) / elemData[elemId].zoomLevel;
targetElement.style.transformOrigin = "0 0";
targetElement.style.transform = `translate(${elemData[elemId].panX}px, ${elemData[elemId].panY}px) scale(${newZoomLevel})`;
toggleOverlap("on");
if (isExtension) {
targetElement.style.overflow = "visible";
}
return newZoomLevel;
}
// Change the zoom level based on user interaction
function changeZoomLevel(operation, e) {
if (isModifierKey(e, hotkeysConfig.canvas_hotkey_zoom)) {
e.preventDefault();
let zoomPosX, zoomPosY;
let delta = 0.2;
if (elemData[elemId].zoomLevel > 7) {
delta = 0.9;
} else if (elemData[elemId].zoomLevel > 2) {
delta = 0.6;
}
zoomPosX = e.clientX;
zoomPosY = e.clientY;
fullScreenMode = false;
elemData[elemId].zoomLevel = updateZoom(
elemData[elemId].zoomLevel +
(operation === "+" ? delta : -delta),
zoomPosX - targetElement.getBoundingClientRect().left,
zoomPosY - targetElement.getBoundingClientRect().top
);
targetElement.isZoomed = true;
}
}
/**
* This function fits the target element to the screen by calculating
* the required scale and offsets. It also updates the global variables
* zoomLevel, panX, and panY to reflect the new state.
*/
function fitToElement() {
//Reset Zoom
targetElement.style.transform = `translate(${0}px, ${0}px) scale(${1})`;
let parentElement;
if (isExtension) {
parentElement = targetElement.closest('[id^="component-"]');
} else {
parentElement = targetElement.parentElement;
}
// Get element and screen dimensions
const elementWidth = targetElement.offsetWidth;
const elementHeight = targetElement.offsetHeight;
const screenWidth = parentElement.clientWidth;
const screenHeight = parentElement.clientHeight;
// Get element's coordinates relative to the parent element
const elementRect = targetElement.getBoundingClientRect();
const parentRect = parentElement.getBoundingClientRect();
const elementX = elementRect.x - parentRect.x;
// Calculate scale and offsets
const scaleX = screenWidth / elementWidth;
const scaleY = screenHeight / elementHeight;
const scale = Math.min(scaleX, scaleY);
const transformOrigin =
window.getComputedStyle(targetElement).transformOrigin;
const [originX, originY] = transformOrigin.split(" ");
const originXValue = parseFloat(originX);
const originYValue = parseFloat(originY);
const offsetX =
(screenWidth - elementWidth * scale) / 2 -
originXValue * (1 - scale);
const offsetY =
(screenHeight - elementHeight * scale) / 2.5 -
originYValue * (1 - scale);
// Apply scale and offsets to the element
targetElement.style.transform = `translate(${offsetX}px, ${offsetY}px) scale(${scale})`;
// Update global variables
elemData[elemId].zoomLevel = scale;
elemData[elemId].panX = offsetX;
elemData[elemId].panY = offsetY;
fullScreenMode = false;
toggleOverlap("off");
}
/**
* This function fits the target element to the screen by calculating
* the required scale and offsets. It also updates the global variables
* zoomLevel, panX, and panY to reflect the new state.
*/
// Fullscreen mode
function fitToScreen() {
const canvas = gradioApp().querySelector(
`${elemId} canvas[key="interface"]`
);
if (!canvas) return;
if (canvas.offsetWidth > 862 || isExtension) {
targetElement.style.width = (canvas.offsetWidth + 2) + "px";
}
if (isExtension) {
targetElement.style.overflow = "visible";
}
if (fullScreenMode) {
resetZoom();
fullScreenMode = false;
return;
}
//Reset Zoom
targetElement.style.transform = `translate(${0}px, ${0}px) scale(${1})`;
// Get scrollbar width to right-align the image
const scrollbarWidth =
window.innerWidth - document.documentElement.clientWidth;
// Get element and screen dimensions
const elementWidth = targetElement.offsetWidth;
const elementHeight = targetElement.offsetHeight;
const screenWidth = window.innerWidth - scrollbarWidth;
const screenHeight = window.innerHeight;
// Get element's coordinates relative to the page
const elementRect = targetElement.getBoundingClientRect();
const elementY = elementRect.y;
const elementX = elementRect.x;
// Calculate scale and offsets
const scaleX = screenWidth / elementWidth;
const scaleY = screenHeight / elementHeight;
const scale = Math.min(scaleX, scaleY);
// Get the current transformOrigin
const computedStyle = window.getComputedStyle(targetElement);
const transformOrigin = computedStyle.transformOrigin;
const [originX, originY] = transformOrigin.split(" ");
const originXValue = parseFloat(originX);
const originYValue = parseFloat(originY);
// Calculate offsets with respect to the transformOrigin
const offsetX =
(screenWidth - elementWidth * scale) / 2 -
elementX -
originXValue * (1 - scale);
const offsetY =
(screenHeight - elementHeight * scale) / 2 -
elementY -
originYValue * (1 - scale);
// Apply scale and offsets to the element
targetElement.style.transform = `translate(${offsetX}px, ${offsetY}px) scale(${scale})`;
// Update global variables
elemData[elemId].zoomLevel = scale;
elemData[elemId].panX = offsetX;
elemData[elemId].panY = offsetY;
fullScreenMode = true;
toggleOverlap("on");
}
// Handle keydown events
function handleKeyDown(event) {
// Disable key locks to make pasting from the buffer work correctly
if ((event.ctrlKey && event.code === 'KeyV') || (event.ctrlKey && event.code === 'KeyC') || event.code === "F5") {
return;
}
// before activating shortcut, ensure user is not actively typing in an input field
if (!hotkeysConfig.canvas_blur_prompt) {
if (event.target.nodeName === 'TEXTAREA' || event.target.nodeName === 'INPUT') {
return;
}
}
const hotkeyActions = {
[hotkeysConfig.canvas_hotkey_reset]: resetZoom,
[hotkeysConfig.canvas_hotkey_overlap]: toggleOverlap,
[hotkeysConfig.canvas_hotkey_fullscreen]: fitToScreen,
[hotkeysConfig.canvas_hotkey_shrink_brush]: () => adjustBrushSize(elemId, 10),
[hotkeysConfig.canvas_hotkey_grow_brush]: () => adjustBrushSize(elemId, -10)
};
const action = hotkeyActions[event.code];
if (action) {
event.preventDefault();
action(event);
}
if (
isModifierKey(event, hotkeysConfig.canvas_hotkey_zoom) ||
isModifierKey(event, hotkeysConfig.canvas_hotkey_adjust)
) {
event.preventDefault();
}
}
// Get Mouse position
function getMousePosition(e) {
mouseX = e.offsetX;
mouseY = e.offsetY;
}
// Simulation of the function to put a long image into the screen.
// We detect if an image has a scroll bar or not, make a fullscreen to reveal the image, then reduce it to fit into the element.
// We hide the image and show it to the user when it is ready.
targetElement.isExpanded = false;
function autoExpand() {
const canvas = document.querySelector(`${elemId} canvas[key="interface"]`);
if (canvas) {
if (hasHorizontalScrollbar(targetElement) && targetElement.isExpanded === false) {
targetElement.style.visibility = "hidden";
setTimeout(() => {
fitToScreen();
resetZoom();
targetElement.style.visibility = "visible";
targetElement.isExpanded = true;
}, 10);
}
}
}
targetElement.addEventListener("mousemove", getMousePosition);
//observers
// Creating an observer with a callback function to handle DOM changes
const observer = new MutationObserver((mutationsList, observer) => {
for (let mutation of mutationsList) {
// If the style attribute of the canvas has changed, by observation it happens only when the picture changes
if (mutation.type === 'attributes' && mutation.attributeName === 'style' &&
mutation.target.tagName.toLowerCase() === 'canvas') {
targetElement.isExpanded = false;
setTimeout(resetZoom, 10);
}
}
});
// Apply auto expand if enabled
if (hotkeysConfig.canvas_auto_expand) {
targetElement.addEventListener("mousemove", autoExpand);
// Set up an observer to track attribute changes
observer.observe(targetElement, {attributes: true, childList: true, subtree: true});
}
// Handle events only inside the targetElement
let isKeyDownHandlerAttached = false;
function handleMouseMove() {
if (!isKeyDownHandlerAttached) {
document.addEventListener("keydown", handleKeyDown);
isKeyDownHandlerAttached = true;
activeElement = elemId;
}
}
function handleMouseLeave() {
if (isKeyDownHandlerAttached) {
document.removeEventListener("keydown", handleKeyDown);
isKeyDownHandlerAttached = false;
activeElement = null;
}
}
// Add mouse event handlers
targetElement.addEventListener("mousemove", handleMouseMove);
targetElement.addEventListener("mouseleave", handleMouseLeave);
// Reset zoom when click on another tab
elements.img2imgTabs.addEventListener("click", resetZoom);
elements.img2imgTabs.addEventListener("click", () => {
// targetElement.style.width = "";
if (parseInt(targetElement.style.width) > 865) {
setTimeout(fitToElement, 0);
}
});
targetElement.addEventListener("wheel", e => {
// change zoom level
const operation = e.deltaY > 0 ? "-" : "+";
changeZoomLevel(operation, e);
// Handle brush size adjustment with ctrl key pressed
if (isModifierKey(e, hotkeysConfig.canvas_hotkey_adjust)) {
e.preventDefault();
// Increase or decrease brush size based on scroll direction
adjustBrushSize(elemId, e.deltaY);
}
});
// Handle the move event for pan functionality. Updates the panX and panY variables and applies the new transform to the target element.
function handleMoveKeyDown(e) {
// Disable key locks to make pasting from the buffer work correctly
if ((e.ctrlKey && e.code === 'KeyV') || (e.ctrlKey && event.code === 'KeyC') || e.code === "F5") {
return;
}
// before activating shortcut, ensure user is not actively typing in an input field
if (!hotkeysConfig.canvas_blur_prompt) {
if (e.target.nodeName === 'TEXTAREA' || e.target.nodeName === 'INPUT') {
return;
}
}
if (e.code === hotkeysConfig.canvas_hotkey_move) {
if (!e.ctrlKey && !e.metaKey && isKeyDownHandlerAttached) {
e.preventDefault();
document.activeElement.blur();
isMoving = true;
}
}
}
function handleMoveKeyUp(e) {
if (e.code === hotkeysConfig.canvas_hotkey_move) {
isMoving = false;
}
}
document.addEventListener("keydown", handleMoveKeyDown);
document.addEventListener("keyup", handleMoveKeyUp);
// Detect zoom level and update the pan speed.
function updatePanPosition(movementX, movementY) {
let panSpeed = 2;
if (elemData[elemId].zoomLevel > 8) {
panSpeed = 3.5;
}
elemData[elemId].panX += movementX * panSpeed;
elemData[elemId].panY += movementY * panSpeed;
// Delayed redraw of an element
requestAnimationFrame(() => {
targetElement.style.transform = `translate(${elemData[elemId].panX}px, ${elemData[elemId].panY}px) scale(${elemData[elemId].zoomLevel})`;
toggleOverlap("on");
});
}
function handleMoveByKey(e) {
if (isMoving && elemId === activeElement) {
updatePanPosition(e.movementX, e.movementY);
targetElement.style.pointerEvents = "none";
if (isExtension) {
targetElement.style.overflow = "visible";
}
} else {
targetElement.style.pointerEvents = "auto";
}
}
// Prevents sticking to the mouse
window.onblur = function() {
isMoving = false;
};
// Checks for extension
function checkForOutBox() {
const parentElement = targetElement.closest('[id^="component-"]');
if (parentElement.offsetWidth < targetElement.offsetWidth && !targetElement.isExpanded) {
resetZoom();
targetElement.isExpanded = true;
}
if (parentElement.offsetWidth < targetElement.offsetWidth && elemData[elemId].zoomLevel == 1) {
resetZoom();
}
if (parentElement.offsetWidth < targetElement.offsetWidth && targetElement.offsetWidth * elemData[elemId].zoomLevel > parentElement.offsetWidth && elemData[elemId].zoomLevel < 1 && !targetElement.isZoomed) {
resetZoom();
}
}
if (isExtension) {
targetElement.addEventListener("mousemove", checkForOutBox);
}
window.addEventListener('resize', (e) => {
resetZoom();
if (isExtension) {
targetElement.isExpanded = false;
targetElement.isZoomed = false;
}
});
gradioApp().addEventListener("mousemove", handleMoveByKey);
}
applyZoomAndPan(elementIDs.sketch, false);
applyZoomAndPan(elementIDs.inpaint, false);
applyZoomAndPan(elementIDs.inpaintSketch, false);
// Make the function global so that other extensions can take advantage of this solution
const applyZoomAndPanIntegration = async(id, elementIDs) => {
const mainEl = document.querySelector(id);
if (id.toLocaleLowerCase() === "none") {
for (const elementID of elementIDs) {
const el = await waitForElement(elementID);
if (!el) break;
applyZoomAndPan(elementID);
}
return;
}
if (!mainEl) return;
mainEl.addEventListener("click", async() => {
for (const elementID of elementIDs) {
const el = await waitForElement(elementID);
if (!el) break;
applyZoomAndPan(elementID);
}
}, {once: true});
};
window.applyZoomAndPan = applyZoomAndPan; // Only 1 elements, argument elementID, for example applyZoomAndPan("#txt2img_controlnet_ControlNet_input_image")
window.applyZoomAndPanIntegration = applyZoomAndPanIntegration; // for any extension
/*
The function `applyZoomAndPanIntegration` takes two arguments:
1. `id`: A string identifier for the element to which zoom and pan functionality will be applied on click.
If the `id` value is "none", the functionality will be applied to all elements specified in the second argument without a click event.
2. `elementIDs`: An array of string identifiers for elements. Zoom and pan functionality will be applied to each of these elements on click of the element specified by the first argument.
If "none" is specified in the first argument, the functionality will be applied to each of these elements without a click event.
Example usage:
applyZoomAndPanIntegration("#txt2img_controlnet", ["#txt2img_controlnet_ControlNet_input_image"]);
In this example, zoom and pan functionality will be applied to the element with the identifier "txt2img_controlnet_ControlNet_input_image" upon clicking the element with the identifier "txt2img_controlnet".
*/
// More examples
// Add integration with ControlNet txt2img One TAB
// applyZoomAndPanIntegration("#txt2img_controlnet", ["#txt2img_controlnet_ControlNet_input_image"]);
// Add integration with ControlNet txt2img Tabs
// applyZoomAndPanIntegration("#txt2img_controlnet",Array.from({ length: 10 }, (_, i) => `#txt2img_controlnet_ControlNet-${i}_input_image`));
// Add integration with Inpaint Anything
// applyZoomAndPanIntegration("None", ["#ia_sam_image", "#ia_sel_mask"]);
});
extensions-builtin/canvas-zoom-and-pan/scripts/hotkey_config.py
-17
View File
@@ -1,17 +0,0 @@
import gradio as gr
from modules import shared
shared.options_templates.update(shared.options_section(('canvas_hotkey', "Canvas Hotkeys"), {
"canvas_hotkey_zoom": shared.OptionInfo("Alt", "Zoom canvas", gr.Radio, {"choices": ["Shift","Ctrl", "Alt"]}).info("If you choose 'Shift' you cannot scroll horizontally, 'Alt' can cause a little trouble in firefox"),
"canvas_hotkey_adjust": shared.OptionInfo("Ctrl", "Adjust brush size", gr.Radio, {"choices": ["Shift","Ctrl", "Alt"]}).info("If you choose 'Shift' you cannot scroll horizontally, 'Alt' can cause a little trouble in firefox"),
"canvas_hotkey_shrink_brush": shared.OptionInfo("Q", "Shrink the brush size"),
"canvas_hotkey_grow_brush": shared.OptionInfo("W", "Enlarge the brush size"),
"canvas_hotkey_move": shared.OptionInfo("F", "Moving the canvas").info("To work correctly in firefox, turn off 'Automatically search the page text when typing' in the browser settings"),
"canvas_hotkey_fullscreen": shared.OptionInfo("S", "Fullscreen Mode, maximizes the picture so that it fits into the screen and stretches it to its full width "),
"canvas_hotkey_reset": shared.OptionInfo("R", "Reset zoom and canvas positon"),
"canvas_hotkey_overlap": shared.OptionInfo("O", "Toggle overlap").info("Technical button, neededs for testing"),
"canvas_show_tooltip": shared.OptionInfo(True, "Enable tooltip on the canvas"),
"canvas_auto_expand": shared.OptionInfo(True, "Automatically expands an image that does not fit completely in the canvas area, similar to manually pressing the S and R buttons"),
"canvas_blur_prompt": shared.OptionInfo(False, "Take the focus off the prompt when working with a canvas"),
"canvas_disabled_functions": shared.OptionInfo(["Overlap"], "Disable function that you don't use", gr.CheckboxGroup, {"choices": ["Zoom","Adjust brush size","Hotkey enlarge brush","Hotkey shrink brush","Moving canvas","Fullscreen","Reset Zoom","Overlap"]}),
}))
extensions-builtin/canvas-zoom-and-pan/style.css
-66
View File
@@ -1,66 +0,0 @@
.canvas-tooltip-info {
position: absolute;
top: 10px;
left: 10px;
cursor: help;
background-color: rgba(0, 0, 0, 0.3);
width: 20px;
height: 20px;
border-radius: 50%;
display: flex;
align-items: center;
justify-content: center;
flex-direction: column;
z-index: 100;
}
.canvas-tooltip-info::after {
content: '';
display: block;
width: 2px;
height: 7px;
background-color: white;
margin-top: 2px;
}
.canvas-tooltip-info::before {
content: '';
display: block;
width: 2px;
height: 2px;
background-color: white;
}
.canvas-tooltip-content {
display: none;
background-color: #f9f9f9;
color: #333;
border: 1px solid #ddd;
padding: 15px;
position: absolute;
top: 40px;
left: 10px;
width: 250px;
font-size: 16px;
opacity: 0;
border-radius: 8px;
box-shadow: 0px 8px 16px 0px rgba(0,0,0,0.2);
z-index: 100;
}
.canvas-tooltip:hover .canvas-tooltip-content {
display: block;
animation: fadeIn 0.5s;
opacity: 1;
}
@keyframes fadeIn {
from {opacity: 0;}
to {opacity: 1;}
}
.styler {
overflow:inherit !important;
}
extensions-builtin/extra-options-section/scripts/extra_options_section.py
+6 -2
View File
@@ -1,7 +1,7 @@
import math
import gradio as gr
from modules import scripts, shared, ui_components, ui_settings, infotext_utils
from modules import scripts, shared, ui_components, ui_settings, infotext_utils, errors
from modules.ui_components import FormColumn
@@ -42,7 +42,11 @@ class ExtraOptionsSection(scripts.Script):
setting_name = extra_options[index]
with FormColumn():
comp = ui_settings.create_setting_component(setting_name)
try:
comp = ui_settings.create_setting_component(setting_name)
except KeyError:
errors.report(f"Can't add extra options for {setting_name} in ui")
continue
self.comps.append(comp)
self.setting_names.append(setting_name)
extensions-builtin/forge_legacy_preprocessors/install.py
+1 -1
View File
@@ -13,7 +13,7 @@ main_req_file = repo_root / "requirements.txt"
def comparable_version(version: str) -> Tuple:
return tuple(version.split("."))
return tuple(map(int, version.split(".")))
def get_installed_version(package: str) -> Optional[str]:
scripts/processing_autosized_crop.py → extensions-builtin/postprocessing-for-training/scripts/postprocessing_autosized_crop.py
+64 -64
View File
@@ -1,64 +1,64 @@
from PIL import Image
from modules import scripts_postprocessing, ui_components
import gradio as gr
def center_crop(image: Image, w: int, h: int):
iw, ih = image.size
if ih / h < iw / w:
sw = w * ih / h
box = (iw - sw) / 2, 0, iw - (iw - sw) / 2, ih
else:
sh = h * iw / w
box = 0, (ih - sh) / 2, iw, ih - (ih - sh) / 2
return image.resize((w, h), Image.Resampling.LANCZOS, box)
def multicrop_pic(image: Image, mindim, maxdim, minarea, maxarea, objective, threshold):
iw, ih = image.size
err = lambda w, h: 1 - (lambda x: x if x < 1 else 1 / x)(iw / ih / (w / h))
wh = max(((w, h) for w in range(mindim, maxdim + 1, 64) for h in range(mindim, maxdim + 1, 64)
if minarea <= w * h <= maxarea and err(w, h) <= threshold),
key=lambda wh: (wh[0] * wh[1], -err(*wh))[::1 if objective == 'Maximize area' else -1],
default=None
)
return wh and center_crop(image, *wh)
class ScriptPostprocessingAutosizedCrop(scripts_postprocessing.ScriptPostprocessing):
name = "Auto-sized crop"
order = 4020
def ui(self):
with ui_components.InputAccordion(False, label="Auto-sized crop") as enable:
gr.Markdown('Each image is center-cropped with an automatically chosen width and height.')
with gr.Row():
mindim = gr.Slider(minimum=64, maximum=2048, step=8, label="Dimension lower bound", value=384, elem_id="postprocess_multicrop_mindim")
maxdim = gr.Slider(minimum=64, maximum=2048, step=8, label="Dimension upper bound", value=768, elem_id="postprocess_multicrop_maxdim")
with gr.Row():
minarea = gr.Slider(minimum=64 * 64, maximum=2048 * 2048, step=1, label="Area lower bound", value=64 * 64, elem_id="postprocess_multicrop_minarea")
maxarea = gr.Slider(minimum=64 * 64, maximum=2048 * 2048, step=1, label="Area upper bound", value=640 * 640, elem_id="postprocess_multicrop_maxarea")
with gr.Row():
objective = gr.Radio(["Maximize area", "Minimize error"], value="Maximize area", label="Resizing objective", elem_id="postprocess_multicrop_objective")
threshold = gr.Slider(minimum=0, maximum=1, step=0.01, label="Error threshold", value=0.1, elem_id="postprocess_multicrop_threshold")
return {
"enable": enable,
"mindim": mindim,
"maxdim": maxdim,
"minarea": minarea,
"maxarea": maxarea,
"objective": objective,
"threshold": threshold,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, mindim, maxdim, minarea, maxarea, objective, threshold):
if not enable:
return
cropped = multicrop_pic(pp.image, mindim, maxdim, minarea, maxarea, objective, threshold)
if cropped is not None:
pp.image = cropped
else:
print(f"skipped {pp.image.width}x{pp.image.height} image (can't find suitable size within error threshold)")
from PIL import Image
from modules import scripts_postprocessing, ui_components
import gradio as gr
def center_crop(image: Image, w: int, h: int):
iw, ih = image.size
if ih / h < iw / w:
sw = w * ih / h
box = (iw - sw) / 2, 0, iw - (iw - sw) / 2, ih
else:
sh = h * iw / w
box = 0, (ih - sh) / 2, iw, ih - (ih - sh) / 2
return image.resize((w, h), Image.Resampling.LANCZOS, box)
def multicrop_pic(image: Image, mindim, maxdim, minarea, maxarea, objective, threshold):
iw, ih = image.size
err = lambda w, h: 1 - (lambda x: x if x < 1 else 1 / x)(iw / ih / (w / h))
wh = max(((w, h) for w in range(mindim, maxdim + 1, 64) for h in range(mindim, maxdim + 1, 64)
if minarea <= w * h <= maxarea and err(w, h) <= threshold),
key=lambda wh: (wh[0] * wh[1], -err(*wh))[::1 if objective == 'Maximize area' else -1],
default=None
)
return wh and center_crop(image, *wh)
class ScriptPostprocessingAutosizedCrop(scripts_postprocessing.ScriptPostprocessing):
name = "Auto-sized crop"
order = 4020
def ui(self):
with ui_components.InputAccordion(False, label="Auto-sized crop") as enable:
gr.Markdown('Each image is center-cropped with an automatically chosen width and height.')
with gr.Row():
mindim = gr.Slider(minimum=64, maximum=2048, step=8, label="Dimension lower bound", value=384, elem_id="postprocess_multicrop_mindim")
maxdim = gr.Slider(minimum=64, maximum=2048, step=8, label="Dimension upper bound", value=768, elem_id="postprocess_multicrop_maxdim")
with gr.Row():
minarea = gr.Slider(minimum=64 * 64, maximum=2048 * 2048, step=1, label="Area lower bound", value=64 * 64, elem_id="postprocess_multicrop_minarea")
maxarea = gr.Slider(minimum=64 * 64, maximum=2048 * 2048, step=1, label="Area upper bound", value=640 * 640, elem_id="postprocess_multicrop_maxarea")
with gr.Row():
objective = gr.Radio(["Maximize area", "Minimize error"], value="Maximize area", label="Resizing objective", elem_id="postprocess_multicrop_objective")
threshold = gr.Slider(minimum=0, maximum=1, step=0.01, label="Error threshold", value=0.1, elem_id="postprocess_multicrop_threshold")
return {
"enable": enable,
"mindim": mindim,
"maxdim": maxdim,
"minarea": minarea,
"maxarea": maxarea,
"objective": objective,
"threshold": threshold,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, mindim, maxdim, minarea, maxarea, objective, threshold):
if not enable:
return
cropped = multicrop_pic(pp.image, mindim, maxdim, minarea, maxarea, objective, threshold)
if cropped is not None:
pp.image = cropped
else:
print(f"skipped {pp.image.width}x{pp.image.height} image (can't find suitable size within error threshold)")
scripts/postprocessing_caption.py → extensions-builtin/postprocessing-for-training/scripts/postprocessing_caption.py
+30 -30
View File
@@ -1,30 +1,30 @@
from modules import scripts_postprocessing, ui_components, deepbooru, shared
import gradio as gr
class ScriptPostprocessingCeption(scripts_postprocessing.ScriptPostprocessing):
name = "Caption"
order = 4040
def ui(self):
with ui_components.InputAccordion(False, label="Caption") as enable:
option = gr.CheckboxGroup(value=["Deepbooru"], choices=["Deepbooru", "BLIP"], show_label=False)
return {
"enable": enable,
"option": option,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, option):
if not enable:
return
captions = [pp.caption]
if "Deepbooru" in option:
captions.append(deepbooru.model.tag(pp.image))
if "BLIP" in option:
captions.append(shared.interrogator.interrogate(pp.image.convert("RGB")))
pp.caption = ", ".join([x for x in captions if x])
from modules import scripts_postprocessing, ui_components, deepbooru, shared
import gradio as gr
class ScriptPostprocessingCeption(scripts_postprocessing.ScriptPostprocessing):
name = "Caption"
order = 4040
def ui(self):
with ui_components.InputAccordion(False, label="Caption") as enable:
option = gr.CheckboxGroup(value=["Deepbooru"], choices=["Deepbooru", "BLIP"], show_label=False)
return {
"enable": enable,
"option": option,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, option):
if not enable:
return
captions = [pp.caption]
if "Deepbooru" in option:
captions.append(deepbooru.model.tag(pp.image))
if "BLIP" in option:
captions.append(shared.interrogator.interrogate(pp.image.convert("RGB")))
pp.caption = ", ".join([x for x in captions if x])
scripts/postprocessing_create_flipped_copies.py → extensions-builtin/postprocessing-for-training/scripts/postprocessing_create_flipped_copies.py
+32 -32
View File
@@ -1,32 +1,32 @@
from PIL import ImageOps, Image
from modules import scripts_postprocessing, ui_components
import gradio as gr
class ScriptPostprocessingCreateFlippedCopies(scripts_postprocessing.ScriptPostprocessing):
name = "Create flipped copies"
order = 4030
def ui(self):
with ui_components.InputAccordion(False, label="Create flipped copies") as enable:
with gr.Row():
option = gr.CheckboxGroup(value=["Horizontal"], choices=["Horizontal", "Vertical", "Both"], show_label=False)
return {
"enable": enable,
"option": option,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, option):
if not enable:
return
if "Horizontal" in option:
pp.extra_images.append(ImageOps.mirror(pp.image))
if "Vertical" in option:
pp.extra_images.append(pp.image.transpose(Image.Transpose.FLIP_TOP_BOTTOM))
if "Both" in option:
pp.extra_images.append(pp.image.transpose(Image.Transpose.FLIP_TOP_BOTTOM).transpose(Image.Transpose.FLIP_LEFT_RIGHT))
from PIL import ImageOps, Image
from modules import scripts_postprocessing, ui_components
import gradio as gr
class ScriptPostprocessingCreateFlippedCopies(scripts_postprocessing.ScriptPostprocessing):
name = "Create flipped copies"
order = 4030
def ui(self):
with ui_components.InputAccordion(False, label="Create flipped copies") as enable:
with gr.Row():
option = gr.CheckboxGroup(value=["Horizontal"], choices=["Horizontal", "Vertical", "Both"], show_label=False)
return {
"enable": enable,
"option": option,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, option):
if not enable:
return
if "Horizontal" in option:
pp.extra_images.append(ImageOps.mirror(pp.image))
if "Vertical" in option:
pp.extra_images.append(pp.image.transpose(Image.Transpose.FLIP_TOP_BOTTOM))
if "Both" in option:
pp.extra_images.append(pp.image.transpose(Image.Transpose.FLIP_TOP_BOTTOM).transpose(Image.Transpose.FLIP_LEFT_RIGHT))
scripts/postprocessing_focal_crop.py → extensions-builtin/postprocessing-for-training/scripts/postprocessing_focal_crop.py
+54 -54
View File
@@ -1,54 +1,54 @@
from modules import scripts_postprocessing, ui_components, errors
import gradio as gr
from modules.textual_inversion import autocrop
class ScriptPostprocessingFocalCrop(scripts_postprocessing.ScriptPostprocessing):
name = "Auto focal point crop"
order = 4010
def ui(self):
with ui_components.InputAccordion(False, label="Auto focal point crop") as enable:
face_weight = gr.Slider(label='Focal point face weight', value=0.9, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_focal_crop_face_weight")
entropy_weight = gr.Slider(label='Focal point entropy weight', value=0.15, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_focal_crop_entropy_weight")
edges_weight = gr.Slider(label='Focal point edges weight', value=0.5, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_focal_crop_edges_weight")
debug = gr.Checkbox(label='Create debug image', elem_id="train_process_focal_crop_debug")
return {
"enable": enable,
"face_weight": face_weight,
"entropy_weight": entropy_weight,
"edges_weight": edges_weight,
"debug": debug,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, face_weight, entropy_weight, edges_weight, debug):
if not enable:
return
if not pp.shared.target_width or not pp.shared.target_height:
return
dnn_model_path = None
try:
dnn_model_path = autocrop.download_and_cache_models()
except Exception:
errors.report("Unable to load face detection model for auto crop selection. Falling back to lower quality haar method.", exc_info=True)
autocrop_settings = autocrop.Settings(
crop_width=pp.shared.target_width,
crop_height=pp.shared.target_height,
face_points_weight=face_weight,
entropy_points_weight=entropy_weight,
corner_points_weight=edges_weight,
annotate_image=debug,
dnn_model_path=dnn_model_path,
)
result, *others = autocrop.crop_image(pp.image, autocrop_settings)
pp.image = result
pp.extra_images = [pp.create_copy(x, nametags=["focal-crop-debug"], disable_processing=True) for x in others]
from modules import scripts_postprocessing, ui_components, errors
import gradio as gr
from modules.textual_inversion import autocrop
class ScriptPostprocessingFocalCrop(scripts_postprocessing.ScriptPostprocessing):
name = "Auto focal point crop"
order = 4010
def ui(self):
with ui_components.InputAccordion(False, label="Auto focal point crop") as enable:
face_weight = gr.Slider(label='Focal point face weight', value=0.9, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_focal_crop_face_weight")
entropy_weight = gr.Slider(label='Focal point entropy weight', value=0.15, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_focal_crop_entropy_weight")
edges_weight = gr.Slider(label='Focal point edges weight', value=0.5, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_focal_crop_edges_weight")
debug = gr.Checkbox(label='Create debug image', elem_id="train_process_focal_crop_debug")
return {
"enable": enable,
"face_weight": face_weight,
"entropy_weight": entropy_weight,
"edges_weight": edges_weight,
"debug": debug,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, face_weight, entropy_weight, edges_weight, debug):
if not enable:
return
if not pp.shared.target_width or not pp.shared.target_height:
return
dnn_model_path = None
try:
dnn_model_path = autocrop.download_and_cache_models()
except Exception:
errors.report("Unable to load face detection model for auto crop selection. Falling back to lower quality haar method.", exc_info=True)
autocrop_settings = autocrop.Settings(
crop_width=pp.shared.target_width,
crop_height=pp.shared.target_height,
face_points_weight=face_weight,
entropy_points_weight=entropy_weight,
corner_points_weight=edges_weight,
annotate_image=debug,
dnn_model_path=dnn_model_path,
)
result, *others = autocrop.crop_image(pp.image, autocrop_settings)
pp.image = result
pp.extra_images = [pp.create_copy(x, nametags=["focal-crop-debug"], disable_processing=True) for x in others]
scripts/postprocessing_split_oversized.py → extensions-builtin/postprocessing-for-training/scripts/postprocessing_split_oversized.py
+71 -71
View File
@@ -1,71 +1,71 @@
import math
from modules import scripts_postprocessing, ui_components
import gradio as gr
def split_pic(image, inverse_xy, width, height, overlap_ratio):
if inverse_xy:
from_w, from_h = image.height, image.width
to_w, to_h = height, width
else:
from_w, from_h = image.width, image.height
to_w, to_h = width, height
h = from_h * to_w // from_w
if inverse_xy:
image = image.resize((h, to_w))
else:
image = image.resize((to_w, h))
split_count = math.ceil((h - to_h * overlap_ratio) / (to_h * (1.0 - overlap_ratio)))
y_step = (h - to_h) / (split_count - 1)
for i in range(split_count):
y = int(y_step * i)
if inverse_xy:
splitted = image.crop((y, 0, y + to_h, to_w))
else:
splitted = image.crop((0, y, to_w, y + to_h))
yield splitted
class ScriptPostprocessingSplitOversized(scripts_postprocessing.ScriptPostprocessing):
name = "Split oversized images"
order = 4000
def ui(self):
with ui_components.InputAccordion(False, label="Split oversized images") as enable:
with gr.Row():
split_threshold = gr.Slider(label='Threshold', value=0.5, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_split_threshold")
overlap_ratio = gr.Slider(label='Overlap ratio', value=0.2, minimum=0.0, maximum=0.9, step=0.05, elem_id="postprocess_overlap_ratio")
return {
"enable": enable,
"split_threshold": split_threshold,
"overlap_ratio": overlap_ratio,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, split_threshold, overlap_ratio):
if not enable:
return
width = pp.shared.target_width
height = pp.shared.target_height
if not width or not height:
return
if pp.image.height > pp.image.width:
ratio = (pp.image.width * height) / (pp.image.height * width)
inverse_xy = False
else:
ratio = (pp.image.height * width) / (pp.image.width * height)
inverse_xy = True
if ratio >= 1.0 or ratio > split_threshold:
return
result, *others = split_pic(pp.image, inverse_xy, width, height, overlap_ratio)
pp.image = result
pp.extra_images = [pp.create_copy(x) for x in others]
import math
from modules import scripts_postprocessing, ui_components
import gradio as gr
def split_pic(image, inverse_xy, width, height, overlap_ratio):
if inverse_xy:
from_w, from_h = image.height, image.width
to_w, to_h = height, width
else:
from_w, from_h = image.width, image.height
to_w, to_h = width, height
h = from_h * to_w // from_w
if inverse_xy:
image = image.resize((h, to_w))
else:
image = image.resize((to_w, h))
split_count = math.ceil((h - to_h * overlap_ratio) / (to_h * (1.0 - overlap_ratio)))
y_step = (h - to_h) / (split_count - 1)
for i in range(split_count):
y = int(y_step * i)
if inverse_xy:
splitted = image.crop((y, 0, y + to_h, to_w))
else:
splitted = image.crop((0, y, to_w, y + to_h))
yield splitted
class ScriptPostprocessingSplitOversized(scripts_postprocessing.ScriptPostprocessing):
name = "Split oversized images"
order = 4000
def ui(self):
with ui_components.InputAccordion(False, label="Split oversized images") as enable:
with gr.Row():
split_threshold = gr.Slider(label='Threshold', value=0.5, minimum=0.0, maximum=1.0, step=0.05, elem_id="postprocess_split_threshold")
overlap_ratio = gr.Slider(label='Overlap ratio', value=0.2, minimum=0.0, maximum=0.9, step=0.05, elem_id="postprocess_overlap_ratio")
return {
"enable": enable,
"split_threshold": split_threshold,
"overlap_ratio": overlap_ratio,
}
def process(self, pp: scripts_postprocessing.PostprocessedImage, enable, split_threshold, overlap_ratio):
if not enable:
return
width = pp.shared.target_width
height = pp.shared.target_height
if not width or not height:
return
if pp.image.height > pp.image.width:
ratio = (pp.image.width * height) / (pp.image.height * width)
inverse_xy = False
else:
ratio = (pp.image.height * width) / (pp.image.width * height)
inverse_xy = True
if ratio >= 1.0 or ratio > split_threshold:
return
result, *others = split_pic(pp.image, inverse_xy, width, height, overlap_ratio)
pp.image = result
pp.extra_images = [pp.create_copy(x) for x in others]
extensions-builtin/sd_forge_controlnet/install.py
+1 -1
View File
@@ -13,7 +13,7 @@ main_req_file = repo_root / "requirements.txt"
def comparable_version(version: str) -> Tuple:
return tuple(version.split("."))
return tuple(map(int, version.split(".")))
def get_installed_version(package: str) -> Optional[str]:
extensions-builtin/sd_forge_controlnet/javascript/active_units.js
+10 -29
View File
@@ -95,7 +95,6 @@
this.attachImageUploadListener();
this.attachImageStateChangeObserver();
this.attachA1111SendInfoObserver();
this.attachPresetDropdownObserver();
this.attachAccordionStateObserver();
}
@@ -119,7 +118,7 @@
*/
getUnitHeaderTextElement() {
return this.tab.querySelector(
`:nth-child(${this.tabIndex + 1}) span.svelte-s1r2yt`
`button > span:nth-child(1)`
);
}
@@ -192,16 +191,17 @@
unitHeader.appendChild(span);
}
getInputImageSrc() {
const img = this.inputImageGroup.querySelector('.cnet-image img');
return img ? img.src : null;
const img = this.inputImageGroup.querySelector('.cnet-image .forge-image');
return (img && img.src.startsWith('data')) ? img.src : null;
}
getPreprocessorPreviewImageSrc() {
const img = this.generatedImageGroup.querySelector('.cnet-image img');
return img ? img.src : null;
const img = this.generatedImageGroup.querySelector('.cnet-image .forge-image');
return (img && img.src.startsWith('data')) ? img.src : null;
}
getMaskImageSrc() {
function isEmptyCanvas(canvas) {
if (!canvas) return true;
if (canvas.width == 0 || canvas.height ==0) return true;
const ctx = canvas.getContext('2d');
// Get the image data
const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
@@ -216,14 +216,14 @@
}
return isPureBlack;
}
const maskImg = this.maskImageGroup.querySelector('.cnet-mask-image img');
const maskImg = this.maskImageGroup.querySelector('.cnet-mask-image .forge-image');
// Hand-drawn mask on mask upload.
const handDrawnMaskCanvas = this.maskImageGroup.querySelector('.cnet-mask-image canvas[key="mask"]');
const handDrawnMaskCanvas = this.maskImageGroup.querySelector('.cnet-mask-image .forge-drawing-canvas');
// Hand-drawn mask on input image upload.
const inputImageHandDrawnMaskCanvas = this.inputImageGroup.querySelector('.cnet-image canvas[key="mask"]');
const inputImageHandDrawnMaskCanvas = this.inputImageGroup.querySelector('.cnet-image .forge-drawing-canvas');
if (!isEmptyCanvas(handDrawnMaskCanvas)) {
return handDrawnMaskCanvas.toDataURL();
} else if (maskImg) {
} else if (maskImg && maskImg.src.startsWith('data')) {
return maskImg.src;
} else if (!isEmptyCanvas(inputImageHandDrawnMaskCanvas)) {
return inputImageHandDrawnMaskCanvas.toDataURL();
@@ -347,25 +347,6 @@
}
}
attachPresetDropdownObserver() {
const presetDropDown = this.tab.querySelector('.cnet-preset-dropdown');
new MutationObserver((mutationsList) => {
for (const mutation of mutationsList) {
if (mutation.removedNodes.length > 0) {
setTimeout(() => {
this.updateActiveState();
this.updateActiveUnitCount();
this.updateActiveControlType();
}, 1000);
return;
}
}
}).observe(presetDropDown, {
childList: true,
subtree: true,
});
}
/**
* Observer that triggers when the ControlNetUnit's accordion(tab) closes.
*/
extensions-builtin/sd_forge_controlnet/lib_controlnet/controlnet_ui/controlnet_ui_group.py
+93 -243
View File
@@ -13,43 +13,44 @@ from lib_controlnet import (
)
from lib_controlnet.logging import logger
from lib_controlnet.controlnet_ui.openpose_editor import OpenposeEditor
from lib_controlnet.controlnet_ui.preset import ControlNetPresetUI
from lib_controlnet.controlnet_ui.tool_button import ToolButton
from lib_controlnet.controlnet_ui.photopea import Photopea
from lib_controlnet.enums import InputMode, HiResFixOption
from modules import shared, script_callbacks
from modules.ui_components import FormRow
from modules_forge.forge_util import HWC3
from lib_controlnet.external_code import UiControlNetUnit
from modules.ui_components import ToolButton
from gradio_rangeslider import RangeSlider
from modules_forge.forge_canvas.canvas import ForgeCanvas
@dataclass
class A1111Context:
"""Contains all components from A1111."""
img2img_batch_input_dir: Optional[gr.components.IOComponent] = None
img2img_batch_output_dir: Optional[gr.components.IOComponent] = None
txt2img_submit_button: Optional[gr.components.IOComponent] = None
img2img_submit_button: Optional[gr.components.IOComponent] = None
img2img_batch_input_dir = None
img2img_batch_output_dir = None
txt2img_submit_button = None
img2img_submit_button = None
# Slider controls from A1111 WebUI.
txt2img_w_slider: Optional[gr.components.IOComponent] = None
txt2img_h_slider: Optional[gr.components.IOComponent] = None
img2img_w_slider: Optional[gr.components.IOComponent] = None
img2img_h_slider: Optional[gr.components.IOComponent] = None
txt2img_w_slider = None
txt2img_h_slider = None
img2img_w_slider = None
img2img_h_slider = None
img2img_img2img_tab: Optional[gr.components.IOComponent] = None
img2img_img2img_sketch_tab: Optional[gr.components.IOComponent] = None
img2img_batch_tab: Optional[gr.components.IOComponent] = None
img2img_inpaint_tab: Optional[gr.components.IOComponent] = None
img2img_inpaint_sketch_tab: Optional[gr.components.IOComponent] = None
img2img_inpaint_upload_tab: Optional[gr.components.IOComponent] = None
img2img_img2img_tab = None
img2img_img2img_sketch_tab = None
img2img_batch_tab = None
img2img_inpaint_tab = None
img2img_inpaint_sketch_tab = None
img2img_inpaint_upload_tab = None
img2img_inpaint_area: Optional[gr.components.IOComponent] = None
txt2img_enable_hr: Optional[gr.components.IOComponent] = None
img2img_inpaint_area = None
txt2img_enable_hr = None
@property
def img2img_inpaint_tabs(self) -> Tuple[gr.components.IOComponent]:
def img2img_inpaint_tabs(self):
return (
self.img2img_inpaint_tab,
self.img2img_inpaint_sketch_tab,
@@ -57,7 +58,7 @@ class A1111Context:
)
@property
def img2img_non_inpaint_tabs(self) -> Tuple[gr.components.IOComponent]:
def img2img_non_inpaint_tabs(self):
return (
self.img2img_img2img_tab,
self.img2img_img2img_sketch_tab,
@@ -81,7 +82,7 @@ class A1111Context:
if name not in optional_components.values()
)
def set_component(self, component: gr.components.IOComponent):
def set_component(self, component):
id_mapping = {
"img2img_batch_input_dir": "img2img_batch_input_dir",
"img2img_batch_output_dir": "img2img_batch_output_dir",
@@ -187,16 +188,13 @@ class ControlNetUiGroup(object):
self.batch_image_dir = None
self.merge_tab = None
self.batch_input_gallery = None
self.merge_upload_button = None
self.merge_clear_button = None
self.batch_mask_gallery = None
self.create_canvas = None
self.canvas_width = None
self.canvas_height = None
self.canvas_create_button = None
self.canvas_cancel_button = None
self.open_new_canvas_button = None
self.webcam_enable = None
self.webcam_mirror = None
self.send_dimen_button = None
self.pixel_perfect = None
self.preprocessor_preview = None
@@ -207,6 +205,7 @@ class ControlNetUiGroup(object):
self.model = None
self.refresh_models = None
self.weight = None
self.timestep_range = None
self.guidance_start = None
self.guidance_end = None
self.advanced = None
@@ -217,7 +216,6 @@ class ControlNetUiGroup(object):
self.resize_mode = None
self.use_preview_as_input = None
self.openpose_editor = None
self.preset_panel = None
self.upload_independent_img_in_img2img = None
self.image_upload_panel = None
self.save_detected_map = None
@@ -249,43 +247,34 @@ class ControlNetUiGroup(object):
with gr.Group(visible=not self.is_img2img) as self.image_upload_panel:
self.save_detected_map = gr.Checkbox(value=True, visible=False)
with gr.Tabs():
with gr.Tabs(visible=True):
with gr.Tab(label="Single Image") as self.upload_tab:
with gr.Row(elem_classes=["cnet-image-row"], equal_height=True):
with gr.Group(elem_classes=["cnet-input-image-group"]):
self.image = gr.Image(
source="upload",
brush_radius=20,
mirror_webcam=False,
type="numpy",
tool="sketch",
self.image = ForgeCanvas(
elem_id=f"{elem_id_tabname}_{tabname}_input_image",
elem_classes=["cnet-image"],
brush_color=shared.opts.img2img_inpaint_mask_brush_color
if hasattr(
shared.opts, "img2img_inpaint_mask_brush_color"
)
else None,
)
self.image.preprocess = functools.partial(
svg_preprocess, preprocess=self.image.preprocess
contrast_scribbles=True,
height=300,
numpy=True
)
self.openpose_editor.render_upload()
with gr.Group(
visible=False, elem_classes=["cnet-generated-image-group"]
visible=False, elem_classes=["cnet-generated-image-group"]
) as self.generated_image_group:
self.generated_image = gr.Image(
value=None,
label="Preprocessor Preview",
self.generated_image = ForgeCanvas(
elem_id=f"{elem_id_tabname}_{tabname}_generated_image",
elem_classes=["cnet-image"],
interactive=True,
height=242,
) # Gradio's magic number. Only 242 works.
height=300,
no_scribbles=True,
no_upload=True,
numpy=True
)
with gr.Group(
elem_classes=["cnet-generated-image-control-group"]
elem_classes=["cnet-generated-image-control-group"]
):
if self.photopea:
self.photopea.render_child_trigger()
@@ -299,22 +288,18 @@ class ControlNetUiGroup(object):
)
with gr.Group(
visible=False, elem_classes=["cnet-mask-image-group"]
visible=False, elem_classes=["cnet-mask-image-group"]
) as self.mask_image_group:
self.mask_image = gr.Image(
value=None,
label="Mask",
self.mask_image = ForgeCanvas(
elem_id=f"{elem_id_tabname}_{tabname}_mask_image",
elem_classes=["cnet-mask-image"],
interactive=True,
brush_radius=20,
type="numpy",
tool="sketch",
brush_color=shared.opts.img2img_inpaint_mask_brush_color
if hasattr(
shared.opts, "img2img_inpaint_mask_brush_color"
)
else None,
height=300,
scribble_color='#FFFFFF',
scribble_width=1,
scribble_alpha_fixed=True,
scribble_color_fixed=True,
scribble_softness_fixed=True,
numpy=True
)
with gr.Tab(label="Batch Folder") as self.batch_tab:
@@ -337,28 +322,14 @@ class ControlNetUiGroup(object):
self.batch_input_gallery = gr.Gallery(
columns=[4], rows=[2], object_fit="contain", height="auto", label="Images"
)
with gr.Row():
self.merge_upload_button = gr.UploadButton(
"Upload Images",
file_types=["image"],
file_count="multiple",
)
self.merge_clear_button = gr.Button("Clear Images")
with gr.Group(visible=False, elem_classes=["cnet-mask-gallery-group"]) as self.batch_mask_gallery_group:
with gr.Column():
self.batch_mask_gallery = gr.Gallery(
columns=[4], rows=[2], object_fit="contain", height="auto", label="Masks"
)
with gr.Row():
self.mask_merge_upload_button = gr.UploadButton(
"Upload Masks",
file_types=["image"],
file_count="multiple",
)
self.mask_merge_clear_button = gr.Button("Clear Masks")
if self.photopea:
self.photopea.attach_photopea_output(self.generated_image)
self.photopea.attach_photopea_output(self.generated_image.background)
with gr.Accordion(
label="Open New Canvas", visible=False
@@ -397,23 +368,13 @@ class ControlNetUiGroup(object):
self.open_new_canvas_button = ToolButton(
value=ControlNetUiGroup.open_symbol,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_open_new_canvas_button",
elem_classes=["cnet-toolbutton"],
tooltip=ControlNetUiGroup.tooltips[ControlNetUiGroup.open_symbol],
)
self.webcam_enable = ToolButton(
value=ControlNetUiGroup.camera_symbol,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_webcam_enable",
tooltip=ControlNetUiGroup.tooltips[ControlNetUiGroup.camera_symbol],
)
self.webcam_mirror = ToolButton(
value=ControlNetUiGroup.reverse_symbol,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_webcam_mirror",
tooltip=ControlNetUiGroup.tooltips[
ControlNetUiGroup.reverse_symbol
],
)
self.send_dimen_button = ToolButton(
value=ControlNetUiGroup.tossup_symbol,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_send_dimen_button",
elem_classes=["cnet-toolbutton"],
tooltip=ControlNetUiGroup.tooltips[ControlNetUiGroup.tossup_symbol],
)
@@ -481,7 +442,7 @@ class ControlNetUiGroup(object):
value=ControlNetUiGroup.trigger_symbol,
visible=not self.is_img2img,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_trigger_preprocessor",
elem_classes=["cnet-run-preprocessor"],
elem_classes=["cnet-run-preprocessor", "cnet-toolbutton"],
tooltip=ControlNetUiGroup.tooltips[ControlNetUiGroup.trigger_symbol],
)
self.model = gr.Dropdown(
@@ -493,6 +454,7 @@ class ControlNetUiGroup(object):
self.refresh_models = ToolButton(
value=ControlNetUiGroup.refresh_symbol,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_refresh_models",
elem_classes=["cnet-toolbutton"],
tooltip=ControlNetUiGroup.tooltips[ControlNetUiGroup.refresh_symbol],
)
@@ -506,24 +468,22 @@ class ControlNetUiGroup(object):
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_control_weight_slider",
elem_classes="controlnet_control_weight_slider",
)
self.guidance_start = gr.Slider(
label="Starting Control Step",
value=self.default_unit.guidance_start,
minimum=0.0,
self.timestep_range = RangeSlider(
label='Timestep Range',
minimum=0,
maximum=1.0,
interactive=True,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_start_control_step_slider",
elem_classes="controlnet_start_control_step_slider",
)
self.guidance_end = gr.Slider(
label="Ending Control Step",
value=self.default_unit.guidance_end,
minimum=0.0,
maximum=1.0,
interactive=True,
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_ending_control_step_slider",
elem_classes="controlnet_ending_control_step_slider",
value=(self.default_unit.guidance_start, self.default_unit.guidance_end),
elem_id=f"{elem_id_tabname}_{tabname}_controlnet_control_step_slider",
elem_classes="controlnet_control_step_slider",
)
self.guidance_start = gr.State(self.default_unit.guidance_start)
self.guidance_end = gr.State(self.default_unit.guidance_end)
self.timestep_range.change(
lambda x: (x[0], x[1]),
inputs=[self.timestep_range],
outputs=[self.guidance_start, self.guidance_end]
)
# advanced options
with gr.Column(visible=False) as self.advanced:
@@ -581,18 +541,6 @@ class ControlNetUiGroup(object):
visible=False,
)
# self.loopback = gr.Checkbox(
# label="[Batch Loopback] Automatically send generated images to this ControlNet unit in batch generation",
# value=self.default_unit.loopback,
# elem_id=f"{elem_id_tabname}_{tabname}_controlnet_automatically_send_generated_images_checkbox",
# elem_classes="controlnet_loopback_checkbox",
# visible=False,
# )
self.preset_panel = ControlNetPresetUI(
id_prefix=f"{elem_id_tabname}_{tabname}_"
)
self.batch_image_dir_state = gr.State("")
self.output_dir_state = gr.State("")
unit_args = (
@@ -602,14 +550,16 @@ class ControlNetUiGroup(object):
self.batch_mask_dir,
self.batch_input_gallery,
self.batch_mask_gallery,
self.generated_image,
self.mask_image,
self.generated_image.background,
self.mask_image.background,
self.mask_image.foreground,
self.hr_option,
self.enabled,
self.module,
self.model,
self.weight,
self.image,
self.image.background,
self.image.foreground,
self.resize_mode,
self.processor_res,
self.threshold_a,
@@ -665,41 +615,18 @@ class ControlNetUiGroup(object):
else:
return round(num + (8 - rem))
if image:
interm = np.asarray(image.get("image"))
return closesteight(interm.shape[1]), closesteight(interm.shape[0])
if image is not None:
return closesteight(image.shape[1]), closesteight(image.shape[0])
else:
return gr.Slider.update(), gr.Slider.update()
self.send_dimen_button.click(
fn=send_dimensions,
inputs=[self.image],
inputs=[self.image.background],
outputs=[self.width_slider, self.height_slider],
show_progress=False,
)
def register_webcam_toggle(self):
def webcam_toggle():
self.webcam_enabled = not self.webcam_enabled
return {
"value": None,
"source": "webcam" if self.webcam_enabled else "upload",
"__type__": "update",
}
self.webcam_enable.click(
webcam_toggle, inputs=None, outputs=self.image, show_progress=False
)
def register_webcam_mirror_toggle(self):
def webcam_mirror_toggle():
self.webcam_mirrored = not self.webcam_mirrored
return {"mirror_webcam": self.webcam_mirrored, "__type__": "update"}
self.webcam_mirror.click(
webcam_mirror_toggle, inputs=None, outputs=self.image, show_progress=False
)
def register_refresh_all_models(self):
def refresh_all_models():
global_state.update_controlnet_filenames()
@@ -799,16 +726,17 @@ class ControlNetUiGroup(object):
)
def register_run_annotator(self):
def run_annotator(image, module, pres, pthr_a, pthr_b, t2i_w, t2i_h, pp, rm):
def run_annotator(image, mask, module, pres, pthr_a, pthr_b, t2i_w, t2i_h, pp, rm):
if image is None:
return (
gr.update(value=None, visible=True),
gr.update(visible=True),
None,
gr.update(),
*self.openpose_editor.update(""),
)
img = HWC3(image["image"])
mask = HWC3(image["mask"])
img = HWC3(image)
mask = HWC3(mask)
if not (mask > 5).any():
mask = None
@@ -862,8 +790,8 @@ class ControlNetUiGroup(object):
result = external_code.visualize_inpaint_mask(result)
return (
# Update to `generated_image`
gr.update(value=result, visible=True, interactive=False),
gr.update(visible=True),
result,
# preprocessor_preview
gr.update(value=True),
# openpose editor
@@ -873,7 +801,8 @@ class ControlNetUiGroup(object):
self.trigger_preprocessor.click(
fn=run_annotator,
inputs=[
self.image,
self.image.background,
self.image.foreground,
self.module,
self.processor_res,
self.threshold_a,
@@ -884,7 +813,8 @@ class ControlNetUiGroup(object):
self.resize_mode,
],
outputs=[
self.generated_image,
self.generated_image.block,
self.generated_image.background,
self.preprocessor_preview,
*self.openpose_editor.outputs(),
],
@@ -909,7 +839,7 @@ class ControlNetUiGroup(object):
fn=shift_preview,
inputs=[self.preprocessor_preview],
outputs=[
self.generated_image,
self.generated_image.background,
self.generated_image_group,
self.use_preview_as_input,
self.openpose_editor.download_link,
@@ -920,27 +850,27 @@ class ControlNetUiGroup(object):
def register_create_canvas(self):
self.open_new_canvas_button.click(
lambda: gr.Accordion.update(visible=True),
lambda: gr.update(visible=True),
inputs=None,
outputs=self.create_canvas,
show_progress=False,
)
self.canvas_cancel_button.click(
lambda: gr.Accordion.update(visible=False),
lambda: gr.update(visible=False),
inputs=None,
outputs=self.create_canvas,
show_progress=False,
)
def fn_canvas(h, w):
return np.zeros(shape=(h, w, 3), dtype=np.uint8), gr.Accordion.update(
return np.zeros(shape=(h, w, 3), dtype=np.uint8), gr.update(
visible=False
)
self.canvas_create_button.click(
fn=fn_canvas,
inputs=[self.canvas_height, self.canvas_width],
outputs=[self.image, self.create_canvas],
outputs=[self.image.background, self.create_canvas],
show_progress=False,
)
@@ -956,7 +886,7 @@ class ControlNetUiGroup(object):
fn_same_checked,
inputs=self.upload_independent_img_in_img2img,
outputs=[
self.image,
self.image.background,
self.batch_image_dir,
self.preprocessor_preview,
self.image_upload_panel,
@@ -993,7 +923,7 @@ class ControlNetUiGroup(object):
self.mask_upload.change(
fn=on_checkbox_click,
inputs=[self.mask_upload, self.height_slider, self.width_slider],
outputs=[self.mask_image_group, self.mask_image, self.batch_mask_dir,
outputs=[self.mask_image_group, self.mask_image.background, self.batch_mask_dir,
self.batch_mask_gallery_group, self.batch_mask_gallery],
show_progress=False,
)
@@ -1073,106 +1003,27 @@ class ControlNetUiGroup(object):
event_subscriber(
fn=clear_preview,
inputs=self.use_preview_as_input,
outputs=[self.use_preview_as_input, self.generated_image],
outputs=[self.use_preview_as_input, self.generated_image.background],
show_progress=False
)
def register_multi_images_upload(self):
"""Register callbacks on merge tab multiple images upload."""
self.merge_clear_button.click(
fn=lambda: [],
inputs=[],
outputs=[self.batch_input_gallery],
).then(
fn=lambda x: gr.update(value=x + 1),
inputs=[self.dummy_gradio_update_trigger],
outputs=[self.dummy_gradio_update_trigger],
)
self.mask_merge_clear_button.click(
fn=lambda: [],
inputs=[],
outputs=[self.batch_mask_gallery],
).then(
fn=lambda x: gr.update(value=x + 1),
inputs=[self.dummy_gradio_update_trigger],
outputs=[self.dummy_gradio_update_trigger],
)
def upload_file(files, current_files):
return {file_d["name"] for file_d in current_files} | {
file.name for file in files
}
self.merge_upload_button.upload(
upload_file,
inputs=[self.merge_upload_button, self.batch_input_gallery],
outputs=[self.batch_input_gallery],
queue=False,
).then(
fn=lambda x: gr.update(value=x + 1),
inputs=[self.dummy_gradio_update_trigger],
outputs=[self.dummy_gradio_update_trigger],
)
self.mask_merge_upload_button.upload(
upload_file,
inputs=[self.mask_merge_upload_button, self.batch_mask_gallery],
outputs=[self.batch_mask_gallery],
queue=False,
).then(
fn=lambda x: gr.update(value=x + 1),
inputs=[self.dummy_gradio_update_trigger],
outputs=[self.dummy_gradio_update_trigger],
)
return
def register_core_callbacks(self):
"""Register core callbacks that only involves gradio components defined
within this ui group."""
self.register_webcam_toggle()
self.register_webcam_mirror_toggle()
self.register_refresh_all_models()
self.register_build_sliders()
self.register_shift_preview()
self.register_create_canvas()
self.register_clear_preview()
self.register_multi_images_upload()
self.openpose_editor.register_callbacks(
self.generated_image,
self.use_preview_as_input,
self.model,
)
assert self.type_filter is not None
self.preset_panel.register_callbacks(
self,
self.type_filter,
*[
getattr(self, key)
for key in external_code.ControlNetUnit.infotext_fields()
],
)
if self.is_img2img:
self.register_img2img_same_input()
def register_sd_model_changed(self):
def sd_version_changed(type_filter: str, current_model: str, setting_value: str, setting_name: str):
"""When SD version changes, update model dropdown choices."""
if setting_name != "sd_model_checkpoint":
return gr.update()
filtered_model_list = global_state.get_filtered_controlnet_names(type_filter)
assert len(filtered_model_list) > 0
default_model = filtered_model_list[1] if len(filtered_model_list) > 1 else filtered_model_list[0]
return gr.Dropdown.update(
choices=filtered_model_list,
value=current_model if current_model in filtered_model_list else default_model
)
script_callbacks.on_setting_updated_subscriber(dict(
fn=sd_version_changed,
inputs=[self.type_filter, self.model],
outputs=[self.model],
))
def register_callbacks(self):
"""Register callbacks that involves A1111 context gradio components."""
# Prevent infinite recursion.
@@ -1184,7 +1035,6 @@ class ControlNetUiGroup(object):
self.register_run_annotator()
self.register_sync_batch_dir()
self.register_shift_upload_mask()
self.register_sd_model_changed()
if self.is_img2img:
self.register_shift_crop_input_image()
else:
extensions-builtin/sd_forge_controlnet/lib_controlnet/controlnet_ui/openpose_editor.py
+1 -1
View File
@@ -112,7 +112,7 @@ class OpenposeEditor(object):
self.render_button.click(
fn=render_pose,
inputs=[self.pose_input],
outputs=[generated_image, use_preview_as_input, *self.outputs()],
outputs=[generated_image.background, use_preview_as_input, *self.outputs()],
)
def update_upload_link(model: str) -> Dict:
extensions-builtin/sd_forge_controlnet/lib_controlnet/controlnet_ui/preset.py
-313
View File
@@ -1,313 +0,0 @@
import os
import gradio as gr
from typing import Dict, List
from modules import scripts
from lib_controlnet.infotext import parse_unit, serialize_unit
from lib_controlnet.controlnet_ui.tool_button import ToolButton
from lib_controlnet.logging import logger
from lib_controlnet.external_code import ControlNetUnit, UiControlNetUnit
from lib_controlnet.global_state import get_preprocessor
from modules_forge.supported_preprocessor import Preprocessor
save_symbol = "\U0001f4be" # 💾
delete_symbol = "\U0001f5d1\ufe0f" # 🗑️
refresh_symbol = "\U0001f504" # 🔄
reset_symbol = "\U000021A9" # ↩
NEW_PRESET = "New Preset"
def load_presets(preset_dir: str) -> Dict[str, str]:
if not os.path.exists(preset_dir):
os.makedirs(preset_dir)
return {}
presets = {}
for filename in os.listdir(preset_dir):
if filename.endswith(".txt"):
with open(os.path.join(preset_dir, filename), "r") as f:
name = filename.replace(".txt", "")
if name == NEW_PRESET:
continue
presets[name] = f.read()
return presets
def infer_control_type(module: str) -> str:
preprocessor: Preprocessor = get_preprocessor(module)
assert preprocessor is not None
return preprocessor.tags[0] if preprocessor.tags else "All"
class ControlNetPresetUI(object):
preset_directory = os.path.join(scripts.basedir(), "presets")
presets = load_presets(preset_directory)
def __init__(self, id_prefix: str):
with gr.Row():
self.dropdown = gr.Dropdown(
label="Presets",
show_label=True,
elem_classes=["cnet-preset-dropdown"],
choices=ControlNetPresetUI.dropdown_choices(),
value=NEW_PRESET,
)
self.reset_button = ToolButton(
value=reset_symbol,
elem_classes=["cnet-preset-reset"],
tooltip="Reset preset",
visible=False,
)
self.save_button = ToolButton(
value=save_symbol,
elem_classes=["cnet-preset-save"],
tooltip="Save preset",
)
self.delete_button = ToolButton(
value=delete_symbol,
elem_classes=["cnet-preset-delete"],
tooltip="Delete preset",
)
self.refresh_button = ToolButton(
value=refresh_symbol,
elem_classes=["cnet-preset-refresh"],
tooltip="Refresh preset",
)
with gr.Box(
elem_classes=["popup-dialog", "cnet-preset-enter-name"],
elem_id=f"{id_prefix}_cnet_preset_enter_name",
) as self.name_dialog:
with gr.Row():
self.preset_name = gr.Textbox(
label="Preset name",
show_label=True,
lines=1,
elem_classes=["cnet-preset-name"],
)
self.confirm_preset_name = ToolButton(
value=save_symbol,
elem_classes=["cnet-preset-confirm-name"],
tooltip="Save preset",
)
def register_callbacks(
self,
uigroup,
control_type: gr.Radio,
*ui_states,
):
def init_with_ui_states(*ui_states) -> ControlNetUnit:
return ControlNetUnit(**{
field: value
for field, value in zip(ControlNetUnit.infotext_fields(), ui_states)
})
def apply_preset(name: str, control_type: str, *ui_states):
if name == NEW_PRESET:
return (
gr.update(visible=False),
*(
(gr.skip(),)
* (len(ControlNetUnit.infotext_fields()) + 1)
),
)
assert name in ControlNetPresetUI.presets
infotext = ControlNetPresetUI.presets[name]
preset_unit = parse_unit(infotext)
current_unit = init_with_ui_states(*ui_states)
preset_unit.image = None
current_unit.image = None
# Do not compare module param that are not used in preset.
for module_param in ("processor_res", "threshold_a", "threshold_b"):
if getattr(preset_unit, module_param) == -1:
setattr(current_unit, module_param, -1)
# No update necessary.
if vars(current_unit) == vars(preset_unit):
return (
gr.update(visible=False),
*(
(gr.skip(),)
* (len(ControlNetUnit.infotext_fields()) + 1)
),
)
unit = preset_unit
try:
new_control_type = infer_control_type(unit.module)
except ValueError as e:
logger.error(e)
new_control_type = control_type
if new_control_type != control_type:
uigroup.prevent_next_n_module_update += 1
if preset_unit.module != current_unit.module:
uigroup.prevent_next_n_slider_value_update += 1
if preset_unit.pixel_perfect != current_unit.pixel_perfect:
uigroup.prevent_next_n_slider_value_update += 1
return (
gr.update(visible=True),
gr.update(value=new_control_type),
*[
gr.update(value=value) if value is not None else gr.update()
for field in ControlNetUnit.infotext_fields()
for value in (getattr(unit, field),)
],
)
for element, action in (
(self.dropdown, "change"),
(self.reset_button, "click"),
):
getattr(element, action)(
fn=apply_preset,
inputs=[self.dropdown, control_type, *ui_states],
outputs=[self.delete_button, control_type, *ui_states],
show_progress="hidden",
).then(
fn=lambda: gr.update(visible=False),
inputs=None,
outputs=[self.reset_button],
)
def save_preset(name: str, *ui_states):
if name == NEW_PRESET:
return gr.update(visible=True), gr.update(), gr.update()
ControlNetPresetUI.save_preset(
name, init_with_ui_states(*ui_states)
)
return (
gr.update(), # name dialog
gr.update(choices=ControlNetPresetUI.dropdown_choices(), value=name),
gr.update(visible=False), # Reset button
)
self.save_button.click(
fn=save_preset,
inputs=[self.dropdown, *ui_states],
outputs=[self.name_dialog, self.dropdown, self.reset_button],
show_progress="hidden",
).then(
fn=None,
_js=f"""
(name) => {{
if (name === "{NEW_PRESET}")
popup(gradioApp().getElementById('{self.name_dialog.elem_id}'));
}}""",
inputs=[self.dropdown],
)
def delete_preset(name: str):
ControlNetPresetUI.delete_preset(name)
return gr.Dropdown.update(
choices=ControlNetPresetUI.dropdown_choices(),
value=NEW_PRESET,
), gr.update(visible=False)
self.delete_button.click(
fn=delete_preset,
inputs=[self.dropdown],
outputs=[self.dropdown, self.reset_button],
show_progress="hidden",
)
self.name_dialog.visible = False
def save_new_preset(new_name: str, *ui_states):
if new_name == NEW_PRESET:
logger.warn(f"Cannot save preset with reserved name '{NEW_PRESET}'")
return gr.update(visible=False), gr.update()
ControlNetPresetUI.save_preset(
new_name, init_with_ui_states(*ui_states)
)
return gr.update(visible=False), gr.update(
choices=ControlNetPresetUI.dropdown_choices(), value=new_name
)
self.confirm_preset_name.click(
fn=save_new_preset,
inputs=[self.preset_name, *ui_states],
outputs=[self.name_dialog, self.dropdown],
show_progress="hidden",
).then(fn=None, _js="closePopup")
self.refresh_button.click(
fn=ControlNetPresetUI.refresh_preset,
inputs=None,
outputs=[self.dropdown],
show_progress="hidden",
)
def update_reset_button(preset_name: str, *ui_states):
if preset_name == NEW_PRESET:
return gr.update(visible=False)
infotext = ControlNetPresetUI.presets[preset_name]
preset_unit = parse_unit(infotext)
current_unit = init_with_ui_states(*ui_states)
preset_unit.image = None
current_unit.image = None
# Do not compare module param that are not used in preset.
for module_param in ("processor_res", "threshold_a", "threshold_b"):
if getattr(preset_unit, module_param) == -1:
setattr(current_unit, module_param, -1)
return gr.update(visible=vars(current_unit) != vars(preset_unit))
for ui_state in ui_states:
if isinstance(ui_state, gr.Image):
continue
for action in ("edit", "click", "change", "clear", "release"):
if action == "release" and not isinstance(ui_state, gr.Slider):
continue
if hasattr(ui_state, action):
getattr(ui_state, action)(
fn=update_reset_button,
inputs=[self.dropdown, *ui_states],
outputs=[self.reset_button],
)
@staticmethod
def dropdown_choices() -> List[str]:
return list(ControlNetPresetUI.presets.keys()) + [NEW_PRESET]
@staticmethod
def save_preset(name: str, unit: ControlNetUnit):
infotext = serialize_unit(unit)
with open(
os.path.join(ControlNetPresetUI.preset_directory, f"{name}.txt"), "w"
) as f:
f.write(infotext)
ControlNetPresetUI.presets[name] = infotext
@staticmethod
def delete_preset(name: str):
if name not in ControlNetPresetUI.presets:
return
del ControlNetPresetUI.presets[name]
file = os.path.join(ControlNetPresetUI.preset_directory, f"{name}.txt")
if os.path.exists(file):
os.unlink(file)
@staticmethod
def refresh_preset():
ControlNetPresetUI.presets = load_presets(ControlNetPresetUI.preset_directory)
return gr.update(choices=ControlNetPresetUI.dropdown_choices())
extensions-builtin/sd_forge_controlnet/lib_controlnet/controlnet_ui/tool_button.py
-12
View File
@@ -1,12 +0,0 @@
import gradio as gr
class ToolButton(gr.Button, gr.components.FormComponent):
"""Small button with single emoji as text, fits inside gradio forms"""
def __init__(self, **kwargs):
super().__init__(variant="tool",
elem_classes=kwargs.pop('elem_classes', []) + ["cnet-toolbutton"],
**kwargs)
def get_block_name(self):
return "button"
extensions-builtin/sd_forge_controlnet/lib_controlnet/external_code.py
+2 -47
View File
@@ -155,76 +155,31 @@ class GradioImageMaskPair(TypedDict):
@dataclass
class ControlNetUnit:
"""Represents an entire ControlNet processing unit.
To add a new field to this class
## If the new field can be specified on UI, you need to
- Add a new field of the same name in constructor of `ControlNetUiGroup`
- Initialize the new `ControlNetUiGroup` field in `ControlNetUiGroup.render`
as a Gradio `IOComponent`.
- Add the new `ControlNetUiGroup` field to `unit_args` in
`ControlNetUiGroup.render`. The order of parameters matters.
## If the new field needs to appear in infotext, you need to
- Add a new item in `ControlNetUnit.infotext_fields`.
API-only fields cannot appear in infotext.
"""
# Following fields should only be used in the UI.
# ====== Start of UI only fields ======
# Specifies the input mode for the unit, defaulting to a simple mode.
input_mode: InputMode = InputMode.SIMPLE
# Determines whether to use the preview image as input; defaults to False.
use_preview_as_input: bool = False
# Directory path for batch processing of images.
batch_image_dir: str = ''
# Directory path for batch processing of masks.
batch_mask_dir: str = ''
# Optional list of gallery images for batch input; defaults to None.
batch_input_gallery: Optional[List[str]] = None
# Optional list of gallery masks for batch processing; defaults to None.
batch_mask_gallery: Optional[List[str]] = None
# Holds the preview image as a NumPy array; defaults to None.
generated_image: Optional[np.ndarray] = None
# ====== End of UI only fields ======
# Following fields are used in both the API and the UI.
# Holds the mask image; defaults to None.
mask_image: Optional[GradioImageMaskPair] = None
# Specifies how this unit should be applied in each pass of high-resolution fix.
# Ignored if high-resolution fix is not enabled.
mask_image_fg: Optional[GradioImageMaskPair] = None
hr_option: Union[HiResFixOption, int, str] = HiResFixOption.BOTH
# Indicates whether the unit is enabled; defaults to True.
enabled: bool = True
# Name of the module being used; defaults to "None".
module: str = "None"
# Name of the model being used; defaults to "None".
model: str = "None"
# Weight of the unit in the overall processing; defaults to 1.0.
weight: float = 1.0
# Optional image for input; defaults to None.
image: Optional[GradioImageMaskPair] = None
# Specifies the mode of image resizing; defaults to inner fit.
image_fg: Optional[GradioImageMaskPair] = None
resize_mode: Union[ResizeMode, int, str] = ResizeMode.INNER_FIT
# Resolution for processing by the unit; defaults to -1 (unspecified).
processor_res: int = -1
# Threshold A for processing; defaults to -1 (unspecified).
threshold_a: float = -1
# Threshold B for processing; defaults to -1 (unspecified).
threshold_b: float = -1
# Start value for guidance; defaults to 0.0.
guidance_start: float = 0.0
# End value for guidance; defaults to 1.0.
guidance_end: float = 1.0
# Enables pixel-perfect processing; defaults to False.
pixel_perfect: bool = False
# Control mode for the unit; defaults to balanced.
control_mode: Union[ControlMode, int, str] = ControlMode.BALANCED
# Following fields should only be used in the API.
# ====== Start of API only fields ======
# Whether to save the detected map for this unit; defaults to True.
save_detected_map: bool = True
# ====== End of API only fields ======
@staticmethod
def infotext_fields():
extensions-builtin/sd_forge_controlnet/scripts/controlnet.py
+16 -10
View File
@@ -157,7 +157,7 @@ class ControlNetForForgeOfficial(scripts.Script):
if unit.input_mode == external_code.InputMode.MERGE:
for idx, item in enumerate(unit.batch_input_gallery):
img_path = item['name']
img_path = item[0]
logger.info(f'Try to read image: {img_path}')
img = np.ascontiguousarray(cv2.imread(img_path)[:, :, ::-1]).copy()
mask = None
@@ -197,30 +197,36 @@ class ControlNetForForgeOfficial(scripts.Script):
using_a1111_data = False
unit_image = unit.image
unit_image_fg = unit.image_fg[:, :, 3] if unit.image_fg is not None else None
if unit.use_preview_as_input and unit.generated_image is not None:
image = unit.generated_image
elif unit.image is None:
resize_mode = external_code.resize_mode_from_value(p.resize_mode)
image = HWC3(np.asarray(a1111_i2i_image))
using_a1111_data = True
elif (unit.image['image'] < 5).all() and (unit.image['mask'] > 5).any():
image = unit.image['mask']
elif (unit_image < 5).all() and (unit_image_fg > 5).any():
image = unit_image_fg
else:
image = unit.image['image']
image = unit_image
if not isinstance(image, np.ndarray):
raise ValueError("controlnet is enabled but no input image is given")
image = HWC3(image)
unit_mask_image = unit.mask_image
unit_mask_image_fg = unit.mask_image_fg[:, :, 3] if unit.mask_image_fg is not None else None
if using_a1111_data:
mask = HWC3(np.asarray(a1111_i2i_mask)) if a1111_i2i_mask is not None else None
elif unit.mask_image is not None and (unit.mask_image['image'] > 5).any():
mask = unit.mask_image['image']
elif unit.mask_image is not None and (unit.mask_image['mask'] > 5).any():
mask = unit.mask_image['mask']
elif unit.image is not None and (unit.image['mask'] > 5).any():
mask = unit.image['mask']
elif unit_mask_image_fg is not None and (unit_mask_image_fg > 5).any():
mask = unit_mask_image_fg
elif unit_mask_image is not None and (unit_mask_image > 5).any():
mask = unit_mask_image
elif unit_image_fg is not None and (unit_image_fg > 5).any():
mask = unit_image_fg
else:
mask = None
extensions-builtin/sd_forge_controlnet/style.css
+24 -1
View File
@@ -225,4 +225,27 @@
border-radius: var(--radius-sm);
background: var(--background-fill-primary);
color: var(--block-label-text-color);
}
}
.controlnet_control_type_filter_group label {
background: unset !important;
border: unset !important;
margin-left: -10px !important;
}
.controlnet_control_type_filter_group > span {
display: none !important;
}
.controlnet_control_type_filter_group > .wrap {
margin-top: -20px !important;
}
.cnet-toolbutton {
background: unset !important;
border: unset !important;
}
.range-slider {
margin-top: -8px;
}
extensions-builtin/sd_forge_controlnet/tests/conftest.py
-7
View File
@@ -1,7 +0,0 @@
import os
def pytest_configure(config):
# We don't want to fail on Py.test command line arguments being
# parsed by webui:
os.environ.setdefault("IGNORE_CMD_ARGS_ERRORS", "1")
extensions-builtin/sd_forge_controlnet/tests/images/1girl.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 482 KiB

extensions-builtin/sd_forge_controlnet/tests/images/mask.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 244 B

extensions-builtin/sd_forge_controlnet/tests/images/mask_small.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 226 B

extensions-builtin/sd_forge_controlnet/tests/images/portrait/1.webp
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 20 KiB

extensions-builtin/sd_forge_controlnet/tests/images/portrait/2.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 37 KiB

extensions-builtin/sd_forge_controlnet/tests/images/portrait/3.jpeg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 22 KiB

extensions-builtin/sd_forge_controlnet/tests/images/portrait/4.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 6.4 KiB

extensions-builtin/sd_forge_controlnet/tests/images/portrait/5.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 202 KiB

extensions-builtin/sd_forge_controlnet/tests/images/portrait/6.jpg
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 15 KiB

extensions-builtin/sd_forge_controlnet/tests/web_api/__init__.py
View File
extensions-builtin/sd_forge_controlnet/tests/web_api/detect_test.py
-63
View File
@@ -1,63 +0,0 @@
import pytest
import requests
from typing import List
from .template import (
APITestTemplate,
realistic_girl_face_img,
save_base64,
get_dest_dir,
disable_in_cq,
)
def get_modules() -> List[str]:
return requests.get(APITestTemplate.BASE_URL + "controlnet/module_list").json()[
"module_list"
]
def detect_template(payload, output_name: str):
url = APITestTemplate.BASE_URL + "controlnet/detect"
resp = requests.post(url, json=payload)
assert resp.status_code == 200
resp_json = resp.json()
assert "images" in resp_json
assert len(resp_json["images"]) == len(payload["controlnet_input_images"])
if not APITestTemplate.is_cq_run:
for i, img in enumerate(resp_json["images"]):
if img == "Detect result is not image":
continue
dest = get_dest_dir() / f"{output_name}_{i}.png"
save_base64(img, dest)
return resp_json
@disable_in_cq
@pytest.mark.parametrize("module", get_modules())
def test_detect_all_modules(module: str):
payload = dict(
controlnet_input_images=[realistic_girl_face_img],
controlnet_module=module,
)
detect_template(payload, f"detect_{module}")
def test_detect_simple():
detect_template(
dict(
controlnet_input_images=[realistic_girl_face_img],
controlnet_module="canny", # Canny does not require model download.
),
"simple_detect",
)
def test_detect_multiple_inputs():
detect_template(
dict(
controlnet_input_images=[realistic_girl_face_img, realistic_girl_face_img],
controlnet_module="canny", # Canny does not require model download.
),
"multiple_inputs_detect",
)
extensions-builtin/sd_forge_controlnet/tests/web_api/generation_test.py
-171
View File
@@ -1,171 +0,0 @@
import pytest
from .template import (
APITestTemplate,
girl_img,
mask_img,
disable_in_cq,
get_model,
)
@pytest.mark.parametrize("gen_type", ["img2img", "txt2img"])
def test_no_unit(gen_type):
assert APITestTemplate(
f"test_no_unit{gen_type}",
gen_type,
payload_overrides={},
unit_overrides=[],
input_image=girl_img,
).exec()
@pytest.mark.parametrize("gen_type", ["img2img", "txt2img"])
def test_multiple_iter(gen_type):
assert APITestTemplate(
f"test_multiple_iter{gen_type}",
gen_type,
payload_overrides={"n_iter": 2},
unit_overrides={},
input_image=girl_img,
).exec()
@pytest.mark.parametrize("gen_type", ["img2img", "txt2img"])
def test_batch_size(gen_type):
assert APITestTemplate(
f"test_batch_size{gen_type}",
gen_type,
payload_overrides={"batch_size": 2},
unit_overrides={},
input_image=girl_img,
).exec()
@pytest.mark.parametrize("gen_type", ["img2img", "txt2img"])
def test_2_units(gen_type):
assert APITestTemplate(
f"test_2_units{gen_type}",
gen_type,
payload_overrides={},
unit_overrides=[{}, {}],
input_image=girl_img,
).exec()
@pytest.mark.parametrize("gen_type", ["img2img", "txt2img"])
def test_preprocessor(gen_type):
assert APITestTemplate(
f"test_preprocessor{gen_type}",
gen_type,
payload_overrides={},
unit_overrides={"module": "canny"},
input_image=girl_img,
).exec()
@pytest.mark.parametrize("param_name", ("processor_res", "threshold_a", "threshold_b"))
@pytest.mark.parametrize("gen_type", ["img2img", "txt2img"])
def test_invalid_param(gen_type, param_name):
assert APITestTemplate(
f"test_invalid_param{(gen_type, param_name)}",
gen_type,
payload_overrides={},
unit_overrides={param_name: -1},
input_image=girl_img,
).exec()
@pytest.mark.parametrize("save_map", [True, False])
@pytest.mark.parametrize("gen_type", ["img2img", "txt2img"])
def test_save_map(gen_type, save_map):
assert APITestTemplate(
f"test_save_map{(gen_type, save_map)}",
gen_type,
payload_overrides={},
unit_overrides={"save_detected_map": save_map},
input_image=girl_img,
).exec(expected_output_num=2 if save_map else 1)
@disable_in_cq
def test_masked_controlnet_txt2img():
assert APITestTemplate(
f"test_masked_controlnet_txt2img",
"txt2img",
payload_overrides={},
unit_overrides={
"image": girl_img,
"mask_image": mask_img,
},
).exec()
@disable_in_cq
def test_masked_controlnet_img2img():
assert APITestTemplate(
f"test_masked_controlnet_img2img",
"img2img",
payload_overrides={
"init_images": [girl_img],
},
# Note: Currently you must give ControlNet unit input image to specify
# mask.
# TODO: Fix this for img2img.
unit_overrides={
"image": girl_img,
"mask_image": mask_img,
},
).exec()
@disable_in_cq
def test_txt2img_inpaint():
assert APITestTemplate(
"txt2img_inpaint",
"txt2img",
payload_overrides={},
unit_overrides={
"image": girl_img,
"mask_image": mask_img,
"model": get_model("v11p_sd15_inpaint"),
"module": "inpaint_only",
},
).exec()
@disable_in_cq
def test_img2img_inpaint():
assert APITestTemplate(
"img2img_inpaint",
"img2img",
payload_overrides={
"init_images": [girl_img],
"mask": mask_img,
},
unit_overrides={
"model": get_model("v11p_sd15_inpaint"),
"module": "inpaint_only",
},
).exec()
# Currently failing.
# TODO Fix lama outpaint.
@disable_in_cq
def test_lama_outpaint():
assert APITestTemplate(
"txt2img_lama_outpaint",
"txt2img",
payload_overrides={
"width": 768,
"height": 768,
},
# Outpaint should not need a mask.
unit_overrides={
"image": girl_img,
"model": get_model("v11p_sd15_inpaint"),
"module": "inpaint_only+lama",
"resize_mode": "Resize and Fill", # OUTER_FIT
},
).exec()
extensions-builtin/sd_forge_controlnet/tests/web_api/template.py
-347
View File
@@ -1,347 +0,0 @@
import io
import os
import cv2
import base64
import functools
from typing import Dict, Any, List, Union, Literal, Optional
from pathlib import Path
import datetime
from enum import Enum
import numpy as np
import pytest
import requests
from PIL import Image
def disable_in_cq(func):
"""Skips the decorated test func in CQ run."""
@functools.wraps(func)
def wrapped_func(*args, **kwargs):
if APITestTemplate.is_cq_run:
pytest.skip()
return func(*args, **kwargs)
return wrapped_func
PayloadOverrideType = Dict[str, Any]
timestamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
test_result_dir = Path(__file__).parent / "results" / f"test_result_{timestamp}"
test_expectation_dir = Path(__file__).parent / "expectations"
os.makedirs(test_expectation_dir, exist_ok=True)
resource_dir = Path(__file__).parents[1] / "images"
def get_dest_dir():
if APITestTemplate.is_set_expectation_run:
return test_expectation_dir
else:
return test_result_dir
def save_base64(base64img: str, dest: Path):
Image.open(io.BytesIO(base64.b64decode(base64img.split(",", 1)[0]))).save(dest)
def read_image(img_path: Path) -> str:
img = cv2.imread(str(img_path))
_, bytes = cv2.imencode(".png", img)
encoded_image = base64.b64encode(bytes).decode("utf-8")
return encoded_image
def read_image_dir(img_dir: Path, suffixes=('.png', '.jpg', '.jpeg', '.webp')) -> List[str]:
"""Try read all images in given img_dir."""
img_dir = str(img_dir)
images = []
for filename in os.listdir(img_dir):
if filename.endswith(suffixes):
img_path = os.path.join(img_dir, filename)
try:
images.append(read_image(img_path))
except IOError:
print(f"Error opening {img_path}")
return images
girl_img = read_image(resource_dir / "1girl.png")
mask_img = read_image(resource_dir / "mask.png")
mask_small_img = read_image(resource_dir / "mask_small.png")
portrait_imgs = read_image_dir(resource_dir / "portrait")
realistic_girl_face_img = portrait_imgs[0]
general_negative_prompt = """
(worst quality:2), (low quality:2), (normal quality:2), lowres, normal quality,
((monochrome)), ((grayscale)), skin spots, acnes, skin blemishes, age spot,
backlight,(ugly:1.331), (duplicate:1.331), (morbid:1.21), (mutilated:1.21),
(tranny:1.331), mutated hands, (poorly drawn hands:1.331), blurry, (bad anatomy:1.21),
(bad proportions:1.331), extra limbs, (missing arms:1.331), (extra legs:1.331),
(fused fingers:1.61051), (too many fingers:1.61051), (unclear eyes:1.331), bad hands,
missing fingers, extra digit, bad body, easynegative, nsfw"""
class StableDiffusionVersion(Enum):
"""The version family of stable diffusion model."""
UNKNOWN = 0
SD1x = 1
SD2x = 2
SDXL = 3
sd_version = StableDiffusionVersion(
int(os.environ.get("CONTROLNET_TEST_SD_VERSION", StableDiffusionVersion.SD1x.value))
)
is_full_coverage = os.environ.get("CONTROLNET_TEST_FULL_COVERAGE", None) is not None
class APITestTemplate:
is_set_expectation_run = os.environ.get("CONTROLNET_SET_EXP", "True") == "True"
is_cq_run = os.environ.get("FORGE_CQ_TEST", "False") == "True"
BASE_URL = "http://localhost:7860/"
def __init__(
self,
name: str,
gen_type: Union[Literal["img2img"], Literal["txt2img"]],
payload_overrides: PayloadOverrideType,
unit_overrides: Union[PayloadOverrideType, List[PayloadOverrideType]],
input_image: Optional[str] = None,
):
self.name = name
self.url = APITestTemplate.BASE_URL + "sdapi/v1/" + gen_type
self.payload = {
**(txt2img_payload if gen_type == "txt2img" else img2img_payload),
**payload_overrides,
}
if gen_type == "img2img" and input_image is not None:
self.payload["init_images"] = [input_image]
# CQ runs on CPU. Reduce steps to increase test speed.
if "steps" not in payload_overrides and APITestTemplate.is_cq_run:
self.payload["steps"] = 3
unit_overrides = (
unit_overrides
if isinstance(unit_overrides, (list, tuple))
else [unit_overrides]
)
self.payload["alwayson_scripts"]["ControlNet"]["args"] = [
{
**default_unit,
**unit_override,
**({"image": input_image} if gen_type == "txt2img" and input_image is not None else {}),
}
for unit_override in unit_overrides
]
self.active_unit_count = len(unit_overrides)
def exec(self, *args, **kwargs) -> bool:
if APITestTemplate.is_cq_run:
return self.exec_cq(*args, **kwargs)
else:
return self.exec_local(*args, **kwargs)
def exec_cq(self, expected_output_num: Optional[int] = None, *args, **kwargs) -> bool:
"""Execute test in CQ environment."""
res = requests.post(url=self.url, json=self.payload)
if res.status_code != 200:
print(f"Unexpected status code {res.status_code}")
return False
response = res.json()
if "images" not in response:
print(response.keys())
return False
if expected_output_num is None:
expected_output_num = self.payload["n_iter"] * self.payload["batch_size"] + self.active_unit_count
if len(response["images"]) != expected_output_num:
print(f"{len(response['images'])} != {expected_output_num}")
return False
return True
def exec_local(self, result_only: bool = True, *args, **kwargs) -> bool:
"""Execute test in local environment."""
if not APITestTemplate.is_set_expectation_run:
os.makedirs(test_result_dir, exist_ok=True)
failed = False
response = requests.post(url=self.url, json=self.payload).json()
if "images" not in response:
print(response.keys())
return False
dest_dir = get_dest_dir()
results = response["images"][:1] if result_only else response["images"]
for i, base64image in enumerate(results):
img_file_name = f"{self.name}_{i}.png"
save_base64(base64image, dest_dir / img_file_name)
if not APITestTemplate.is_set_expectation_run:
try:
img1 = cv2.imread(os.path.join(test_expectation_dir, img_file_name))
img2 = cv2.imread(os.path.join(test_result_dir, img_file_name))
except Exception as e:
print(f"Get exception reading imgs: {e}")
failed = True
continue
if img1 is None:
print(f"Warn: No expectation file found {img_file_name}.")
continue
if not expect_same_image(
img1,
img2,
diff_img_path=str(test_result_dir
/ img_file_name.replace(".png", "_diff.png")),
):
failed = True
return not failed
def expect_same_image(img1, img2, diff_img_path: str) -> bool:
# Calculate the difference between the two images
diff = cv2.absdiff(img1, img2)
# Set a threshold to highlight the different pixels
threshold = 30
diff_highlighted = np.where(diff > threshold, 255, 0).astype(np.uint8)
# Assert that the two images are similar within a tolerance
similar = np.allclose(img1, img2, rtol=0.5, atol=1)
if not similar:
# Save the diff_highlighted image to inspect the differences
cv2.imwrite(diff_img_path, diff_highlighted)
matching_pixels = np.isclose(img1, img2, rtol=0.5, atol=1)
similar_in_general = (matching_pixels.sum() / matching_pixels.size) >= 0.95
return similar_in_general
def get_model(model_name: str) -> str:
""" Find an available model with specified model name."""
if model_name.lower() == "none":
return "None"
r = requests.get(APITestTemplate.BASE_URL + "controlnet/model_list")
result = r.json()
if "model_list" not in result:
raise ValueError("No model available")
candidates = [
model
for model in result["model_list"]
if model_name.lower() in model.lower()
]
if not candidates:
raise ValueError("No suitable model available")
return candidates[0]
default_unit = {
"control_mode": 0,
"enabled": True,
"guidance_end": 1,
"guidance_start": 0,
"pixel_perfect": True,
"processor_res": 512,
"resize_mode": 1,
"threshold_a": 64,
"threshold_b": 64,
"weight": 1,
"module": "canny",
"model": get_model("sd15_canny"),
}
img2img_payload = {
"batch_size": 1,
"cfg_scale": 7,
"height": 768,
"width": 512,
"n_iter": 1,
"steps": 10,
"sampler_name": "Euler a",
"prompt": "(masterpiece: 1.3), (highres: 1.3), best quality,",
"negative_prompt": "",
"seed": 42,
"seed_enable_extras": False,
"seed_resize_from_h": 0,
"seed_resize_from_w": 0,
"subseed": -1,
"subseed_strength": 0,
"override_settings": {},
"override_settings_restore_afterwards": False,
"do_not_save_grid": False,
"do_not_save_samples": False,
"s_churn": 0,
"s_min_uncond": 0,
"s_noise": 1,
"s_tmax": None,
"s_tmin": 0,
"script_args": [],
"script_name": None,
"styles": [],
"alwayson_scripts": {"ControlNet": {"args": [default_unit]}},
"denoising_strength": 0.75,
"initial_noise_multiplier": 1,
"inpaint_full_res": 0,
"inpaint_full_res_padding": 32,
"inpainting_fill": 1,
"inpainting_mask_invert": 0,
"mask_blur_x": 4,
"mask_blur_y": 4,
"mask_blur": 4,
"resize_mode": 0,
}
txt2img_payload = {
"alwayson_scripts": {"ControlNet": {"args": [default_unit]}},
"batch_size": 1,
"cfg_scale": 7,
"comments": {},
"disable_extra_networks": False,
"do_not_save_grid": False,
"do_not_save_samples": False,
"enable_hr": False,
"height": 768,
"hr_negative_prompt": "",
"hr_prompt": "",
"hr_resize_x": 0,
"hr_resize_y": 0,
"hr_scale": 2,
"hr_second_pass_steps": 0,
"hr_upscaler": "Latent",
"n_iter": 1,
"negative_prompt": "",
"override_settings": {},
"override_settings_restore_afterwards": True,
"prompt": "(masterpiece: 1.3), (highres: 1.3), best quality,",
"restore_faces": False,
"s_churn": 0.0,
"s_min_uncond": 0,
"s_noise": 1.0,
"s_tmax": None,
"s_tmin": 0.0,
"sampler_name": "Euler a",
"script_args": [],
"script_name": None,
"seed": 42,
"seed_enable_extras": True,
"seed_resize_from_h": -1,
"seed_resize_from_w": -1,
"steps": 10,
"styles": [],
"subseed": -1,
"subseed_strength": 0,
"tiling": False,
"width": 512,
}
extensions-builtin/sd_forge_controlnet_example/preload.py
-6
View File
@@ -1,6 +0,0 @@
def preload(parser):
parser.add_argument(
"--show-controlnet-example",
action="store_true",
help="Show development example extension for ControlNet.",
)
extensions-builtin/sd_forge_controlnet_example/scripts/sd_forge_controlnet_example.py
-160
View File
@@ -1,160 +0,0 @@
# Use --show-controlnet-example to see this extension.
import cv2
import gradio as gr
import torch
from modules import scripts
from modules.shared_cmd_options import cmd_opts
from modules_forge.shared import supported_preprocessors
from modules.modelloader import load_file_from_url
from ldm_patched.modules.controlnet import load_controlnet
from modules_forge.controlnet import apply_controlnet_advanced
from modules_forge.forge_util import numpy_to_pytorch
from modules_forge.shared import controlnet_dir
class ControlNetExampleForge(scripts.Script):
model = None
def title(self):
return "ControlNet Example for Developers"
def show(self, is_img2img):
# make this extension visible in both txt2img and img2img tab.
return scripts.AlwaysVisible
def ui(self, *args, **kwargs):
with gr.Accordion(open=False, label=self.title()):
gr.HTML('This is an example controlnet extension for developers.')
gr.HTML('You see this extension because you used --show-controlnet-example')
input_image = gr.Image(source='upload', type='numpy')
funny_slider = gr.Slider(label='This slider does nothing. It just shows you how to transfer parameters.',
minimum=0.0, maximum=1.0, value=0.5)
return input_image, funny_slider
def process(self, p, *script_args, **kwargs):
input_image, funny_slider = script_args
# This slider does nothing. It just shows you how to transfer parameters.
del funny_slider
if input_image is None:
return
# controlnet_canny_path = load_file_from_url(
# url='https://huggingface.co/lllyasviel/sd_control_collection/resolve/main/sai_xl_canny_256lora.safetensors',
# model_dir=model_dir,
# file_name='sai_xl_canny_256lora.safetensors'
# )
controlnet_canny_path = load_file_from_url(
url='https://huggingface.co/lllyasviel/fav_models/resolve/main/fav/control_v11p_sd15_canny_fp16.safetensors',
model_dir=controlnet_dir,
file_name='control_v11p_sd15_canny_fp16.safetensors'
)
print('The model [control_v11p_sd15_canny_fp16.safetensors] download finished.')
self.model = load_controlnet(controlnet_canny_path)
print('Controlnet loaded.')
return
def process_before_every_sampling(self, p, *script_args, **kwargs):
# This will be called before every sampling.
# If you use highres fix, this will be called twice.
input_image, funny_slider = script_args
if input_image is None or self.model is None:
return
B, C, H, W = kwargs['noise'].shape # latent_shape
height = H * 8
width = W * 8
batch_size = p.batch_size
preprocessor = supported_preprocessors['canny']
# detect control at certain resolution
control_image = preprocessor(
input_image, resolution=512, slider_1=100, slider_2=200, slider_3=None)
# here we just use nearest neighbour to align input shape.
# You may want crop and resize, or crop and fill, or others.
control_image = cv2.resize(
control_image, (width, height), interpolation=cv2.INTER_NEAREST)
# Output preprocessor result. Now called every sampling. Cache in your own way.
p.extra_result_images.append(control_image)
print('Preprocessor Canny finished.')
control_image_bchw = numpy_to_pytorch(control_image).movedim(-1, 1)
unet = p.sd_model.forge_objects.unet
# Unet has input, middle, output blocks, and we can give different weights
# to each layers in all blocks.
# Below is an example for stronger control in middle block.
# This is helpful for some high-res fix passes. (p.is_hr_pass)
positive_advanced_weighting = {
'input': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2],
'middle': [1.0],
'output': [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2]
}
negative_advanced_weighting = {
'input': [0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95, 1.05, 1.15, 1.25],
'middle': [1.05],
'output': [0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85, 0.95, 1.05, 1.15, 1.25]
}
# The advanced_frame_weighting is a weight applied to each image in a batch.
# The length of this list must be same with batch size
# For example, if batch size is 5, the below list is [0.2, 0.4, 0.6, 0.8, 1.0]
# If you view the 5 images as 5 frames in a video, this will lead to
# progressively stronger control over time.
advanced_frame_weighting = [float(i + 1) / float(batch_size) for i in range(batch_size)]
# The advanced_sigma_weighting allows you to dynamically compute control
# weights given diffusion timestep (sigma).
# For example below code can softly make beginning steps stronger than ending steps.
sigma_max = unet.model.model_sampling.sigma_max
sigma_min = unet.model.model_sampling.sigma_min
advanced_sigma_weighting = lambda s: (s - sigma_min) / (sigma_max - sigma_min)
# You can even input a tensor to mask all control injections
# The mask will be automatically resized during inference in UNet.
# The size should be B 1 H W and the H and W are not important
# because they will be resized automatically
advanced_mask_weighting = torch.ones(size=(1, 1, 512, 512))
# But in this simple example we do not use them
positive_advanced_weighting = None
negative_advanced_weighting = None
advanced_frame_weighting = None
advanced_sigma_weighting = None
advanced_mask_weighting = None
unet = apply_controlnet_advanced(unet=unet, controlnet=self.model, image_bchw=control_image_bchw,
strength=0.6, start_percent=0.0, end_percent=0.8,
positive_advanced_weighting=positive_advanced_weighting,
negative_advanced_weighting=negative_advanced_weighting,
advanced_frame_weighting=advanced_frame_weighting,
advanced_sigma_weighting=advanced_sigma_weighting,
advanced_mask_weighting=advanced_mask_weighting)
p.sd_model.forge_objects.unet = unet
# Below codes will add some logs to the texts below the image outputs on UI.
# The extra_generation_params does not influence results.
p.extra_generation_params.update(dict(
controlnet_info='You should see these texts below output images!',
))
return
# Use --show-controlnet-example to see this extension.
if not cmd_opts.show_controlnet_example:
del ControlNetExampleForge
extensions-builtin/soft-inpainting/scripts/soft_inpainting.py
+26 -13
View File
@@ -3,6 +3,7 @@ import gradio as gr
import math
from modules.ui_components import InputAccordion
import modules.scripts as scripts
from modules.torch_utils import float64
class SoftInpaintingSettings:
@@ -57,10 +58,14 @@ def latent_blend(settings, a, b, t):
# NOTE: We use inplace operations wherever possible.
# [4][w][h] to [1][4][w][h]
t2 = t.unsqueeze(0)
# [4][w][h] to [1][1][w][h] - the [4] seem redundant.
t3 = t[0].unsqueeze(0).unsqueeze(0)
if len(t.shape) == 3:
# [4][w][h] to [1][4][w][h]
t2 = t.unsqueeze(0)
# [4][w][h] to [1][1][w][h] - the [4] seem redundant.
t3 = t[0].unsqueeze(0).unsqueeze(0)
else:
t2 = t
t3 = t[:, 0][:, None]
one_minus_t2 = 1 - t2
one_minus_t3 = 1 - t3
@@ -75,13 +80,11 @@ def latent_blend(settings, a, b, t):
# Calculate the magnitude of the interpolated vectors. (We will remove this magnitude.)
# 64-bit operations are used here to allow large exponents.
current_magnitude = torch.norm(image_interp, p=2, dim=1, keepdim=True).to(torch.float64).add_(0.00001)
current_magnitude = torch.norm(image_interp, p=2, dim=1, keepdim=True).to(float64(image_interp)).add_(0.00001)
# Interpolate the powered magnitudes, then un-power them (bring them back to a power of 1).
a_magnitude = torch.norm(a, p=2, dim=1, keepdim=True).to(torch.float64).pow_(
settings.inpaint_detail_preservation) * one_minus_t3
b_magnitude = torch.norm(b, p=2, dim=1, keepdim=True).to(torch.float64).pow_(
settings.inpaint_detail_preservation) * t3
a_magnitude = torch.norm(a, p=2, dim=1, keepdim=True).to(float64(a)).pow_(settings.inpaint_detail_preservation) * one_minus_t3
b_magnitude = torch.norm(b, p=2, dim=1, keepdim=True).to(float64(b)).pow_(settings.inpaint_detail_preservation) * t3
desired_magnitude = a_magnitude
desired_magnitude.add_(b_magnitude).pow_(1 / settings.inpaint_detail_preservation)
del a_magnitude, b_magnitude, t3, one_minus_t3
@@ -104,7 +107,7 @@ def latent_blend(settings, a, b, t):
def get_modified_nmask(settings, nmask, sigma):
"""
Converts a negative mask representing the transparency of the original latent vectors being overlayed
Converts a negative mask representing the transparency of the original latent vectors being overlaid
to a mask that is scaled according to the denoising strength for this step.
Where:
@@ -135,7 +138,10 @@ def apply_adaptive_masks(
from PIL import Image, ImageOps, ImageFilter
# TODO: Bias the blending according to the latent mask, add adjustable parameter for bias control.
latent_mask = nmask[0].float()
if len(nmask.shape) == 3:
latent_mask = nmask[0].float()
else:
latent_mask = nmask[:, 0].float()
# convert the original mask into a form we use to scale distances for thresholding
mask_scalar = 1 - (torch.clamp(latent_mask, min=0, max=1) ** (settings.mask_blend_scale / 2))
mask_scalar = (0.5 * (1 - settings.composite_mask_influence)
@@ -157,7 +163,14 @@ def apply_adaptive_masks(
percentile_min=0.25, percentile_max=0.75, min_width=1)
# The distance at which opacity of original decreases to 50%
half_weighted_distance = settings.composite_difference_threshold * mask_scalar
if len(mask_scalar.shape) == 3:
if mask_scalar.shape[0] > i:
half_weighted_distance = settings.composite_difference_threshold * mask_scalar[i]
else:
half_weighted_distance = settings.composite_difference_threshold * mask_scalar[0]
else:
half_weighted_distance = settings.composite_difference_threshold * mask_scalar
converted_mask = converted_mask / half_weighted_distance
converted_mask = 1 / (1 + converted_mask ** settings.composite_difference_contrast)
@@ -472,7 +485,7 @@ el_ids = SoftInpaintingSettings(
class Script(scripts.Script):
def __init__(self):
self.section = "inpaint"
# self.section = "inpaint"
self.masks_for_overlay = None
self.overlay_images = None
html/extra-networks-copy-path-button.html
+1 -1
View File
@@ -1,5 +1,5 @@
<div class="copy-path-button card-button"
title="Copy path to clipboard"
onclick="extraNetworksCopyCardPath(event, '{filename}')"
onclick="extraNetworksCopyCardPath(event)"
data-clipboard-text="{filename}">
</div>
html/extra-networks-edit-item-button.html
+1 -1
View File
@@ -1,4 +1,4 @@
<div class="edit-button card-button"
title="Edit metadata"
onclick="extraNetworksEditUserMetadata(event, '{tabname}', '{extra_networks_tabname}', '{name}')">
onclick="extraNetworksEditUserMetadata(event, '{tabname}', '{extra_networks_tabname}')">
</div>
html/extra-networks-metadata-button.html
+1 -1
View File
@@ -1,4 +1,4 @@
<div class="metadata-button card-button"
title="Show internal metadata"
onclick="extraNetworksRequestMetadata(event, '{extra_networks_tabname}', '{name}')">
onclick="extraNetworksRequestMetadata(event, '{extra_networks_tabname}')">
</div>
html/extra-networks-pane-dirs.html
+8
View File
@@ -0,0 +1,8 @@
<div class="extra-network-pane-content-dirs">
<div id='{tabname}_{extra_networks_tabname}_dirs' class='extra-network-dirs'>
{dirs_html}
</div>
<div id='{tabname}_{extra_networks_tabname}_cards' class='extra-network-cards'>
{items_html}
</div>
</div>
html/extra-networks-pane-tree.html
+8
View File
@@ -0,0 +1,8 @@
<div class="extra-network-pane-content-tree resize-handle-row">
<div id='{tabname}_{extra_networks_tabname}_tree' class='extra-network-tree' style='flex-basis: {extra_networks_tree_view_default_width}px'>
{tree_html}
</div>
<div id='{tabname}_{extra_networks_tabname}_cards' class='extra-network-cards' style='flex-grow: 1;'>
{items_html}
</div>
</div>
html/extra-networks-pane.html
+45 -19
View File
@@ -1,23 +1,53 @@
<div id='{tabname}_{extra_networks_tabname}_pane' class='extra-network-pane'>
<div id='{tabname}_{extra_networks_tabname}_pane' class='extra-network-pane {tree_view_div_default_display_class}'>
<div class="extra-network-control" id="{tabname}_{extra_networks_tabname}_controls" style="display:none" >
<div class="extra-network-control--search">
<input
id="{tabname}_{extra_networks_tabname}_extra_search"
class="extra-network-control--search-text"
type="search"
placeholder="Filter files"
placeholder="Search"
>
</div>
<small>Sort: </small>
<div
id="{tabname}_{extra_networks_tabname}_extra_sort"
class="extra-network-control--sort"
data-sortmode="{data_sortmode}"
data-sortkey="{data_sortkey}"
id="{tabname}_{extra_networks_tabname}_extra_sort_path"
class="extra-network-control--sort{sort_path_active}"
data-sortkey="default"
title="Sort by path"
onclick="extraNetworksControlSortOnClick(event, '{tabname}', '{extra_networks_tabname}');"
>
<i class="extra-network-control--sort-icon"></i>
<i class="extra-network-control--icon extra-network-control--sort-icon"></i>
</div>
<div
id="{tabname}_{extra_networks_tabname}_extra_sort_name"
class="extra-network-control--sort{sort_name_active}"
data-sortkey="name"
title="Sort by name"
onclick="extraNetworksControlSortOnClick(event, '{tabname}', '{extra_networks_tabname}');"
>
<i class="extra-network-control--icon extra-network-control--sort-icon"></i>
</div>
<div
id="{tabname}_{extra_networks_tabname}_extra_sort_date_created"
class="extra-network-control--sort{sort_date_created_active}"
data-sortkey="date_created"
title="Sort by date created"
onclick="extraNetworksControlSortOnClick(event, '{tabname}', '{extra_networks_tabname}');"
>
<i class="extra-network-control--icon extra-network-control--sort-icon"></i>
</div>
<div
id="{tabname}_{extra_networks_tabname}_extra_sort_date_modified"
class="extra-network-control--sort{sort_date_modified_active}"
data-sortkey="date_modified"
title="Sort by date modified"
onclick="extraNetworksControlSortOnClick(event, '{tabname}', '{extra_networks_tabname}');"
>
<i class="extra-network-control--icon extra-network-control--sort-icon"></i>
</div>
<small> </small>
<div
id="{tabname}_{extra_networks_tabname}_extra_sort_dir"
class="extra-network-control--sort-dir"
@@ -25,15 +55,18 @@
title="Sort ascending"
onclick="extraNetworksControlSortDirOnClick(event, '{tabname}', '{extra_networks_tabname}');"
>
<i class="extra-network-control--sort-dir-icon"></i>
<i class="extra-network-control--icon extra-network-control--sort-dir-icon"></i>
</div>
<small> </small>
<div
id="{tabname}_{extra_networks_tabname}_extra_tree_view"
class="extra-network-control--tree-view {tree_view_btn_extra_class}"
title="Enable Tree View"
onclick="extraNetworksControlTreeViewOnClick(event, '{tabname}', '{extra_networks_tabname}');"
>
<i class="extra-network-control--tree-view-icon"></i>
<i class="extra-network-control--icon extra-network-control--tree-view-icon"></i>
</div>
<div
id="{tabname}_{extra_networks_tabname}_extra_refresh"
@@ -41,15 +74,8 @@
title="Refresh page"
onclick="extraNetworksControlRefreshOnClick(event, '{tabname}', '{extra_networks_tabname}');"
>
<i class="extra-network-control--refresh-icon"></i>
<i class="extra-network-control--icon extra-network-control--refresh-icon"></i>
</div>
</div>
<div class="extra-network-pane-content">
<div id='{tabname}_{extra_networks_tabname}_tree' class='extra-network-tree {tree_view_div_extra_class}'>
{tree_html}
</div>
<div id='{tabname}_{extra_networks_tabname}_cards' class='extra-network-cards'>
{items_html}
</div>
</div>
</div>
{pane_content}
</div>
html/footer.html
+1 -1
View File
@@ -1,7 +1,7 @@
<div>
<a href="{api_docs}">API</a>
 • 
<a href="https://github.com/AUTOMATIC1111/stable-diffusion-webui">Github</a>
<a href="https://github.com/lllyasviel/stable-diffusion-webui-forge">Github</a>
 • 
<a href="https://gradio.app">Gradio</a>
 • 
javascript/aspectRatioOverlay.js
+20 -28
View File
@@ -1,10 +1,8 @@
let currentWidth = null;
let currentHeight = null;
let arFrameTimeout = setTimeout(function() {}, 0);
let currentWidth;
let currentHeight;
let arFrameTimeout;
function dimensionChange(e, is_width, is_height) {
if (is_width) {
currentWidth = e.target.value * 1.0;
}
@@ -22,18 +20,18 @@ function dimensionChange(e, is_width, is_height) {
var tabIndex = get_tab_index('mode_img2img');
if (tabIndex == 0) { // img2img
targetElement = gradioApp().querySelector('#img2img_image div[data-testid=image] img');
targetElement = gradioApp().querySelector('#img2img_image div[data-testid=image] canvas');
} else if (tabIndex == 1) { //Sketch
targetElement = gradioApp().querySelector('#img2img_sketch div[data-testid=image] img');
targetElement = gradioApp().querySelector('#img2img_sketch div[data-testid=image] canvas');
} else if (tabIndex == 2) { // Inpaint
targetElement = gradioApp().querySelector('#img2maskimg div[data-testid=image] img');
targetElement = gradioApp().querySelector('#img2maskimg div[data-testid=image] canvas');
} else if (tabIndex == 3) { // Inpaint sketch
targetElement = gradioApp().querySelector('#inpaint_sketch div[data-testid=image] img');
targetElement = gradioApp().querySelector('#inpaint_sketch div[data-testid=image] canvas');
} else if (tabIndex == 4) { // Inpaint upload
targetElement = gradioApp().querySelector('#img_inpaint_base div[data-testid=image] img');
}
if (targetElement) {
var arPreviewRect = gradioApp().querySelector('#imageARPreview');
if (!arPreviewRect) {
arPreviewRect = document.createElement('div');
@@ -41,26 +39,23 @@ function dimensionChange(e, is_width, is_height) {
gradioApp().appendChild(arPreviewRect);
}
var viewportOffset = targetElement.getBoundingClientRect();
var viewportscale = Math.min(targetElement.clientWidth / targetElement.width, targetElement.clientHeight / targetElement.height);
var viewportscale = Math.min(targetElement.clientWidth / targetElement.naturalWidth, targetElement.clientHeight / targetElement.naturalHeight);
var scaledx = targetElement.width * viewportscale;
var scaledy = targetElement.height * viewportscale;
var scaledx = targetElement.naturalWidth * viewportscale;
var scaledy = targetElement.naturalHeight * viewportscale;
var cleintRectTop = (viewportOffset.top + window.scrollY);
var cleintRectLeft = (viewportOffset.left + window.scrollX);
var cleintRectCentreY = cleintRectTop + (targetElement.clientHeight / 2);
var cleintRectCentreX = cleintRectLeft + (targetElement.clientWidth / 2);
var clientRectTop = (viewportOffset.top + window.scrollY);
var clientRectLeft = (viewportOffset.left + window.scrollX);
var clientRectCentreY = clientRectTop + (targetElement.clientHeight / 2);
var clientRectCentreX = clientRectLeft + (targetElement.clientWidth / 2);
var arscale = Math.min(scaledx / currentWidth, scaledy / currentHeight);
var arscaledx = currentWidth * arscale;
var arscaledy = currentHeight * arscale;
var arRectTop = cleintRectCentreY - (arscaledy / 2);
var arRectLeft = cleintRectCentreX - (arscaledx / 2);
var arRectTop = clientRectCentreY - (arscaledy / 2);
var arRectLeft = clientRectCentreX - (arscaledx / 2);
var arRectWidth = arscaledx;
var arRectHeight = arscaledy;
@@ -75,21 +70,18 @@ function dimensionChange(e, is_width, is_height) {
}, 2000);
arPreviewRect.style.display = 'block';
}
}
onAfterUiUpdate(function() {
var arPreviewRect = gradioApp().querySelector('#imageARPreview');
if (arPreviewRect) {
arPreviewRect.style.display = 'none';
}
var tabImg2img = gradioApp().querySelector("#tab_img2img");
if (tabImg2img) {
var inImg2img = tabImg2img.style.display == "block";
if (inImg2img) {
if (tabImg2img.style.display == "block") {
let inputs = gradioApp().querySelectorAll('input');
inputs.forEach(function(e) {
var is_width = e.parentElement.id == "img2img_width";
javascript/contextMenus.js
+18 -31
View File
@@ -8,9 +8,6 @@ var contextMenuInit = function() {
};
function showContextMenu(event, element, menuEntries) {
let posx = event.clientX + document.body.scrollLeft + document.documentElement.scrollLeft;
let posy = event.clientY + document.body.scrollTop + document.documentElement.scrollTop;
let oldMenu = gradioApp().querySelector('#context-menu');
if (oldMenu) {
oldMenu.remove();
@@ -23,10 +20,8 @@ var contextMenuInit = function() {
contextMenu.style.background = baseStyle.background;
contextMenu.style.color = baseStyle.color;
contextMenu.style.fontFamily = baseStyle.fontFamily;
contextMenu.style.top = posy + 'px';
contextMenu.style.left = posx + 'px';
contextMenu.style.top = event.pageY + 'px';
contextMenu.style.left = event.pageX + 'px';
const contextMenuList = document.createElement('ul');
contextMenuList.className = 'context-menu-items';
@@ -43,21 +38,6 @@ var contextMenuInit = function() {
});
gradioApp().appendChild(contextMenu);
let menuWidth = contextMenu.offsetWidth + 4;
let menuHeight = contextMenu.offsetHeight + 4;
let windowWidth = window.innerWidth;
let windowHeight = window.innerHeight;
if ((windowWidth - posx) < menuWidth) {
contextMenu.style.left = windowWidth - menuWidth + "px";
}
if ((windowHeight - posy) < menuHeight) {
contextMenu.style.top = windowHeight - menuHeight + "px";
}
}
function appendContextMenuOption(targetElementSelector, entryName, entryFunction) {
@@ -107,16 +87,23 @@ var contextMenuInit = function() {
oldMenu.remove();
}
});
gradioApp().addEventListener("contextmenu", function(e) {
let oldMenu = gradioApp().querySelector('#context-menu');
if (oldMenu) {
oldMenu.remove();
}
menuSpecs.forEach(function(v, k) {
if (e.composedPath()[0].matches(k)) {
showContextMenu(e, e.composedPath()[0], v);
e.preventDefault();
['contextmenu', 'touchstart'].forEach((eventType) => {
gradioApp().addEventListener(eventType, function(e) {
let ev = e;
if (eventType.startsWith('touch')) {
if (e.touches.length !== 2) return;
ev = e.touches[0];
}
let oldMenu = gradioApp().querySelector('#context-menu');
if (oldMenu) {
oldMenu.remove();
}
menuSpecs.forEach(function(v, k) {
if (e.composedPath()[0].matches(k)) {
showContextMenu(ev, e.composedPath()[0], v);
e.preventDefault();
}
});
});
});
eventListenerApplied = true;
javascript/dragdrop.js
Vendored
+31 -5
View File
@@ -56,6 +56,15 @@ function eventHasFiles(e) {
return false;
}
function isURL(url) {
try {
const _ = new URL(url);
return true;
} catch {
return false;
}
}
function dragDropTargetIsPrompt(target) {
if (target?.placeholder && target?.placeholder.indexOf("Prompt") >= 0) return true;
if (target?.parentNode?.parentNode?.className?.indexOf("prompt") > 0) return true;
@@ -74,22 +83,39 @@ window.document.addEventListener('dragover', e => {
e.dataTransfer.dropEffect = 'copy';
});
window.document.addEventListener('drop', e => {
window.document.addEventListener('drop', async e => {
const target = e.composedPath()[0];
if (!eventHasFiles(e)) return;
const url = e.dataTransfer.getData('text/uri-list') || e.dataTransfer.getData('text/plain');
if (!eventHasFiles(e) && !isURL(url)) return;
if (dragDropTargetIsPrompt(target)) {
e.stopPropagation();
e.preventDefault();
let prompt_target = get_tab_index('tabs') == 1 ? "img2img_prompt_image" : "txt2img_prompt_image";
const isImg2img = get_tab_index('tabs') == 1;
let prompt_image_target = isImg2img ? "img2img_prompt_image" : "txt2img_prompt_image";
const imgParent = gradioApp().getElementById(prompt_target);
const imgParent = gradioApp().getElementById(prompt_image_target);
const files = e.dataTransfer.files;
const fileInput = imgParent.querySelector('input[type="file"]');
if (fileInput) {
if (eventHasFiles(e) && fileInput) {
fileInput.files = files;
fileInput.dispatchEvent(new Event('change'));
} else if (url) {
try {
const request = await fetch(url);
if (!request.ok) {
console.error('Error fetching URL:', url, request.status);
return;
}
const data = new DataTransfer();
data.items.add(new File([await request.blob()], 'image.png'));
fileInput.files = data.files;
fileInput.dispatchEvent(new Event('change'));
} catch (error) {
console.error('Error fetching URL:', url, error);
return;
}
}
}
javascript/edit-attention.js
+8
View File
@@ -64,6 +64,14 @@ function keyupEditAttention(event) {
selectionEnd++;
}
// deselect surrounding whitespace
while (text[selectionStart] == " " && selectionStart < selectionEnd) {
selectionStart++;
}
while (text[selectionEnd - 1] == " " && selectionEnd > selectionStart) {
selectionEnd--;
}
target.setSelectionRange(selectionStart, selectionEnd);
return true;
}
javascript/extraNetworks.js
+121 -56
View File
@@ -39,12 +39,12 @@ function setupExtraNetworksForTab(tabname) {
// tabname_full = {tabname}_{extra_networks_tabname}
var tabname_full = elem.id;
var search = gradioApp().querySelector("#" + tabname_full + "_extra_search");
var sort_mode = gradioApp().querySelector("#" + tabname_full + "_extra_sort");
var sort_dir = gradioApp().querySelector("#" + tabname_full + "_extra_sort_dir");
var refresh = gradioApp().querySelector("#" + tabname_full + "_extra_refresh");
var currentSort = '';
// If any of the buttons above don't exist, we want to skip this iteration of the loop.
if (!search || !sort_mode || !sort_dir || !refresh) {
if (!search || !sort_dir || !refresh) {
return; // `return` is equivalent of `continue` but for forEach loops.
}
@@ -52,7 +52,7 @@ function setupExtraNetworksForTab(tabname) {
var searchTerm = search.value.toLowerCase();
gradioApp().querySelectorAll('#' + tabname + '_extra_tabs div.card').forEach(function(elem) {
var searchOnly = elem.querySelector('.search_only');
var text = Array.prototype.map.call(elem.querySelectorAll('.search_terms'), function(t) {
var text = Array.prototype.map.call(elem.querySelectorAll('.search_terms, .description'), function(t) {
return t.textContent.toLowerCase();
}).join(" ");
@@ -71,42 +71,46 @@ function setupExtraNetworksForTab(tabname) {
};
var applySort = function(force) {
var cards = gradioApp().querySelectorAll('#' + tabname + '_extra_tabs div.card');
var cards = gradioApp().querySelectorAll('#' + tabname_full + ' div.card');
var parent = gradioApp().querySelector('#' + tabname_full + "_cards");
var reverse = sort_dir.dataset.sortdir == "Descending";
var sortKey = sort_mode.dataset.sortmode.toLowerCase().replace("sort", "").replaceAll(" ", "_").replace(/_+$/, "").trim() || "name";
sortKey = "sort" + sortKey.charAt(0).toUpperCase() + sortKey.slice(1);
var sortKeyStore = sortKey + "-" + (reverse ? "Descending" : "Ascending") + "-" + cards.length;
var activeSearchElem = gradioApp().querySelector('#' + tabname_full + "_controls .extra-network-control--sort.extra-network-control--enabled");
var sortKey = activeSearchElem ? activeSearchElem.dataset.sortkey : "default";
var sortKeyDataField = "sort" + sortKey.charAt(0).toUpperCase() + sortKey.slice(1);
var sortKeyStore = sortKey + "-" + sort_dir.dataset.sortdir + "-" + cards.length;
if (sortKeyStore == sort_mode.dataset.sortkey && !force) {
if (sortKeyStore == currentSort && !force) {
return;
}
sort_mode.dataset.sortkey = sortKeyStore;
currentSort = sortKeyStore;
cards.forEach(function(card) {
card.originalParentElement = card.parentElement;
});
var sortedCards = Array.from(cards);
sortedCards.sort(function(cardA, cardB) {
var a = cardA.dataset[sortKey];
var b = cardB.dataset[sortKey];
var a = cardA.dataset[sortKeyDataField];
var b = cardB.dataset[sortKeyDataField];
if (!isNaN(a) && !isNaN(b)) {
return parseInt(a) - parseInt(b);
}
return (a < b ? -1 : (a > b ? 1 : 0));
});
if (reverse) {
sortedCards.reverse();
}
cards.forEach(function(card) {
card.remove();
});
parent.innerHTML = '';
var frag = document.createDocumentFragment();
sortedCards.forEach(function(card) {
card.originalParentElement.appendChild(card);
frag.appendChild(card);
});
parent.appendChild(frag);
};
search.addEventListener("input", applyFilter);
search.addEventListener("input", function() {
applyFilter();
});
applySort();
applyFilter();
extraNetworksApplySort[tabname_full] = applySort;
@@ -272,6 +276,15 @@ function saveCardPreview(event, tabname, filename) {
event.preventDefault();
}
function extraNetworksSearchButton(tabname, extra_networks_tabname, event) {
var searchTextarea = gradioApp().querySelector("#" + tabname + "_" + extra_networks_tabname + "_extra_search");
var button = event.target;
var text = button.classList.contains("search-all") ? "" : button.textContent.trim();
searchTextarea.value = text;
updateInput(searchTextarea);
}
function extraNetworksTreeProcessFileClick(event, btn, tabname, extra_networks_tabname) {
/**
* Processes `onclick` events when user clicks on files in tree.
@@ -290,7 +303,7 @@ function extraNetworksTreeProcessDirectoryClick(event, btn, tabname, extra_netwo
* Processes `onclick` events when user clicks on directories in tree.
*
* Here is how the tree reacts to clicks for various states:
* unselected unopened directory: Diretory is selected and expanded.
* unselected unopened directory: Directory is selected and expanded.
* unselected opened directory: Directory is selected.
* selected opened directory: Directory is collapsed and deselected.
* chevron is clicked: Directory is expanded or collapsed. Selected state unchanged.
@@ -383,36 +396,17 @@ function extraNetworksTreeOnClick(event, tabname, extra_networks_tabname) {
}
function extraNetworksControlSortOnClick(event, tabname, extra_networks_tabname) {
/**
* Handles `onclick` events for the Sort Mode button.
*
* Modifies the data attributes of the Sort Mode button to cycle between
* various sorting modes.
*
* @param event The generated event.
* @param tabname The name of the active tab in the sd webui. Ex: txt2img, img2img, etc.
* @param extra_networks_tabname The id of the active extraNetworks tab. Ex: lora, checkpoints, etc.
*/
var curr_mode = event.currentTarget.dataset.sortmode;
var el_sort_dir = gradioApp().querySelector("#" + tabname + "_" + extra_networks_tabname + "_extra_sort_dir");
var sort_dir = el_sort_dir.dataset.sortdir;
if (curr_mode == "path") {
event.currentTarget.dataset.sortmode = "name";
event.currentTarget.dataset.sortkey = "sortName-" + sort_dir + "-640";
event.currentTarget.setAttribute("title", "Sort by filename");
} else if (curr_mode == "name") {
event.currentTarget.dataset.sortmode = "date_created";
event.currentTarget.dataset.sortkey = "sortDate_created-" + sort_dir + "-640";
event.currentTarget.setAttribute("title", "Sort by date created");
} else if (curr_mode == "date_created") {
event.currentTarget.dataset.sortmode = "date_modified";
event.currentTarget.dataset.sortkey = "sortDate_modified-" + sort_dir + "-640";
event.currentTarget.setAttribute("title", "Sort by date modified");
} else {
event.currentTarget.dataset.sortmode = "path";
event.currentTarget.dataset.sortkey = "sortPath-" + sort_dir + "-640";
event.currentTarget.setAttribute("title", "Sort by path");
}
/** Handles `onclick` events for Sort Mode buttons. */
var self = event.currentTarget;
var parent = event.currentTarget.parentElement;
parent.querySelectorAll('.extra-network-control--sort').forEach(function(x) {
x.classList.remove('extra-network-control--enabled');
});
self.classList.add('extra-network-control--enabled');
applyExtraNetworkSort(tabname + "_" + extra_networks_tabname);
}
@@ -447,8 +441,12 @@ function extraNetworksControlTreeViewOnClick(event, tabname, extra_networks_tabn
* @param tabname The name of the active tab in the sd webui. Ex: txt2img, img2img, etc.
* @param extra_networks_tabname The id of the active extraNetworks tab. Ex: lora, checkpoints, etc.
*/
gradioApp().getElementById(tabname + "_" + extra_networks_tabname + "_tree").classList.toggle("hidden");
event.currentTarget.classList.toggle("extra-network-control--enabled");
var button = event.currentTarget;
button.classList.toggle("extra-network-control--enabled");
var show = !button.classList.contains("extra-network-control--enabled");
var pane = gradioApp().getElementById(tabname + "_" + extra_networks_tabname + "_pane");
pane.classList.toggle("extra-network-dirs-hidden", show);
}
function extraNetworksControlRefreshOnClick(event, tabname, extra_networks_tabname) {
@@ -509,12 +507,76 @@ function popupId(id) {
popup(storedPopupIds[id]);
}
function extraNetworksFlattenMetadata(obj) {
const result = {};
// Convert any stringified JSON objects to actual objects
for (const key of Object.keys(obj)) {
if (typeof obj[key] === 'string') {
try {
const parsed = JSON.parse(obj[key]);
if (parsed && typeof parsed === 'object') {
obj[key] = parsed;
}
} catch (error) {
continue;
}
}
}
// Flatten the object
for (const key of Object.keys(obj)) {
if (typeof obj[key] === 'object' && obj[key] !== null) {
const nested = extraNetworksFlattenMetadata(obj[key]);
for (const nestedKey of Object.keys(nested)) {
result[`${key}/${nestedKey}`] = nested[nestedKey];
}
} else {
result[key] = obj[key];
}
}
// Special case for handling modelspec keys
for (const key of Object.keys(result)) {
if (key.startsWith("modelspec.")) {
result[key.replaceAll(".", "/")] = result[key];
delete result[key];
}
}
// Add empty keys to designate hierarchy
for (const key of Object.keys(result)) {
const parts = key.split("/");
for (let i = 1; i < parts.length; i++) {
const parent = parts.slice(0, i).join("/");
if (!result[parent]) {
result[parent] = "";
}
}
}
return result;
}
function extraNetworksShowMetadata(text) {
try {
let parsed = JSON.parse(text);
if (parsed && typeof parsed === 'object') {
parsed = extraNetworksFlattenMetadata(parsed);
const table = createVisualizationTable(parsed, 0);
popup(table);
return;
}
} catch (error) {
console.error(error);
}
var elem = document.createElement('pre');
elem.classList.add('popup-metadata');
elem.textContent = text;
popup(elem);
return;
}
function requestGet(url, data, handler, errorHandler) {
@@ -543,16 +605,18 @@ function requestGet(url, data, handler, errorHandler) {
xhr.send(js);
}
function extraNetworksCopyCardPath(event, path) {
navigator.clipboard.writeText(path);
function extraNetworksCopyCardPath(event) {
navigator.clipboard.writeText(event.target.getAttribute("data-clipboard-text"));
event.stopPropagation();
}
function extraNetworksRequestMetadata(event, extraPage, cardName) {
function extraNetworksRequestMetadata(event, extraPage) {
var showError = function() {
extraNetworksShowMetadata("there was an error getting metadata");
};
var cardName = event.target.parentElement.parentElement.getAttribute("data-name");
requestGet("./sd_extra_networks/metadata", {page: extraPage, item: cardName}, function(data) {
if (data && data.metadata) {
extraNetworksShowMetadata(data.metadata);
@@ -566,7 +630,7 @@ function extraNetworksRequestMetadata(event, extraPage, cardName) {
var extraPageUserMetadataEditors = {};
function extraNetworksEditUserMetadata(event, tabname, extraPage, cardName) {
function extraNetworksEditUserMetadata(event, tabname, extraPage) {
var id = tabname + '_' + extraPage + '_edit_user_metadata';
var editor = extraPageUserMetadataEditors[id];
@@ -578,6 +642,7 @@ function extraNetworksEditUserMetadata(event, tabname, extraPage, cardName) {
extraPageUserMetadataEditors[id] = editor;
}
var cardName = event.target.parentElement.parentElement.getAttribute("data-name");
editor.nameTextarea.value = cardName;
updateInput(editor.nameTextarea);
javascript/gradio.js
+7
View File
@@ -0,0 +1,7 @@
// added to fix a weird error in gradio 4.19 at page load
Object.defineProperty(Array.prototype, 'toLowerCase', {
value: function() {
return this;
}
});
javascript/imageviewer.js
+23 -17
View File
@@ -6,6 +6,8 @@ function closeModal() {
function showModal(event) {
const source = event.target || event.srcElement;
const modalImage = gradioApp().getElementById("modalImage");
const modalToggleLivePreviewBtn = gradioApp().getElementById("modal_toggle_live_preview");
modalToggleLivePreviewBtn.innerHTML = opts.js_live_preview_in_modal_lightbox ? "&#x1F5C7;" : "&#x1F5C6;";
const lb = gradioApp().getElementById("lightboxModal");
modalImage.src = source.src;
if (modalImage.style.display === 'none') {
@@ -51,14 +53,7 @@ function modalImageSwitch(offset) {
var galleryButtons = all_gallery_buttons();
if (galleryButtons.length > 1) {
var currentButton = selected_gallery_button();
var result = -1;
galleryButtons.forEach(function(v, i) {
if (v == currentButton) {
result = i;
}
});
var result = selected_gallery_index();
if (result != -1) {
var nextButton = galleryButtons[negmod((result + offset), galleryButtons.length)];
@@ -131,19 +126,15 @@ function setupImageForLightbox(e) {
e.style.cursor = 'pointer';
e.style.userSelect = 'none';
var isFirefox = navigator.userAgent.toLowerCase().indexOf('firefox') > -1;
// For Firefox, listening on click first switched to next image then shows the lightbox.
// If you know how to fix this without switching to mousedown event, please.
// For other browsers the event is click to make it possiblr to drag picture.
var event = isFirefox ? 'mousedown' : 'click';
e.addEventListener(event, function(evt) {
e.addEventListener('mousedown', function(evt) {
if (evt.button == 1) {
open(evt.target.src);
evt.preventDefault();
return;
}
}, true);
e.addEventListener('click', function(evt) {
if (!opts.js_modal_lightbox || evt.button != 0) return;
modalZoomSet(gradioApp().getElementById('modalImage'), opts.js_modal_lightbox_initially_zoomed);
@@ -163,6 +154,13 @@ function modalZoomToggle(event) {
event.stopPropagation();
}
function modalLivePreviewToggle(event) {
const modalToggleLivePreview = gradioApp().getElementById("modal_toggle_live_preview");
opts.js_live_preview_in_modal_lightbox = !opts.js_live_preview_in_modal_lightbox;
modalToggleLivePreview.innerHTML = opts.js_live_preview_in_modal_lightbox ? "&#x1F5C7;" : "&#x1F5C6;";
event.stopPropagation();
}
function modalTileImageToggle(event) {
const modalImage = gradioApp().getElementById("modalImage");
const modal = gradioApp().getElementById("lightboxModal");
@@ -179,7 +177,7 @@ function modalTileImageToggle(event) {
}
onAfterUiUpdate(function() {
var fullImg_preview = gradioApp().querySelectorAll('.gradio-gallery > div > img');
var fullImg_preview = gradioApp().querySelectorAll('.gradio-gallery > button > button > img');
if (fullImg_preview != null) {
fullImg_preview.forEach(setupImageForLightbox);
}
@@ -220,6 +218,14 @@ document.addEventListener("DOMContentLoaded", function() {
modalSave.title = "Save Image(s)";
modalControls.appendChild(modalSave);
const modalToggleLivePreview = document.createElement('span');
modalToggleLivePreview.className = 'modalToggleLivePreview cursor';
modalToggleLivePreview.id = "modal_toggle_live_preview";
modalToggleLivePreview.innerHTML = "&#x1F5C6;";
modalToggleLivePreview.onclick = modalLivePreviewToggle;
modalToggleLivePreview.title = "Toggle live preview";
modalControls.appendChild(modalToggleLivePreview);
const modalClose = document.createElement('span');
modalClose.className = 'modalClose cursor';
modalClose.innerHTML = '&times;';
javascript/profilerVisualization.js
+113 -92
View File
@@ -33,120 +33,141 @@ function createRow(table, cellName, items) {
return res;
}
function showProfile(path, cutoff = 0.05) {
requestGet(path, {}, function(data) {
var table = document.createElement('table');
table.className = 'popup-table';
function createVisualizationTable(data, cutoff = 0, sort = "") {
var table = document.createElement('table');
table.className = 'popup-table';
data.records['total'] = data.total;
var keys = Object.keys(data.records).sort(function(a, b) {
return data.records[b] - data.records[a];
var keys = Object.keys(data);
if (sort === "number") {
keys = keys.sort(function(a, b) {
return data[b] - data[a];
});
var items = keys.map(function(x) {
return {key: x, parts: x.split('/'), time: data.records[x]};
} else {
keys = keys.sort();
}
var items = keys.map(function(x) {
return {key: x, parts: x.split('/'), value: data[x]};
});
var maxLength = items.reduce(function(a, b) {
return Math.max(a, b.parts.length);
}, 0);
var cols = createRow(
table,
'th',
[
cutoff === 0 ? 'key' : 'record',
cutoff === 0 ? 'value' : 'seconds'
]
);
cols[0].colSpan = maxLength;
function arraysEqual(a, b) {
return !(a < b || b < a);
}
var addLevel = function(level, parent, hide) {
var matching = items.filter(function(x) {
return x.parts[level] && !x.parts[level + 1] && arraysEqual(x.parts.slice(0, level), parent);
});
var maxLength = items.reduce(function(a, b) {
return Math.max(a, b.parts.length);
}, 0);
var cols = createRow(table, 'th', ['record', 'seconds']);
cols[0].colSpan = maxLength;
function arraysEqual(a, b) {
return !(a < b || b < a);
if (sort === "number") {
matching = matching.sort(function(a, b) {
return b.value - a.value;
});
} else {
matching = matching.sort();
}
var othersTime = 0;
var othersList = [];
var othersRows = [];
var childrenRows = [];
matching.forEach(function(x) {
var visible = (cutoff === 0 && !hide) || (x.value >= cutoff && !hide);
var addLevel = function(level, parent, hide) {
var matching = items.filter(function(x) {
return x.parts[level] && !x.parts[level + 1] && arraysEqual(x.parts.slice(0, level), parent);
});
var sorted = matching.sort(function(a, b) {
return b.time - a.time;
});
var othersTime = 0;
var othersList = [];
var othersRows = [];
var childrenRows = [];
sorted.forEach(function(x) {
var visible = x.time >= cutoff && !hide;
var cells = [];
for (var i = 0; i < maxLength; i++) {
cells.push(x.parts[i]);
}
cells.push(cutoff === 0 ? x.value : x.value.toFixed(3));
var cols = createRow(table, 'td', cells);
for (i = 0; i < level; i++) {
cols[i].className = 'muted';
}
var cells = [];
for (var i = 0; i < maxLength; i++) {
cells.push(x.parts[i]);
}
cells.push(x.time.toFixed(3));
var cols = createRow(table, 'td', cells);
for (i = 0; i < level; i++) {
cols[i].className = 'muted';
}
var tr = cols[0].parentNode;
if (!visible) {
tr.classList.add("hidden");
}
var tr = cols[0].parentNode;
if (!visible) {
tr.classList.add("hidden");
}
if (x.time >= cutoff) {
childrenRows.push(tr);
} else {
othersTime += x.time;
othersList.push(x.parts[level]);
othersRows.push(tr);
}
var children = addLevel(level + 1, parent.concat([x.parts[level]]), true);
if (children.length > 0) {
var cell = cols[level];
var onclick = function() {
cell.classList.remove("link");
cell.removeEventListener("click", onclick);
children.forEach(function(x) {
x.classList.remove("hidden");
});
};
cell.classList.add("link");
cell.addEventListener("click", onclick);
}
});
if (othersTime > 0) {
var cells = [];
for (var i = 0; i < maxLength; i++) {
cells.push(parent[i]);
}
cells.push(othersTime.toFixed(3));
cells[level] = 'others';
var cols = createRow(table, 'td', cells);
for (i = 0; i < level; i++) {
cols[i].className = 'muted';
}
if (cutoff === 0 || x.value >= cutoff) {
childrenRows.push(tr);
} else {
othersTime += x.value;
othersList.push(x.parts[level]);
othersRows.push(tr);
}
var children = addLevel(level + 1, parent.concat([x.parts[level]]), true);
if (children.length > 0) {
var cell = cols[level];
var tr = cell.parentNode;
var onclick = function() {
tr.classList.add("hidden");
cell.classList.remove("link");
cell.removeEventListener("click", onclick);
othersRows.forEach(function(x) {
children.forEach(function(x) {
x.classList.remove("hidden");
});
};
cell.title = othersList.join(", ");
cell.classList.add("link");
cell.addEventListener("click", onclick);
}
});
if (hide) {
tr.classList.add("hidden");
}
childrenRows.push(tr);
if (othersTime > 0) {
var cells = [];
for (var i = 0; i < maxLength; i++) {
cells.push(parent[i]);
}
cells.push(othersTime.toFixed(3));
cells[level] = 'others';
var cols = createRow(table, 'td', cells);
for (i = 0; i < level; i++) {
cols[i].className = 'muted';
}
return childrenRows;
};
var cell = cols[level];
var tr = cell.parentNode;
var onclick = function() {
tr.classList.add("hidden");
cell.classList.remove("link");
cell.removeEventListener("click", onclick);
othersRows.forEach(function(x) {
x.classList.remove("hidden");
});
};
addLevel(0, []);
cell.title = othersList.join(", ");
cell.classList.add("link");
cell.addEventListener("click", onclick);
if (hide) {
tr.classList.add("hidden");
}
childrenRows.push(tr);
}
return childrenRows;
};
addLevel(0, []);
return table;
}
function showProfile(path, cutoff = 0.05) {
requestGet(path, {}, function(data) {
data.records['total'] = data.total;
const table = createVisualizationTable(data.records, cutoff, "number");
popup(table);
});
}
javascript/progressbar.js
+22
View File
@@ -76,6 +76,26 @@ function requestProgress(id_task, progressbarContainer, gallery, atEnd, onProgre
var dateStart = new Date();
var wasEverActive = false;
var parentProgressbar = progressbarContainer.parentNode;
var wakeLock = null;
var requestWakeLock = async function() {
if (!opts.prevent_screen_sleep_during_generation || wakeLock) return;
try {
wakeLock = await navigator.wakeLock.request('screen');
} catch (err) {
console.error('Wake Lock is not supported.');
}
};
var releaseWakeLock = async function() {
if (!opts.prevent_screen_sleep_during_generation || !wakeLock) return;
try {
await wakeLock.release();
wakeLock = null;
} catch (err) {
console.error('Wake Lock release failed', err);
}
};
var divProgress = document.createElement('div');
divProgress.className = 'progressDiv';
@@ -89,6 +109,7 @@ function requestProgress(id_task, progressbarContainer, gallery, atEnd, onProgre
var livePreview = null;
var removeProgressBar = function() {
releaseWakeLock();
if (!divProgress) return;
setTitle("");
@@ -100,6 +121,7 @@ function requestProgress(id_task, progressbarContainer, gallery, atEnd, onProgre
};
var funProgress = function(id_task) {
requestWakeLock();
request("./internal/progress", {id_task: id_task, live_preview: false}, function(res) {
if (res.completed) {
removeProgressBar();
javascript/resizeHandle.js
+50 -12
View File
@@ -2,6 +2,7 @@
const GRADIO_MIN_WIDTH = 320;
const PAD = 16;
const DEBOUNCE_TIME = 100;
const DOUBLE_TAP_DELAY = 200; //ms
const R = {
tracking: false,
@@ -10,6 +11,7 @@
leftCol: null,
leftColStartWidth: null,
screenX: null,
lastTapTime: null,
};
let resizeTimer;
@@ -20,6 +22,9 @@
}
function displayResizeHandle(parent) {
if (!parent.needHideOnMoblie) {
return true;
}
if (window.innerWidth < GRADIO_MIN_WIDTH * 2 + PAD * 4) {
parent.style.display = 'flex';
parent.resizeHandle.style.display = "none";
@@ -39,7 +44,7 @@
const ratio = newParentWidth / oldParentWidth;
const newWidthL = Math.max(Math.floor(ratio * widthL), GRADIO_MIN_WIDTH);
const newWidthL = Math.max(Math.floor(ratio * widthL), parent.minLeftColWidth);
setLeftColGridTemplate(parent, newWidthL);
R.parentWidth = newParentWidth;
@@ -47,6 +52,14 @@
}
function setup(parent) {
function onDoubleClick(evt) {
evt.preventDefault();
evt.stopPropagation();
parent.style.gridTemplateColumns = parent.style.originalGridTemplateColumns;
}
const leftCol = parent.firstElementChild;
const rightCol = parent.lastElementChild;
@@ -54,7 +67,24 @@
parent.style.display = 'grid';
parent.style.gap = '0';
const gridTemplateColumns = `${parent.children[0].style.flexGrow}fr ${PAD}px ${parent.children[1].style.flexGrow}fr`;
let leftColTemplate = "";
if (parent.children[0].style.flexGrow) {
leftColTemplate = `${parent.children[0].style.flexGrow}fr`;
parent.minLeftColWidth = GRADIO_MIN_WIDTH;
parent.minRightColWidth = GRADIO_MIN_WIDTH;
parent.needHideOnMoblie = true;
} else {
leftColTemplate = parent.children[0].style.flexBasis;
parent.minLeftColWidth = parent.children[0].style.flexBasis.slice(0, -2) / 2;
parent.minRightColWidth = 0;
parent.needHideOnMoblie = false;
}
if (!leftColTemplate) {
leftColTemplate = '1fr';
}
const gridTemplateColumns = `${leftColTemplate} ${PAD}px ${parent.children[1].style.flexGrow}fr`;
parent.style.gridTemplateColumns = gridTemplateColumns;
parent.style.originalGridTemplateColumns = gridTemplateColumns;
@@ -69,6 +99,14 @@
if (evt.button !== 0) return;
} else {
if (evt.changedTouches.length !== 1) return;
const currentTime = new Date().getTime();
if (R.lastTapTime && currentTime - R.lastTapTime <= DOUBLE_TAP_DELAY) {
onDoubleClick(evt);
return;
}
R.lastTapTime = currentTime;
}
evt.preventDefault();
@@ -89,12 +127,7 @@
});
});
resizeHandle.addEventListener('dblclick', (evt) => {
evt.preventDefault();
evt.stopPropagation();
parent.style.gridTemplateColumns = parent.style.originalGridTemplateColumns;
});
resizeHandle.addEventListener('dblclick', onDoubleClick);
afterResize(parent);
}
@@ -119,7 +152,7 @@
} else {
delta = R.screenX - evt.changedTouches[0].screenX;
}
const leftColWidth = Math.max(Math.min(R.leftColStartWidth - delta, R.parent.offsetWidth - GRADIO_MIN_WIDTH - PAD), GRADIO_MIN_WIDTH);
const leftColWidth = Math.max(Math.min(R.leftColStartWidth - delta, R.parent.offsetWidth - R.parent.minRightColWidth - PAD), R.parent.minLeftColWidth);
setLeftColGridTemplate(R.parent, leftColWidth);
}
});
@@ -158,10 +191,15 @@
setupResizeHandle = setup;
})();
onUiLoaded(function() {
function setupAllResizeHandles() {
for (var elem of gradioApp().querySelectorAll('.resize-handle-row')) {
if (!elem.querySelector('.resize-handle')) {
if (!elem.querySelector('.resize-handle') && !elem.children[0].classList.contains("hidden")) {
setupResizeHandle(elem);
}
}
});
}
onUiLoaded(setupAllResizeHandles);
javascript/ui.js
+26 -29
View File
@@ -26,13 +26,18 @@ function selected_gallery_index() {
return all_gallery_buttons().findIndex(elem => elem.classList.contains('selected'));
}
function gallery_container_buttons(gallery_container) {
return gradioApp().querySelectorAll(`#${gallery_container} .thumbnail-item.thumbnail-small`);
}
function selected_gallery_index_id(gallery_container) {
return Array.from(gallery_container_buttons(gallery_container)).findIndex(elem => elem.classList.contains('selected'));
}
function extract_image_from_gallery(gallery) {
if (gallery.length == 0) {
return [null];
}
if (gallery.length == 1) {
return [gallery[0]];
}
var index = selected_gallery_index();
@@ -41,7 +46,7 @@ function extract_image_from_gallery(gallery) {
index = 0;
}
return [gallery[index]];
return [[gallery[index]]];
}
window.args_to_array = Array.from; // Compatibility with e.g. extensions that may expect this to be around
@@ -113,14 +118,6 @@ function get_img2img_tab_index() {
function create_submit_args(args) {
var res = Array.from(args);
// As it is currently, txt2img and img2img send back the previous output args (txt2img_gallery, generation_info, html_info) whenever you generate a new image.
// This can lead to uploading a huge gallery of previously generated images, which leads to an unnecessary delay between submitting and beginning to generate.
// I don't know why gradio is sending outputs along with inputs, but we can prevent sending the image gallery here, which seems to be an issue for some.
// If gradio at some point stops sending outputs, this may break something
if (Array.isArray(res[res.length - 3])) {
res[res.length - 3] = null;
}
return res;
}
@@ -141,11 +138,10 @@ function showSubmitInterruptingPlaceholder(tabname) {
function showRestoreProgressButton(tabname, show) {
var button = gradioApp().getElementById(tabname + "_restore_progress");
if (!button) return;
button.style.display = show ? "flex" : "none";
button.style.setProperty('display', show ? 'flex' : 'none', 'important');
}
function submit() {
function submit(args) {
showSubmitButtons('txt2img', false);
var id = randomId();
@@ -157,22 +153,22 @@ function submit() {
showRestoreProgressButton('txt2img', false);
});
var res = create_submit_args(arguments);
var res = create_submit_args(args);
res[0] = id;
return res;
}
function submit_txt2img_upscale() {
var res = submit(...arguments);
function submit_txt2img_upscale(args) {
var res = submit(...args);
res[2] = selected_gallery_index();
return res;
}
function submit_img2img() {
function submit_img2img(args) {
showSubmitButtons('img2img', false);
var id = randomId();
@@ -184,15 +180,14 @@ function submit_img2img() {
showRestoreProgressButton('img2img', false);
});
var res = create_submit_args(arguments);
var res = create_submit_args(args);
res[0] = id;
res[1] = get_tab_index('mode_img2img');
return res;
}
function submit_extras() {
function submit_extras(args) {
showSubmitButtons('extras', false);
var id = randomId();
@@ -201,11 +196,10 @@ function submit_extras() {
showSubmitButtons('extras', true);
});
var res = create_submit_args(arguments);
var res = create_submit_args(args);
res[0] = id;
console.log(res);
return res;
}
@@ -214,6 +208,7 @@ function restoreProgressTxt2img() {
var id = localGet("txt2img_task_id");
if (id) {
showSubmitInterruptingPlaceholder('txt2img');
requestProgress(id, gradioApp().getElementById('txt2img_gallery_container'), gradioApp().getElementById('txt2img_gallery'), function() {
showSubmitButtons('txt2img', true);
}, null, 0);
@@ -228,6 +223,7 @@ function restoreProgressImg2img() {
var id = localGet("img2img_task_id");
if (id) {
showSubmitInterruptingPlaceholder('img2img');
requestProgress(id, gradioApp().getElementById('img2img_gallery_container'), gradioApp().getElementById('img2img_gallery'), function() {
showSubmitButtons('img2img', true);
}, null, 0);
@@ -303,6 +299,7 @@ onAfterUiUpdate(function() {
var jsdata = textarea.value;
opts = JSON.parse(jsdata);
executeCallbacks(optionsAvailableCallbacks); /*global optionsAvailableCallbacks*/
executeCallbacks(optionsChangedCallbacks); /*global optionsChangedCallbacks*/
Object.defineProperty(textarea, 'value', {
@@ -341,8 +338,8 @@ onOptionsChanged(function() {
let txt2img_textarea, img2img_textarea = undefined;
function restart_reload() {
document.body.style.backgroundColor = "var(--background-fill-primary)";
document.body.innerHTML = '<h1 style="font-family:monospace;margin-top:20%;color:lightgray;text-align:center;">Reloading...</h1>';
var requestPing = function() {
requestGet("./internal/ping", {}, function(data) {
location.reload();
@@ -371,9 +368,9 @@ function selectCheckpoint(name) {
gradioApp().getElementById('change_checkpoint').click();
}
function currentImg2imgSourceResolution(w, h, scaleBy) {
var img = gradioApp().querySelector('#mode_img2img > div[style="display: block;"] img');
return img ? [img.naturalWidth, img.naturalHeight, scaleBy] : [0, 0, scaleBy];
function currentImg2imgSourceResolution(w, h, r) {
var img = gradioApp().querySelector('#mode_img2img > div[style="display: block;"] :is(img, canvas)');
return img ? [img.naturalWidth || img.width, img.naturalHeight || img.height, r] : [0, 0, r];
}
function updateImg2imgResizeToTextAfterChangingImage() {
@@ -416,7 +413,7 @@ function switchWidthHeight(tabname) {
var onEditTimers = {};
// calls func after afterMs milliseconds has passed since the input elem has beed enited by user
// calls func after afterMs milliseconds has passed since the input elem has been edited by user
function onEdit(editId, elem, afterMs, func) {
var edited = function() {
var existingTimer = onEditTimers[editId];
javascript/ui_settings_hints.js
+10 -4
View File
@@ -14,10 +14,16 @@ onOptionsChanged(function() {
if (!commentBefore && !commentAfter) return;
var span = null;
if (div.classList.contains('gradio-checkbox')) span = div.querySelector('label span');
else if (div.classList.contains('gradio-checkboxgroup')) span = div.querySelector('span').firstChild;
else if (div.classList.contains('gradio-radio')) span = div.querySelector('span').firstChild;
else span = div.querySelector('label span').firstChild;
if (div.classList.contains('gradio-checkbox')) {
span = div.querySelector('label span');
} else if (div.classList.contains('gradio-checkboxgroup')) {
span = div.querySelector('span').firstChild;
} else if (div.classList.contains('gradio-radio')) {
span = div.querySelector('span').firstChild;
} else {
var elem = div.querySelector('label span');
if (elem) span = elem.firstChild;
}
if (!span) return;
ldm_patched/ldm/models/autoencoder.py
+2 -2
View File
@@ -182,7 +182,7 @@ class AutoencodingEngineLegacy(AutoencodingEngine):
return params
def encode(
self, x: torch.Tensor, return_reg_log: bool = False
self, x: torch.Tensor, regulation=None, return_reg_log: bool = False
) -> Union[torch.Tensor, Tuple[torch.Tensor, dict]]:
if self.max_batch_size is None:
z = self.encoder(x)
@@ -198,7 +198,7 @@ class AutoencodingEngineLegacy(AutoencodingEngine):
z.append(z_batch)
z = torch.cat(z, 0)
z, reg_log = self.regularization(z)
z, reg_log = self.regularization(z) if regulation is None else regulation(z)
if return_reg_log:
return z, reg_log
return z
ldm_patched/modules/model_patcher.py
+3
View File
@@ -81,6 +81,9 @@ class ModelPatcher:
def set_model_vae_decode_wrapper(self, wrapper_function):
self.model_options["model_vae_decode_wrapper"] = wrapper_function
def set_model_vae_regulation(self, vae_regulation):
self.model_options["model_vae_regulation"] = vae_regulation
def set_model_patch(self, patch, name):
to = self.model_options["transformer_options"]
if "patches" not in to:
ldm_patched/modules/sd.py
+3 -1
View File
@@ -296,6 +296,8 @@ class VAE:
if model_management.VAE_ALWAYS_TILED:
return self.encode_tiled(pixel_samples)
regulation = self.patcher.model_options.get("model_vae_regulation", None)
pixel_samples = pixel_samples.movedim(-1,1)
try:
memory_used = self.memory_used_encode(pixel_samples.shape, self.vae_dtype)
@@ -306,7 +308,7 @@ class VAE:
samples = torch.empty((pixel_samples.shape[0], self.latent_channels, round(pixel_samples.shape[2] // self.downscale_ratio), round(pixel_samples.shape[3] // self.downscale_ratio)), device=self.output_device)
for x in range(0, pixel_samples.shape[0], batch_number):
pixels_in = (2. * pixel_samples[x:x+batch_number] - 1.).to(self.vae_dtype).to(self.device)
samples[x:x+batch_number] = self.first_stage_model.encode(pixels_in).to(self.output_device).float()
samples[x:x+batch_number] = self.first_stage_model.encode(pixels_in, regulation).to(self.output_device).float()
except model_management.OOM_EXCEPTION as e:
print("Warning: Ran out of memory when regular VAE encoding, retrying with tiled VAE encoding.")
modules/api/api.py
+35 -27
View File
@@ -2,13 +2,11 @@ import base64
import io
import os
import time
import itertools
import datetime
import uvicorn
import ipaddress
import requests
import gradio as gr
import numpy as np
from threading import Lock
from io import BytesIO
from fastapi import APIRouter, Depends, FastAPI, Request, Response
@@ -19,13 +17,13 @@ from fastapi.encoders import jsonable_encoder
from secrets import compare_digest
import modules.shared as shared
from modules import sd_samplers, deepbooru, sd_hijack, images, scripts, ui, postprocessing, errors, restart, shared_items, script_callbacks, infotext_utils, sd_models
from modules import sd_samplers, deepbooru, sd_hijack, images, scripts, ui, postprocessing, errors, restart, shared_items, script_callbacks, infotext_utils, sd_models, sd_schedulers
from modules.api import models
from modules.shared import opts
from modules.processing import StableDiffusionProcessingTxt2Img, StableDiffusionProcessingImg2Img, process_images
from modules.textual_inversion.textual_inversion import create_embedding, train_embedding
from modules.hypernetworks.hypernetwork import create_hypernetwork, train_hypernetwork
from PIL import PngImagePlugin, Image
from PIL import PngImagePlugin
from modules.sd_models_config import find_checkpoint_config_near_filename
from modules.realesrgan_model import get_realesrgan_models
from modules import devices
@@ -45,7 +43,7 @@ def script_name_to_index(name, scripts):
def validate_sampler_name(name):
config = sd_samplers.all_samplers_map.get(name, None)
if config is None:
raise HTTPException(status_code=404, detail="Sampler not found")
raise HTTPException(status_code=400, detail="Sampler not found")
return name
@@ -87,7 +85,7 @@ def decode_base64_to_image(encoding):
headers = {'user-agent': opts.api_useragent} if opts.api_useragent else {}
response = requests.get(encoding, timeout=30, headers=headers)
try:
image = Image.open(BytesIO(response.content))
image = images.read(BytesIO(response.content))
return image
except Exception as e:
raise HTTPException(status_code=500, detail="Invalid image url") from e
@@ -95,7 +93,7 @@ def decode_base64_to_image(encoding):
if encoding.startswith("data:image/"):
encoding = encoding.split(";")[1].split(",")[1]
try:
image = Image.open(BytesIO(base64.b64decode(encoding)))
image = images.read(BytesIO(base64.b64decode(encoding)))
return image
except Exception as e:
raise HTTPException(status_code=500, detail="Invalid encoded image") from e
@@ -105,8 +103,6 @@ def encode_pil_to_base64(image):
with io.BytesIO() as output_bytes:
if isinstance(image, str):
return image
if isinstance(image, np.ndarray):
image = Image.fromarray(image)
if opts.samples_format.lower() == 'png':
use_metadata = False
metadata = PngImagePlugin.PngInfo()
@@ -117,7 +113,7 @@ def encode_pil_to_base64(image):
image.save(output_bytes, format="PNG", pnginfo=(metadata if use_metadata else None), quality=opts.jpeg_quality)
elif opts.samples_format.lower() in ("jpg", "jpeg", "webp"):
if image.mode == "RGBA":
if image.mode in ("RGBA", "P"):
image = image.convert("RGB")
parameters = image.info.get('parameters', None)
exif_bytes = piexif.dump({
@@ -211,7 +207,7 @@ class Api:
self.router = APIRouter()
self.app = app
self.queue_lock = queue_lock
api_middleware(self.app)
#api_middleware(self.app) # XXX this will have to be fixed
self.add_api_route("/sdapi/v1/txt2img", self.text2imgapi, methods=["POST"], response_model=models.TextToImageResponse)
self.add_api_route("/sdapi/v1/img2img", self.img2imgapi, methods=["POST"], response_model=models.ImageToImageResponse)
self.add_api_route("/sdapi/v1/extra-single-image", self.extras_single_image_api, methods=["POST"], response_model=models.ExtrasSingleImageResponse)
@@ -225,6 +221,7 @@ class Api:
self.add_api_route("/sdapi/v1/options", self.set_config, methods=["POST"])
self.add_api_route("/sdapi/v1/cmd-flags", self.get_cmd_flags, methods=["GET"], response_model=models.FlagsModel)
self.add_api_route("/sdapi/v1/samplers", self.get_samplers, methods=["GET"], response_model=list[models.SamplerItem])
self.add_api_route("/sdapi/v1/schedulers", self.get_schedulers, methods=["GET"], response_model=list[models.SchedulerItem])
self.add_api_route("/sdapi/v1/upscalers", self.get_upscalers, methods=["GET"], response_model=list[models.UpscalerItem])
self.add_api_route("/sdapi/v1/latent-upscale-modes", self.get_latent_upscale_modes, methods=["GET"], response_model=list[models.LatentUpscalerModeItem])
self.add_api_route("/sdapi/v1/sd-models", self.get_sd_models, methods=["GET"], response_model=list[models.SDModelItem])
@@ -364,7 +361,7 @@ class Api:
return script_args
def apply_infotext(self, request, tabname, *, script_runner=None, mentioned_script_args=None):
"""Processes `infotext` field from the `request`, and sets other fields of the `request` accoring to what's in infotext.
"""Processes `infotext` field from the `request`, and sets other fields of the `request` according to what's in infotext.
If request already has a field set, and that field is encountered in infotext too, the value from infotext is ignored.
@@ -375,7 +372,7 @@ class Api:
return {}
possible_fields = infotext_utils.paste_fields[tabname]["fields"]
set_fields = request.model_dump(exclude_unset=True) if hasattr(request, "request") else request.dict(exclude_unset=True) # pydantic v1/v2 have differenrt names for this
set_fields = request.model_dump(exclude_unset=True) if hasattr(request, "request") else request.dict(exclude_unset=True) # pydantic v1/v2 have different names for this
params = infotext_utils.parse_generation_parameters(request.infotext)
def get_field_value(field, params):
@@ -413,8 +410,8 @@ class Api:
if request.override_settings is None:
request.override_settings = {}
overriden_settings = infotext_utils.get_override_settings(params)
for _, setting_name, value in overriden_settings:
overridden_settings = infotext_utils.get_override_settings(params)
for _, setting_name, value in overridden_settings:
if setting_name not in request.override_settings:
request.override_settings[setting_name] = value
@@ -441,15 +438,19 @@ class Api:
self.apply_infotext(txt2imgreq, "txt2img", script_runner=script_runner, mentioned_script_args=infotext_script_args)
selectable_scripts, selectable_script_idx = self.get_selectable_script(txt2imgreq.script_name, script_runner)
sampler, scheduler = sd_samplers.get_sampler_and_scheduler(txt2imgreq.sampler_name or txt2imgreq.sampler_index, txt2imgreq.scheduler)
populate = txt2imgreq.copy(update={ # Override __init__ params
"sampler_name": validate_sampler_name(txt2imgreq.sampler_name or txt2imgreq.sampler_index),
"sampler_name": validate_sampler_name(sampler),
"do_not_save_samples": not txt2imgreq.save_images,
"do_not_save_grid": not txt2imgreq.save_images,
})
if populate.sampler_name:
populate.sampler_index = None # prevent a warning later on
if not populate.scheduler and scheduler != "Automatic":
populate.scheduler = scheduler
args = vars(populate)
args.pop('script_name', None)
args.pop('script_args', None) # will refeed them to the pipeline directly after initializing them
@@ -484,11 +485,7 @@ class Api:
shared.state.end()
shared.total_tqdm.clear()
b64images = [
encode_pil_to_base64(image)
for image in itertools.chain(processed.images, processed.extra_images)
if send_images
]
b64images = list(map(encode_pil_to_base64, processed.images)) if send_images else []
return models.TextToImageResponse(images=b64images, parameters=vars(txt2imgreq), info=processed.js())
@@ -509,9 +506,10 @@ class Api:
self.apply_infotext(img2imgreq, "img2img", script_runner=script_runner, mentioned_script_args=infotext_script_args)
selectable_scripts, selectable_script_idx = self.get_selectable_script(img2imgreq.script_name, script_runner)
sampler, scheduler = sd_samplers.get_sampler_and_scheduler(img2imgreq.sampler_name or img2imgreq.sampler_index, img2imgreq.scheduler)
populate = img2imgreq.copy(update={ # Override __init__ params
"sampler_name": validate_sampler_name(img2imgreq.sampler_name or img2imgreq.sampler_index),
"sampler_name": validate_sampler_name(sampler),
"do_not_save_samples": not img2imgreq.save_images,
"do_not_save_grid": not img2imgreq.save_images,
"mask": mask,
@@ -519,6 +517,9 @@ class Api:
if populate.sampler_name:
populate.sampler_index = None # prevent a warning later on
if not populate.scheduler and scheduler != "Automatic":
populate.scheduler = scheduler
args = vars(populate)
args.pop('include_init_images', None) # this is meant to be done by "exclude": True in model, but it's for a reason that I cannot determine.
args.pop('script_name', None)
@@ -555,11 +556,7 @@ class Api:
shared.state.end()
shared.total_tqdm.clear()
b64images = [
encode_pil_to_base64(image)
for image in itertools.chain(processed.images, processed.extra_images)
if send_images
]
b64images = list(map(encode_pil_to_base64, processed.images)) if send_images else []
if not img2imgreq.include_init_images:
img2imgreq.init_images = None
@@ -695,6 +692,17 @@ class Api:
def get_samplers(self):
return [{"name": sampler[0], "aliases":sampler[2], "options":sampler[3]} for sampler in sd_samplers.all_samplers]
def get_schedulers(self):
return [
{
"name": scheduler.name,
"label": scheduler.label,
"aliases": scheduler.aliases,
"default_rho": scheduler.default_rho,
"need_inner_model": scheduler.need_inner_model,
}
for scheduler in sd_schedulers.schedulers]
def get_upscalers(self):
return [
{
modules/api/models.py
+41 -27
View File
@@ -1,6 +1,6 @@
import inspect
from pydantic import BaseModel, Field, create_model
from pydantic import BaseModel, Field, create_model, ConfigDict
from typing import Any, Optional, Literal
from inflection import underscore
from modules.processing import StableDiffusionProcessingTxt2Img, StableDiffusionProcessingImg2Img
@@ -92,9 +92,7 @@ class PydanticModelGenerator:
fields = {
d.field: (d.field_type, Field(default=d.field_value, alias=d.field_alias, exclude=d.field_exclude)) for d in self._model_def
}
DynamicModel = create_model(self._model_name, **fields)
DynamicModel.__config__.allow_population_by_field_name = True
DynamicModel.__config__.allow_mutation = True
DynamicModel = create_model(self._model_name, __config__=ConfigDict(populate_by_name=True, frozen=False), **fields)
return DynamicModel
StableDiffusionTxt2ImgProcessingAPI = PydanticModelGenerator(
@@ -102,13 +100,13 @@ StableDiffusionTxt2ImgProcessingAPI = PydanticModelGenerator(
StableDiffusionProcessingTxt2Img,
[
{"key": "sampler_index", "type": str, "default": "Euler"},
{"key": "script_name", "type": str, "default": None},
{"key": "script_name", "type": str | None, "default": None},
{"key": "script_args", "type": list, "default": []},
{"key": "send_images", "type": bool, "default": True},
{"key": "save_images", "type": bool, "default": False},
{"key": "alwayson_scripts", "type": dict, "default": {}},
{"key": "force_task_id", "type": str, "default": None},
{"key": "infotext", "type": str, "default": None},
{"key": "force_task_id", "type": str | None, "default": None},
{"key": "infotext", "type": str | None, "default": None},
]
).generate_model()
@@ -117,27 +115,27 @@ StableDiffusionImg2ImgProcessingAPI = PydanticModelGenerator(
StableDiffusionProcessingImg2Img,
[
{"key": "sampler_index", "type": str, "default": "Euler"},
{"key": "init_images", "type": list, "default": None},
{"key": "init_images", "type": list | None, "default": None},
{"key": "denoising_strength", "type": float, "default": 0.75},
{"key": "mask", "type": str, "default": None},
{"key": "mask", "type": str | None, "default": None},
{"key": "include_init_images", "type": bool, "default": False, "exclude" : True},
{"key": "script_name", "type": str, "default": None},
{"key": "script_name", "type": str | None, "default": None},
{"key": "script_args", "type": list, "default": []},
{"key": "send_images", "type": bool, "default": True},
{"key": "save_images", "type": bool, "default": False},
{"key": "alwayson_scripts", "type": dict, "default": {}},
{"key": "force_task_id", "type": str, "default": None},
{"key": "infotext", "type": str, "default": None},
{"key": "force_task_id", "type": str | None, "default": None},
{"key": "infotext", "type": str | None, "default": None},
]
).generate_model()
class TextToImageResponse(BaseModel):
images: list[str] = Field(default=None, title="Image", description="The generated image in base64 format.")
images: list[str] | None = Field(default=None, title="Image", description="The generated image in base64 format.")
parameters: dict
info: str
class ImageToImageResponse(BaseModel):
images: list[str] = Field(default=None, title="Image", description="The generated image in base64 format.")
images: list[str] | None = Field(default=None, title="Image", description="The generated image in base64 format.")
parameters: dict
info: str
@@ -147,7 +145,7 @@ class ExtrasBaseRequest(BaseModel):
gfpgan_visibility: float = Field(default=0, title="GFPGAN Visibility", ge=0, le=1, allow_inf_nan=False, description="Sets the visibility of GFPGAN, values should be between 0 and 1.")
codeformer_visibility: float = Field(default=0, title="CodeFormer Visibility", ge=0, le=1, allow_inf_nan=False, description="Sets the visibility of CodeFormer, values should be between 0 and 1.")
codeformer_weight: float = Field(default=0, title="CodeFormer Weight", ge=0, le=1, allow_inf_nan=False, description="Sets the weight of CodeFormer, values should be between 0 and 1.")
upscaling_resize: float = Field(default=2, title="Upscaling Factor", ge=1, le=8, description="By how much to upscale the image, only used when resize_mode=0.")
upscaling_resize: float = Field(default=2, title="Upscaling Factor", gt=0, description="By how much to upscale the image, only used when resize_mode=0.")
upscaling_resize_w: int = Field(default=512, title="Target Width", ge=1, description="Target width for the upscaler to hit. Only used when resize_mode=1.")
upscaling_resize_h: int = Field(default=512, title="Target Height", ge=1, description="Target height for the upscaler to hit. Only used when resize_mode=1.")
upscaling_crop: bool = Field(default=True, title="Crop to fit", description="Should the upscaler crop the image to fit in the chosen size?")
@@ -163,7 +161,7 @@ class ExtrasSingleImageRequest(ExtrasBaseRequest):
image: str = Field(default="", title="Image", description="Image to work on, must be a Base64 string containing the image's data.")
class ExtrasSingleImageResponse(ExtraBaseResponse):
image: str = Field(default=None, title="Image", description="The generated image in base64 format.")
image: str | None = Field(default=None, title="Image", description="The generated image in base64 format.")
class FileData(BaseModel):
data: str = Field(title="File data", description="Base64 representation of the file")
@@ -190,15 +188,15 @@ class ProgressResponse(BaseModel):
progress: float = Field(title="Progress", description="The progress with a range of 0 to 1")
eta_relative: float = Field(title="ETA in secs")
state: dict = Field(title="State", description="The current state snapshot")
current_image: str = Field(default=None, title="Current image", description="The current image in base64 format. opts.show_progress_every_n_steps is required for this to work.")
textinfo: str = Field(default=None, title="Info text", description="Info text used by WebUI.")
current_image: str | None = Field(default=None, title="Current image", description="The current image in base64 format. opts.show_progress_every_n_steps is required for this to work.")
textinfo: str | None = Field(default=None, title="Info text", description="Info text used by WebUI.")
class InterrogateRequest(BaseModel):
image: str = Field(default="", title="Image", description="Image to work on, must be a Base64 string containing the image's data.")
model: str = Field(default="clip", title="Model", description="The interrogate model used.")
class InterrogateResponse(BaseModel):
caption: str = Field(default=None, title="Caption", description="The generated caption for the image.")
caption: str | None = Field(default=None, title="Caption", description="The generated caption for the image.")
class TrainResponse(BaseModel):
info: str = Field(title="Train info", description="Response string from train embedding or hypernetwork task.")
@@ -223,7 +221,7 @@ _options = vars(parser)['_option_string_actions']
for key in _options:
if(_options[key].dest != 'help'):
flag = _options[key]
_type = str
_type = str | None
if _options[key].default is not None:
_type = type(_options[key].default)
flags.update({flag.dest: (_type, Field(default=flag.default, description=flag.help))})
@@ -233,9 +231,19 @@ FlagsModel = create_model("Flags", **flags)
class SamplerItem(BaseModel):
name: str = Field(title="Name")
aliases: list[str] = Field(title="Aliases")
options: dict[str, str] = Field(title="Options")
options: dict[str, Any] = Field(title="Options")
class SchedulerItem(BaseModel):
name: str = Field(title="Name")
label: str = Field(title="Label")
aliases: Optional[list[str]] = Field(title="Aliases")
default_rho: Optional[float] = Field(title="Default Rho")
need_inner_model: Optional[bool] = Field(title="Needs Inner Model")
class UpscalerItem(BaseModel):
class Config:
protected_namespaces = ()
name: str = Field(title="Name")
model_name: Optional[str] = Field(title="Model Name")
model_path: Optional[str] = Field(title="Path")
@@ -246,6 +254,9 @@ class LatentUpscalerModeItem(BaseModel):
name: str = Field(title="Name")
class SDModelItem(BaseModel):
class Config:
protected_namespaces = ()
title: str = Field(title="Title")
model_name: str = Field(title="Model Name")
hash: Optional[str] = Field(title="Short hash")
@@ -254,6 +265,9 @@ class SDModelItem(BaseModel):
config: Optional[str] = Field(title="Config file")
class SDVaeItem(BaseModel):
class Config:
protected_namespaces = ()
model_name: str = Field(title="Model Name")
filename: str = Field(title="Filename")
@@ -293,12 +307,12 @@ class MemoryResponse(BaseModel):
class ScriptsList(BaseModel):
txt2img: list = Field(default=None, title="Txt2img", description="Titles of scripts (txt2img)")
img2img: list = Field(default=None, title="Img2img", description="Titles of scripts (img2img)")
txt2img: list | None = Field(default=None, title="Txt2img", description="Titles of scripts (txt2img)")
img2img: list | None = Field(default=None, title="Img2img", description="Titles of scripts (img2img)")
class ScriptArg(BaseModel):
label: str = Field(default=None, title="Label", description="Name of the argument in UI")
label: str | None = Field(default=None, title="Label", description="Name of the argument in UI")
value: Optional[Any] = Field(default=None, title="Value", description="Default value of the argument")
minimum: Optional[Any] = Field(default=None, title="Minimum", description="Minimum allowed value for the argumentin UI")
maximum: Optional[Any] = Field(default=None, title="Minimum", description="Maximum allowed value for the argumentin UI")
@@ -307,9 +321,9 @@ class ScriptArg(BaseModel):
class ScriptInfo(BaseModel):
name: str = Field(default=None, title="Name", description="Script name")
is_alwayson: bool = Field(default=None, title="IsAlwayson", description="Flag specifying whether this script is an alwayson script")
is_img2img: bool = Field(default=None, title="IsImg2img", description="Flag specifying whether this script is an img2img script")
name: str | None = Field(default=None, title="Name", description="Script name")
is_alwayson: bool | None = Field(default=None, title="IsAlwayson", description="Flag specifying whether this script is an alwayson script")
is_img2img: bool | None = Field(default=None, title="IsImg2img", description="Flag specifying whether this script is an img2img script")
args: list[ScriptArg] = Field(title="Arguments", description="List of script's arguments")
class ExtensionItem(BaseModel):
modules/cache.py
+44 -44
View File
@@ -2,48 +2,55 @@ import json
import os
import os.path
import threading
import time
import diskcache
import tqdm
from modules.paths import data_path, script_path
cache_filename = os.environ.get('SD_WEBUI_CACHE_FILE', os.path.join(data_path, "cache.json"))
cache_data = None
cache_dir = os.environ.get('SD_WEBUI_CACHE_DIR', os.path.join(data_path, "cache"))
caches = {}
cache_lock = threading.Lock()
dump_cache_after = None
dump_cache_thread = None
def dump_cache():
"""
Marks cache for writing to disk. 5 seconds after no one else flags the cache for writing, it is written.
"""
"""old function for dumping cache to disk; does nothing since diskcache."""
global dump_cache_after
global dump_cache_thread
pass
def thread_func():
global dump_cache_after
global dump_cache_thread
while dump_cache_after is not None and time.time() < dump_cache_after:
time.sleep(1)
def make_cache(subsection: str) -> diskcache.Cache:
return diskcache.Cache(
os.path.join(cache_dir, subsection),
size_limit=2**32, # 4 GB, culling oldest first
disk_min_file_size=2**18, # keep up to 256KB in Sqlite
)
with cache_lock:
cache_filename_tmp = cache_filename + "-"
with open(cache_filename_tmp, "w", encoding="utf8") as file:
json.dump(cache_data, file, indent=4, ensure_ascii=False)
os.replace(cache_filename_tmp, cache_filename)
def convert_old_cached_data():
try:
with open(cache_filename, "r", encoding="utf8") as file:
data = json.load(file)
except FileNotFoundError:
return
except Exception:
os.replace(cache_filename, os.path.join(script_path, "tmp", "cache.json"))
print('[ERROR] issue occurred while trying to read cache.json; old cache has been moved to tmp/cache.json')
return
dump_cache_after = None
dump_cache_thread = None
total_count = sum(len(keyvalues) for keyvalues in data.values())
with cache_lock:
dump_cache_after = time.time() + 5
if dump_cache_thread is None:
dump_cache_thread = threading.Thread(name='cache-writer', target=thread_func)
dump_cache_thread.start()
with tqdm.tqdm(total=total_count, desc="converting cache") as progress:
for subsection, keyvalues in data.items():
cache_obj = caches.get(subsection)
if cache_obj is None:
cache_obj = make_cache(subsection)
caches[subsection] = cache_obj
for key, value in keyvalues.items():
cache_obj[key] = value
progress.update(1)
def cache(subsection):
@@ -54,28 +61,21 @@ def cache(subsection):
subsection (str): The subsection identifier for the cache.
Returns:
dict: The cache data for the specified subsection.
diskcache.Cache: The cache data for the specified subsection.
"""
global cache_data
if cache_data is None:
cache_obj = caches.get(subsection)
if not cache_obj:
with cache_lock:
if cache_data is None:
try:
with open(cache_filename, "r", encoding="utf8") as file:
cache_data = json.load(file)
except FileNotFoundError:
cache_data = {}
except Exception:
os.replace(cache_filename, os.path.join(script_path, "tmp", "cache.json"))
print('[ERROR] issue occurred while trying to read cache.json, move current cache to tmp/cache.json and create new cache')
cache_data = {}
if not os.path.exists(cache_dir) and os.path.isfile(cache_filename):
convert_old_cached_data()
s = cache_data.get(subsection, {})
cache_data[subsection] = s
cache_obj = caches.get(subsection)
if not cache_obj:
cache_obj = make_cache(subsection)
caches[subsection] = cache_obj
return s
return cache_obj
def cached_data_for_file(subsection, title, filename, func):
modules/call_queue.py
+27 -12
View File
@@ -1,8 +1,9 @@
import os.path
from functools import wraps
import html
import time
from modules import shared, progress, errors, devices, fifo_lock
from modules import shared, progress, errors, devices, fifo_lock, profiling
queue_lock = fifo_lock.FIFOLock()
@@ -46,6 +47,22 @@ def wrap_gradio_gpu_call(func, extra_outputs=None):
def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
@wraps(func)
def f(*args, **kwargs):
try:
res = func(*args, **kwargs)
finally:
shared.state.skipped = False
shared.state.interrupted = False
shared.state.stopping_generation = False
shared.state.job_count = 0
shared.state.job = ""
return res
return wrap_gradio_call_no_job(f, extra_outputs, add_stats)
def wrap_gradio_call_no_job(func, extra_outputs=None, add_stats=False):
@wraps(func)
def f(*args, extra_outputs_array=extra_outputs, **kwargs):
run_memmon = shared.opts.memmon_poll_rate > 0 and not shared.mem_mon.disabled and add_stats
@@ -65,9 +82,6 @@ def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
arg_str += f" (Argument list truncated at {max_debug_str_len}/{len(arg_str)} characters)"
errors.report(f"{message}\n{arg_str}", exc_info=True)
shared.state.job = ""
shared.state.job_count = 0
if extra_outputs_array is None:
extra_outputs_array = [None, '']
@@ -76,11 +90,6 @@ def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
devices.torch_gc()
shared.state.skipped = False
shared.state.interrupted = False
shared.state.stopping_generation = False
shared.state.job_count = 0
if not add_stats:
return tuple(res)
@@ -100,8 +109,8 @@ def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
sys_pct = sys_peak/max(sys_total, 1) * 100
toltip_a = "Active: peak amount of video memory used during generation (excluding cached data)"
toltip_r = "Reserved: total amout of video memory allocated by the Torch library "
toltip_sys = "System: peak amout of video memory allocated by all running programs, out of total capacity"
toltip_r = "Reserved: total amount of video memory allocated by the Torch library "
toltip_sys = "System: peak amount of video memory allocated by all running programs, out of total capacity"
text_a = f"<abbr title='{toltip_a}'>A</abbr>: <span class='measurement'>{active_peak/1024:.2f} GB</span>"
text_r = f"<abbr title='{toltip_r}'>R</abbr>: <span class='measurement'>{reserved_peak/1024:.2f} GB</span>"
@@ -111,9 +120,15 @@ def wrap_gradio_call(func, extra_outputs=None, add_stats=False):
else:
vram_html = ''
if shared.opts.profiling_enable and os.path.exists(shared.opts.profiling_filename):
profiling_html = f"<p class='profile'> [ <a href='{profiling.webpath()}' download>Profile</a> ] </p>"
else:
profiling_html = ''
# last item is always HTML
res[-1] += f"<div class='performance'><p class='time'>Time taken: <wbr><span class='measurement'>{elapsed_text}</span></p>{vram_html}</div>"
res[-1] += f"<div class='performance'><p class='time'>Time taken: <wbr><span class='measurement'>{elapsed_text}</span></p>{vram_html}{profiling_html}</div>"
return tuple(res)
return f
modules/cmd_args.py
+8 -3
View File
@@ -22,6 +22,7 @@ parser.add_argument("--dump-sysinfo", action='store_true', help="launch.py argum
parser.add_argument("--loglevel", type=str, help="log level; one of: CRITICAL, ERROR, WARNING, INFO, DEBUG", default=None)
parser.add_argument("--do-not-download-clip", action='store_true', help="do not download CLIP model even if it's not included in the checkpoint")
parser.add_argument("--data-dir", type=normalized_filepath, default=os.path.dirname(os.path.dirname(os.path.realpath(__file__))), help="base path where all user data is stored")
parser.add_argument("--models-dir", type=normalized_filepath, default=None, help="base path where models are stored; overrides --data-dir")
parser.add_argument("--config", type=normalized_filepath, default=sd_default_config, help="path to config which constructs model",)
parser.add_argument("--ckpt", type=normalized_filepath, default=sd_model_file, help="path to checkpoint of stable diffusion model; if specified, this checkpoint will be added to the list of checkpoints and loaded",)
parser.add_argument("--ckpt-dir", type=normalized_filepath, default=None, help="Path to directory with stable diffusion checkpoints")
@@ -31,7 +32,7 @@ parser.add_argument("--gfpgan-model", type=normalized_filepath, help="GFPGAN mod
parser.add_argument("--no-half", action='store_true', help="do not switch the model to 16-bit floats")
parser.add_argument("--no-half-vae", action='store_true', help="do not switch the VAE model to 16-bit floats")
parser.add_argument("--no-progressbar-hiding", action='store_true', help="do not hide progressbar in gradio UI (we hide it because it slows down ML if you have hardware acceleration in browser)")
parser.add_argument("--max-batch-count", type=int, default=16, help="maximum batch count value for the UI")
parser.add_argument("--max-batch-count", type=int, default=16, help="does not do anything")
parser.add_argument("--embeddings-dir", type=normalized_filepath, default=os.path.join(data_path, 'embeddings'), help="embeddings directory for textual inversion (default: embeddings)")
parser.add_argument("--textual-inversion-templates-dir", type=normalized_filepath, default=os.path.join(script_path, 'textual_inversion_templates'), help="directory with textual inversion templates")
parser.add_argument("--hypernetwork-dir", type=normalized_filepath, default=os.path.join(models_path, 'hypernetworks'), help="hypernetwork directory")
@@ -43,7 +44,7 @@ parser.add_argument("--lowvram", action='store_true', help="enable stable diffus
parser.add_argument("--lowram", action='store_true', help="load stable diffusion checkpoint weights to VRAM instead of RAM")
parser.add_argument("--always-batch-cond-uncond", action='store_true', help="does not do anything")
parser.add_argument("--unload-gfpgan", action='store_true', help="does not do anything.")
parser.add_argument("--precision", type=str, help="evaluate at this precision", choices=["full", "autocast"], default="autocast")
parser.add_argument("--precision", type=str, help="evaluate at this precision", choices=["full", "half", "autocast"], default="autocast")
parser.add_argument("--upcast-sampling", action='store_true', help="upcast sampling. No effect with --no-half. Usually produces similar results to --no-half with better performance while using less memory.")
parser.add_argument("--share", action='store_true', help="use share=True for gradio and make the UI accessible through their site")
parser.add_argument("--ngrok", type=str, help="ngrok authtoken, alternative to gradio --share", default=None)
@@ -55,6 +56,7 @@ parser.add_argument("--gfpgan-models-path", type=normalized_filepath, help="Path
parser.add_argument("--esrgan-models-path", type=normalized_filepath, help="Path to directory with ESRGAN model file(s).", default=os.path.join(models_path, 'ESRGAN'))
parser.add_argument("--bsrgan-models-path", type=normalized_filepath, help="Path to directory with BSRGAN model file(s).", default=os.path.join(models_path, 'BSRGAN'))
parser.add_argument("--realesrgan-models-path", type=normalized_filepath, help="Path to directory with RealESRGAN model file(s).", default=os.path.join(models_path, 'RealESRGAN'))
parser.add_argument("--dat-models-path", type=normalized_filepath, help="Path to directory with DAT model file(s).", default=os.path.join(models_path, 'DAT'))
parser.add_argument("--clip-models-path", type=normalized_filepath, help="Path to directory with CLIP model file(s).", default=None)
parser.add_argument("--xformers", action='store_true', help="enable xformers for cross attention layers")
parser.add_argument("--force-enable-xformers", action='store_true', help="enable xformers for cross attention layers regardless of whether the checking code thinks you can run it; do not make bug reports if this fails to work")
@@ -122,7 +124,10 @@ parser.add_argument('--api-server-stop', action='store_true', help='enable serve
parser.add_argument('--timeout-keep-alive', type=int, default=30, help='set timeout_keep_alive for uvicorn')
parser.add_argument("--disable-all-extensions", action='store_true', help="prevent all extensions from running regardless of any other settings", default=False)
parser.add_argument("--disable-extra-extensions", action='store_true', help="prevent all extensions except built-in from running regardless of any other settings", default=False)
parser.add_argument("--skip-load-model-at-start", action='store_true', help="if load a model at web start, only take effect when --nowebui", )
parser.add_argument("--skip-load-model-at-start", action='store_true', help="if load a model at web start, only take effect when --nowebui")
parser.add_argument("--unix-filenames-sanitization", action='store_true', help="allow any symbols except '/' in filenames. May conflict with your browser and file system")
parser.add_argument("--filenames-max-length", type=int, default=128, help='maximal length of filenames of saved images. If you override it, it can conflict with your file system')
parser.add_argument("--no-prompt-history", action='store_true', help="disable read prompt from last generation feature; settings this argument will not create '--data_path/params.txt' file")
# Arguments added by forge.
parser.add_argument(
modules/codeformer_model.py
+1 -1
View File
@@ -50,7 +50,7 @@ class FaceRestorerCodeFormer(face_restoration_utils.CommonFaceRestoration):
def restore_face(cropped_face_t):
assert self.net is not None
return self.net(cropped_face_t, w=w, adain=True)[0]
return self.net(cropped_face_t, weight=w, adain=True)[0]
return self.restore_with_helper(np_image, restore_face)
modules/errors.py
+1 -1
View File
@@ -109,7 +109,7 @@ def check_versions():
expected_torch_version = "2.1.2"
expected_xformers_version = "0.0.23.post1"
expected_gradio_version = "3.41.2"
expected_gradio_version = "4.39.0"
if version.parse(torch.__version__) < version.parse(expected_torch_version):
print_error_explanation(f"""
modules/extensions.py
+64 -6
View File
@@ -1,6 +1,7 @@
from __future__ import annotations
import configparser
import dataclasses
import os
import threading
import re
@@ -10,6 +11,10 @@ from modules.gitpython_hack import Repo
from modules.paths_internal import extensions_dir, extensions_builtin_dir, script_path # noqa: F401
from modules_forge.config import always_disabled_extensions
extensions: list[Extension] = []
extension_paths: dict[str, Extension] = {}
loaded_extensions: dict[str, Exception] = {}
os.makedirs(extensions_dir, exist_ok=True)
@@ -23,6 +28,13 @@ def active():
return [x for x in extensions if x.enabled]
@dataclasses.dataclass
class CallbackOrderInfo:
name: str
before: list
after: list
class ExtensionMetadata:
filename = "metadata.ini"
config: configparser.ConfigParser
@@ -43,7 +55,7 @@ class ExtensionMetadata:
self.canonical_name = self.config.get("Extension", "Name", fallback=canonical_name)
self.canonical_name = canonical_name.lower().strip()
self.requires = self.get_script_requirements("Requires", "Extension")
self.requires = None
def get_script_requirements(self, field, section, extra_section=None):
"""reads a list of requirements from the config; field is the name of the field in the ini file,
@@ -55,7 +67,15 @@ class ExtensionMetadata:
if extra_section:
x = x + ', ' + self.config.get(extra_section, field, fallback='')
return self.parse_list(x.lower())
listed_requirements = self.parse_list(x.lower())
res = []
for requirement in listed_requirements:
loaded_requirements = (x for x in requirement.split("|") if x in loaded_extensions)
relevant_requirement = next(loaded_requirements, requirement)
res.append(relevant_requirement)
return res
def parse_list(self, text):
"""converts a line from config ("ext1 ext2, ext3 ") into a python list (["ext1", "ext2", "ext3"])"""
@@ -66,6 +86,22 @@ class ExtensionMetadata:
# both "," and " " are accepted as separator
return [x for x in re.split(r"[,\s]+", text.strip()) if x]
def list_callback_order_instructions(self):
for section in self.config.sections():
if not section.startswith("callbacks/"):
continue
callback_name = section[10:]
if not callback_name.startswith(self.canonical_name):
errors.report(f"Callback order section for extension {self.canonical_name} is referencing the wrong extension: {section}")
continue
before = self.parse_list(self.config.get(section, 'Before', fallback=''))
after = self.parse_list(self.config.get(section, 'After', fallback=''))
yield CallbackOrderInfo(callback_name, before, after)
class Extension:
lock = threading.Lock()
@@ -156,14 +192,17 @@ class Extension:
def check_updates(self):
repo = Repo(self.path)
branch_name = f'{repo.remote().name}/{self.branch}'
for fetch in repo.remote().fetch(dry_run=True):
if self.branch and fetch.name != branch_name:
continue
if fetch.flags != fetch.HEAD_UPTODATE:
self.can_update = True
self.status = "new commits"
return
try:
origin = repo.rev_parse('origin')
origin = repo.rev_parse(branch_name)
if repo.head.commit != origin:
self.can_update = True
self.status = "behind HEAD"
@@ -176,8 +215,10 @@ class Extension:
self.can_update = False
self.status = "latest"
def fetch_and_reset_hard(self, commit='origin'):
def fetch_and_reset_hard(self, commit=None):
repo = Repo(self.path)
if commit is None:
commit = f'{repo.remote().name}/{self.branch}'
# Fix: `error: Your local changes to the following files would be overwritten by merge`,
# because WSL2 Docker set 755 file permissions instead of 644, this results to the error.
repo.git.fetch(all=True)
@@ -187,6 +228,8 @@ class Extension:
def list_extensions():
extensions.clear()
extension_paths.clear()
loaded_extensions.clear()
if shared.cmd_opts.disable_all_extensions:
print("*** \"--disable-all-extensions\" arg was used, will not load any extensions ***")
@@ -197,7 +240,6 @@ def list_extensions():
elif shared.opts.disable_all_extensions == "extra":
print("*** \"Disable all extensions\" option was set, will only load built-in extensions ***")
loaded_extensions = {}
# scan through extensions directory and load metadata
for dirname in [extensions_builtin_dir, extensions_dir]:
@@ -231,8 +273,12 @@ def list_extensions():
)
extensions.append(extension)
extension_paths[extension.path] = extension
loaded_extensions[canonical_name] = extension
for extension in extensions:
extension.metadata.requires = extension.metadata.get_script_requirements("Requires", "Extension")
# check for requirements
for extension in extensions:
if not extension.enabled:
@@ -249,4 +295,16 @@ def list_extensions():
continue
extensions: list[Extension] = []
def find_extension(filename):
parentdir = os.path.dirname(os.path.realpath(filename))
while parentdir != filename:
extension = extension_paths.get(parentdir)
if extension is not None:
return extension
filename = parentdir
parentdir = os.path.dirname(filename)
return None
modules/extra_networks.py
+1 -1
View File
@@ -60,7 +60,7 @@ class ExtraNetwork:
Where name matches the name of this ExtraNetwork object, and arg1:arg2:arg3 are any natural number of text arguments
separated by colon.
Even if the user does not mention this ExtraNetwork in his prompt, the call will stil be made, with empty params_list -
Even if the user does not mention this ExtraNetwork in his prompt, the call will still be made, with empty params_list -
in this case, all effects of this extra networks should be disabled.
Can be called multiple times before deactivate() - each new call should override the previous call completely.
modules/gfpgan_model.py
+1 -3
View File
@@ -36,13 +36,11 @@ class FaceRestorerGFPGAN(face_restoration_utils.CommonFaceRestoration):
ext_filter=['.pth'],
):
if 'GFPGAN' in os.path.basename(model_path):
model = modelloader.load_spandrel_model(
return modelloader.load_spandrel_model(
model_path,
device=self.get_device(),
expected_architecture='GFPGAN',
).model
model.different_w = True # see https://github.com/chaiNNer-org/spandrel/pull/81
return model
raise ValueError("No GFPGAN model found")
def restore(self, np_image):
modules/gradio_extensions.py
+166
View File
@@ -0,0 +1,166 @@
import inspect
import warnings
from functools import wraps
import gradio as gr
import gradio.component_meta
from modules import scripts, ui_tempdir, patches
class GradioDeprecationWarning(DeprecationWarning):
pass
def add_classes_to_gradio_component(comp):
"""
this adds gradio-* to the component for css styling (ie gradio-button to gr.Button), as well as some others
"""
comp.elem_classes = [f"gradio-{comp.get_block_name()}", *(getattr(comp, 'elem_classes', None) or [])]
if getattr(comp, 'multiselect', False):
comp.elem_classes.append('multiselect')
def IOComponent_init(self, *args, **kwargs):
self.webui_tooltip = kwargs.pop('tooltip', None)
if scripts.scripts_current is not None:
scripts.scripts_current.before_component(self, **kwargs)
scripts.script_callbacks.before_component_callback(self, **kwargs)
res = original_IOComponent_init(self, *args, **kwargs)
add_classes_to_gradio_component(self)
scripts.script_callbacks.after_component_callback(self, **kwargs)
if scripts.scripts_current is not None:
scripts.scripts_current.after_component(self, **kwargs)
return res
def Block_get_config(self):
config = original_Block_get_config(self)
webui_tooltip = getattr(self, 'webui_tooltip', None)
if webui_tooltip:
config["webui_tooltip"] = webui_tooltip
config.pop('example_inputs', None)
return config
def BlockContext_init(self, *args, **kwargs):
if scripts.scripts_current is not None:
scripts.scripts_current.before_component(self, **kwargs)
scripts.script_callbacks.before_component_callback(self, **kwargs)
res = original_BlockContext_init(self, *args, **kwargs)
add_classes_to_gradio_component(self)
scripts.script_callbacks.after_component_callback(self, **kwargs)
if scripts.scripts_current is not None:
scripts.scripts_current.after_component(self, **kwargs)
return res
def Blocks_get_config_file(self, *args, **kwargs):
config = original_Blocks_get_config_file(self, *args, **kwargs)
for comp_config in config["components"]:
if "example_inputs" in comp_config:
comp_config["example_inputs"] = {"serialized": []}
return config
original_IOComponent_init = patches.patch(__name__, obj=gr.components.Component, field="__init__", replacement=IOComponent_init)
original_Block_get_config = patches.patch(__name__, obj=gr.blocks.Block, field="get_config", replacement=Block_get_config)
original_BlockContext_init = patches.patch(__name__, obj=gr.blocks.BlockContext, field="__init__", replacement=BlockContext_init)
original_Blocks_get_config_file = patches.patch(__name__, obj=gr.blocks.Blocks, field="get_config_file", replacement=Blocks_get_config_file)
ui_tempdir.install_ui_tempdir_override()
def gradio_component_meta_create_or_modify_pyi(component_class, class_name, events):
if hasattr(component_class, 'webui_do_not_create_gradio_pyi_thank_you'):
return
gradio_component_meta_create_or_modify_pyi_original(component_class, class_name, events)
# this prevents creation of .pyi files in webui dir
gradio_component_meta_create_or_modify_pyi_original = patches.patch(__file__, gradio.component_meta, 'create_or_modify_pyi', gradio_component_meta_create_or_modify_pyi)
# this function is broken and does not seem to do anything useful
gradio.component_meta.updateable = lambda x: x
def repair(grclass):
if not getattr(grclass, 'EVENTS', None):
return
@wraps(grclass.__init__)
def __repaired_init__(self, *args, tooltip=None, source=None, original=grclass.__init__, **kwargs):
if source:
kwargs["sources"] = [source]
allowed_kwargs = inspect.signature(original).parameters
fixed_kwargs = {}
for k, v in kwargs.items():
if k in allowed_kwargs:
fixed_kwargs[k] = v
else:
warnings.warn(f"unexpected argument for {grclass.__name__}: {k}", GradioDeprecationWarning, stacklevel=2)
original(self, *args, **fixed_kwargs)
self.webui_tooltip = tooltip
for event in self.EVENTS:
replaced_event = getattr(self, str(event))
def fun(*xargs, _js=None, replaced_event=replaced_event, **xkwargs):
if _js:
xkwargs['js'] = _js
return replaced_event(*xargs, **xkwargs)
setattr(self, str(event), fun)
grclass.__init__ = __repaired_init__
grclass.update = gr.update
for component in set(gr.components.__all__ + gr.layouts.__all__):
repair(getattr(gr, component, None))
class Dependency(gr.events.Dependency):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def then(*xargs, _js=None, **xkwargs):
if _js:
xkwargs['js'] = _js
return original_then(*xargs, **xkwargs)
original_then = self.then
self.then = then
gr.events.Dependency = Dependency
gr.Box = gr.Group
modules/gradio_extensons.py
-83
View File
@@ -1,83 +0,0 @@
import gradio as gr
from modules import scripts, ui_tempdir, patches
def add_classes_to_gradio_component(comp):
"""
this adds gradio-* to the component for css styling (ie gradio-button to gr.Button), as well as some others
"""
comp.elem_classes = [f"gradio-{comp.get_block_name()}", *(comp.elem_classes or [])]
if getattr(comp, 'multiselect', False):
comp.elem_classes.append('multiselect')
def IOComponent_init(self, *args, **kwargs):
self.webui_tooltip = kwargs.pop('tooltip', None)
if scripts.scripts_current is not None:
scripts.scripts_current.before_component(self, **kwargs)
scripts.script_callbacks.before_component_callback(self, **kwargs)
res = original_IOComponent_init(self, *args, **kwargs)
add_classes_to_gradio_component(self)
scripts.script_callbacks.after_component_callback(self, **kwargs)
if scripts.scripts_current is not None:
scripts.scripts_current.after_component(self, **kwargs)
return res
def Block_get_config(self):
config = original_Block_get_config(self)
webui_tooltip = getattr(self, 'webui_tooltip', None)
if webui_tooltip:
config["webui_tooltip"] = webui_tooltip
config.pop('example_inputs', None)
return config
def BlockContext_init(self, *args, **kwargs):
if scripts.scripts_current is not None:
scripts.scripts_current.before_component(self, **kwargs)
scripts.script_callbacks.before_component_callback(self, **kwargs)
res = original_BlockContext_init(self, *args, **kwargs)
add_classes_to_gradio_component(self)
scripts.script_callbacks.after_component_callback(self, **kwargs)
if scripts.scripts_current is not None:
scripts.scripts_current.after_component(self, **kwargs)
return res
def Blocks_get_config_file(self, *args, **kwargs):
config = original_Blocks_get_config_file(self, *args, **kwargs)
for comp_config in config["components"]:
if "example_inputs" in comp_config:
comp_config["example_inputs"] = {"serialized": []}
return config
original_IOComponent_init = patches.patch(__name__, obj=gr.components.IOComponent, field="__init__", replacement=IOComponent_init)
original_Block_get_config = patches.patch(__name__, obj=gr.blocks.Block, field="get_config", replacement=Block_get_config)
original_BlockContext_init = patches.patch(__name__, obj=gr.blocks.BlockContext, field="__init__", replacement=BlockContext_init)
original_Blocks_get_config_file = patches.patch(__name__, obj=gr.blocks.Blocks, field="get_config_file", replacement=Blocks_get_config_file)
ui_tempdir.install_ui_tempdir_override()
modules/hypernetworks/hypernetwork.py
+3 -2
View File
@@ -11,7 +11,7 @@ import tqdm
from einops import rearrange, repeat
from ldm.util import default
from modules import devices, sd_models, shared, sd_samplers, hashes, sd_hijack_checkpoint, errors
from modules.textual_inversion import textual_inversion, logging
from modules.textual_inversion import textual_inversion, saving_settings
from modules.textual_inversion.learn_schedule import LearnRateScheduler
from torch import einsum
from torch.nn.init import normal_, xavier_normal_, xavier_uniform_, kaiming_normal_, kaiming_uniform_, zeros_
@@ -95,6 +95,7 @@ class HypernetworkModule(torch.nn.Module):
zeros_(b)
else:
raise KeyError(f"Key {weight_init} is not defined as initialization!")
devices.torch_npu_set_device()
self.to(devices.device)
def fix_old_state_dict(self, state_dict):
@@ -532,7 +533,7 @@ def train_hypernetwork(id_task, hypernetwork_name: str, learn_rate: float, batch
model_name=checkpoint.model_name, model_hash=checkpoint.shorthash, num_of_dataset_images=len(ds),
**{field: getattr(hypernetwork, field) for field in ['layer_structure', 'activation_func', 'weight_init', 'add_layer_norm', 'use_dropout', ]}
)
logging.save_settings_to_file(log_directory, {**saved_params, **locals()})
saving_settings.save_settings_to_file(log_directory, {**saved_params, **locals()})
latent_sampling_method = ds.latent_sampling_method
modules/images.py
+95 -17
View File
@@ -1,7 +1,7 @@
from __future__ import annotations
import datetime
import functools
import pytz
import io
import math
@@ -12,7 +12,9 @@ import re
import numpy as np
import piexif
import piexif.helper
from PIL import Image, ImageFont, ImageDraw, ImageColor, PngImagePlugin
from PIL import Image, ImageFont, ImageDraw, ImageColor, PngImagePlugin, ImageOps
# pillow_avif needs to be imported somewhere in code for it to work
import pillow_avif # noqa: F401
import string
import json
import hashlib
@@ -52,11 +54,14 @@ def image_grid(imgs, batch_size=1, rows=None):
params = script_callbacks.ImageGridLoopParams(imgs, cols, rows)
script_callbacks.image_grid_callback(params)
w, h = imgs[0].size
grid = Image.new('RGB', size=(params.cols * w, params.rows * h), color='black')
w, h = map(max, zip(*(img.size for img in imgs)))
grid_background_color = ImageColor.getcolor(opts.grid_background_color, 'RGBA')
grid = Image.new('RGBA', size=(params.cols * w, params.rows * h), color=grid_background_color)
for i, img in enumerate(params.imgs):
grid.paste(img, box=(i % params.cols * w, i // params.cols * h))
img_w, img_h = img.size
w_offset, h_offset = 0 if img_w == w else (w - img_w) // 2, 0 if img_h == h else (h - img_h) // 2
grid.paste(img, box=(i % params.cols * w + w_offset, i // params.cols * h + h_offset))
return grid
@@ -244,7 +249,7 @@ def draw_prompt_matrix(im, width, height, all_prompts, margin=0):
return draw_grid_annotations(im, width, height, hor_texts, ver_texts, margin)
def resize_image(resize_mode, im, width, height, upscaler_name=None):
def resize_image(resize_mode, im, width, height, upscaler_name=None, force_RGBA=False):
"""
Resizes an image with the specified resize_mode, width, and height.
@@ -262,7 +267,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None):
upscaler_name = upscaler_name or opts.upscaler_for_img2img
def resize(im, w, h):
if upscaler_name is None or upscaler_name == "None" or im.mode == 'L':
if upscaler_name is None or upscaler_name == "None" or im.mode == 'L' or force_RGBA:
return im.resize((w, h), resample=LANCZOS)
scale = max(w / im.width, h / im.height)
@@ -293,7 +298,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None):
src_h = height if ratio <= src_ratio else im.height * width // im.width
resized = resize(im, src_w, src_h)
res = Image.new("RGB", (width, height))
res = Image.new("RGB" if not force_RGBA else "RGBA", (width, height))
res.paste(resized, box=(width // 2 - src_w // 2, height // 2 - src_h // 2))
else:
@@ -304,7 +309,7 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None):
src_h = height if ratio >= src_ratio else im.height * width // im.width
resized = resize(im, src_w, src_h)
res = Image.new("RGB", (width, height))
res = Image.new("RGB" if not force_RGBA else "RGBA", (width, height))
res.paste(resized, box=(width // 2 - src_w // 2, height // 2 - src_h // 2))
if ratio < src_ratio:
@@ -321,13 +326,16 @@ def resize_image(resize_mode, im, width, height, upscaler_name=None):
return res
invalid_filename_chars = '#<>:"/\\|?*\n\r\t'
if not shared.cmd_opts.unix_filenames_sanitization:
invalid_filename_chars = '#<>:"/\\|?*\n\r\t'
else:
invalid_filename_chars = '/'
invalid_filename_prefix = ' '
invalid_filename_postfix = ' .'
re_nonletters = re.compile(r'[\s' + string.punctuation + ']+')
re_pattern = re.compile(r"(.*?)(?:\[([^\[\]]+)\]|$)")
re_pattern_arg = re.compile(r"(.*)<([^>]*)>$")
max_filename_part_length = 128
max_filename_part_length = shared.cmd_opts.filenames_max_length
NOTHING_AND_SKIP_PREVIOUS_TEXT = object()
@@ -344,8 +352,35 @@ def sanitize_filename_part(text, replace_spaces=True):
return text
@functools.cache
def get_scheduler_str(sampler_name, scheduler_name):
"""Returns {Scheduler} if the scheduler is applicable to the sampler"""
if scheduler_name == 'Automatic':
config = sd_samplers.find_sampler_config(sampler_name)
scheduler_name = config.options.get('scheduler', 'Automatic')
return scheduler_name.capitalize()
@functools.cache
def get_sampler_scheduler_str(sampler_name, scheduler_name):
"""Returns the '{Sampler} {Scheduler}' if the scheduler is applicable to the sampler"""
return f'{sampler_name} {get_scheduler_str(sampler_name, scheduler_name)}'
def get_sampler_scheduler(p, sampler):
"""Returns '{Sampler} {Scheduler}' / '{Scheduler}' / 'NOTHING_AND_SKIP_PREVIOUS_TEXT'"""
if hasattr(p, 'scheduler') and hasattr(p, 'sampler_name'):
if sampler:
sampler_scheduler = get_sampler_scheduler_str(p.sampler_name, p.scheduler)
else:
sampler_scheduler = get_scheduler_str(p.sampler_name, p.scheduler)
return sanitize_filename_part(sampler_scheduler, replace_spaces=False)
return NOTHING_AND_SKIP_PREVIOUS_TEXT
class FilenameGenerator:
replacements = {
'basename': lambda self: self.basename or 'img',
'seed': lambda self: self.seed if self.seed is not None else '',
'seed_first': lambda self: self.seed if self.p.batch_size == 1 else self.p.all_seeds[0],
'seed_last': lambda self: NOTHING_AND_SKIP_PREVIOUS_TEXT if self.p.batch_size == 1 else self.p.all_seeds[-1],
@@ -355,6 +390,8 @@ class FilenameGenerator:
'height': lambda self: self.image.height,
'styles': lambda self: self.p and sanitize_filename_part(", ".join([style for style in self.p.styles if not style == "None"]) or "None", replace_spaces=False),
'sampler': lambda self: self.p and sanitize_filename_part(self.p.sampler_name, replace_spaces=False),
'sampler_scheduler': lambda self: self.p and get_sampler_scheduler(self.p, True),
'scheduler': lambda self: self.p and get_sampler_scheduler(self.p, False),
'model_hash': lambda self: getattr(self.p, "sd_model_hash", shared.sd_model.sd_model_hash),
'model_name': lambda self: sanitize_filename_part(shared.sd_model.sd_checkpoint_info.name_for_extra, replace_spaces=False),
'date': lambda self: datetime.datetime.now().strftime('%Y-%m-%d'),
@@ -380,12 +417,13 @@ class FilenameGenerator:
}
default_time_format = '%Y%m%d%H%M%S'
def __init__(self, p, seed, prompt, image, zip=False):
def __init__(self, p, seed, prompt, image, zip=False, basename=""):
self.p = p
self.seed = seed
self.prompt = prompt
self.image = image
self.zip = zip
self.basename = basename
def get_vae_filename(self):
"""Get the name of the VAE file."""
@@ -566,6 +604,17 @@ def save_image_with_geninfo(image, geninfo, filename, extension=None, existing_p
})
piexif.insert(exif_bytes, filename)
elif extension.lower() == '.avif':
if opts.enable_pnginfo and geninfo is not None:
exif_bytes = piexif.dump({
"Exif": {
piexif.ExifIFD.UserComment: piexif.helper.UserComment.dump(geninfo or "", encoding="unicode")
},
})
else:
exif_bytes = None
image.save(filename,format=image_format, quality=opts.jpeg_quality, exif=exif_bytes)
elif extension.lower() == ".gif":
image.save(filename, format=image_format, comment=geninfo)
else:
@@ -605,12 +654,12 @@ def save_image(image, path, basename, seed=None, prompt=None, extension='png', i
txt_fullfn (`str` or None):
If a text file is saved for this image, this will be its full path. Otherwise None.
"""
namegen = FilenameGenerator(p, seed, prompt, image)
namegen = FilenameGenerator(p, seed, prompt, image, basename=basename)
# WebP and JPG formats have maximum dimension limits of 16383 and 65535 respectively. switch to PNG which has a much higher limit
if (image.height > 65535 or image.width > 65535) and extension.lower() in ("jpg", "jpeg") or (image.height > 16383 or image.width > 16383) and extension.lower() == "webp":
print('Image dimensions too large; saving as PNG')
extension = ".png"
extension = "png"
if save_to_dirs is None:
save_to_dirs = (grid and opts.grid_save_to_dirs) or (not grid and opts.save_to_dirs and not no_prompt)
@@ -744,10 +793,12 @@ def read_info_from_image(image: Image.Image) -> tuple[str | None, dict]:
exif_comment = exif_comment.decode('utf8', errors="ignore")
if exif_comment:
items['exif comment'] = exif_comment
geninfo = exif_comment
elif "comment" in items: # for gif
geninfo = items["comment"].decode('utf8', errors="ignore")
if isinstance(items["comment"], bytes):
geninfo = items["comment"].decode('utf8', errors="ignore")
else:
geninfo = items["comment"]
for field in IGNORED_INFO_KEYS:
items.pop(field, None)
@@ -770,7 +821,7 @@ def image_data(data):
import gradio as gr
try:
image = Image.open(io.BytesIO(data))
image = read(io.BytesIO(data))
textinfo, _ = read_info_from_image(image)
return textinfo, None
except Exception:
@@ -797,3 +848,30 @@ def flatten(img, bgcolor):
return img.convert('RGB')
def read(fp, **kwargs):
image = Image.open(fp, **kwargs)
image = fix_image(image)
return image
def fix_image(image: Image.Image):
if image is None:
return None
try:
image = ImageOps.exif_transpose(image)
image = fix_png_transparency(image)
except Exception:
pass
return image
def fix_png_transparency(image: Image.Image):
if image.mode not in ("RGB", "P") or not isinstance(image.info.get("transparency"), bytes):
return image
image = image.convert("RGBA")
return image
modules/img2img.py
+44 -35
View File
@@ -2,11 +2,10 @@ import os
from contextlib import closing
from pathlib import Path
import numpy as np
from PIL import Image, ImageOps, ImageFilter, ImageEnhance, UnidentifiedImageError
import gradio as gr
from modules import images as imgutil
from modules import images
from modules.infotext_utils import create_override_settings_dict, parse_generation_parameters
from modules.processing import Processed, StableDiffusionProcessingImg2Img, process_images
from modules.shared import opts, state
@@ -18,11 +17,14 @@ import modules.scripts
from modules_forge import main_thread
def process_batch(p, input_dir, output_dir, inpaint_mask_dir, args, to_scale=False, scale_by=1.0, use_png_info=False, png_info_props=None, png_info_dir=None):
def process_batch(p, input, output_dir, inpaint_mask_dir, args, to_scale=False, scale_by=1.0, use_png_info=False, png_info_props=None, png_info_dir=None):
output_dir = output_dir.strip()
processing.fix_seed(p)
images = list(shared.walk_files(input_dir, allowed_extensions=(".png", ".jpg", ".jpeg", ".webp", ".tif", ".tiff")))
if isinstance(input, str):
batch_images = list(shared.walk_files(input, allowed_extensions=(".png", ".jpg", ".jpeg", ".webp", ".tif", ".tiff")))
else:
batch_images = [os.path.abspath(x.name) for x in input]
is_inpaint_batch = False
if inpaint_mask_dir:
@@ -32,9 +34,9 @@ def process_batch(p, input_dir, output_dir, inpaint_mask_dir, args, to_scale=Fal
if is_inpaint_batch:
print(f"\nInpaint batch is enabled. {len(inpaint_masks)} masks found.")
print(f"Will process {len(images)} images, creating {p.n_iter * p.batch_size} new images for each.")
print(f"Will process {len(batch_images)} images, creating {p.n_iter * p.batch_size} new images for each.")
state.job_count = len(images) * p.n_iter
state.job_count = len(batch_images) * p.n_iter
# extract "default" params to use in case getting png info fails
prompt = p.prompt
@@ -47,8 +49,8 @@ def process_batch(p, input_dir, output_dir, inpaint_mask_dir, args, to_scale=Fal
sd_model_checkpoint_override = get_closet_checkpoint_match(override_settings.get("sd_model_checkpoint", None))
batch_results = None
discard_further_results = False
for i, image in enumerate(images):
state.job = f"{i+1} out of {len(images)}"
for i, image in enumerate(batch_images):
state.job = f"{i+1} out of {len(batch_images)}"
if state.skipped:
state.skipped = False
@@ -56,7 +58,7 @@ def process_batch(p, input_dir, output_dir, inpaint_mask_dir, args, to_scale=Fal
break
try:
img = Image.open(image)
img = images.read(image)
except UnidentifiedImageError as e:
print(e)
continue
@@ -87,7 +89,7 @@ def process_batch(p, input_dir, output_dir, inpaint_mask_dir, args, to_scale=Fal
# otherwise user has many masks with the same name but different extensions
mask_image_path = masks_found[0]
mask_image = Image.open(mask_image_path)
mask_image = images.read(mask_image_path)
p.image_mask = mask_image
if use_png_info:
@@ -95,8 +97,8 @@ def process_batch(p, input_dir, output_dir, inpaint_mask_dir, args, to_scale=Fal
info_img = img
if png_info_dir:
info_img_path = os.path.join(png_info_dir, os.path.basename(image))
info_img = Image.open(info_img_path)
geninfo, _ = imgutil.read_info_from_image(info_img)
info_img = images.read(info_img_path)
geninfo, _ = images.read_info_from_image(info_img)
parsed_parameters = parse_generation_parameters(geninfo)
parsed_parameters = {k: v for k, v in parsed_parameters.items() if k in (png_info_props or {})}
except Exception:
@@ -147,38 +149,40 @@ def process_batch(p, input_dir, output_dir, inpaint_mask_dir, args, to_scale=Fal
return batch_results
def img2img_function(id_task: str, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, init_img_with_mask, inpaint_color_sketch, inpaint_color_sketch_orig, init_img_inpaint, init_mask_inpaint, steps: int, sampler_name: str, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, request: gr.Request, *args):
def img2img_function(id_task: str, request: gr.Request, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, sketch_fg, init_img_with_mask, init_img_with_mask_fg, inpaint_color_sketch, inpaint_color_sketch_fg, init_img_inpaint, init_mask_inpaint, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, img2img_batch_source_type: str, img2img_batch_upload: list, *args):
override_settings = create_override_settings_dict(override_settings_texts)
is_batch = mode == 5
height, width = int(height), int(width)
if mode == 0: # img2img
image = init_img
mask = None
elif mode == 1: # img2img sketch
image = sketch
mask = None
image = Image.alpha_composite(sketch, sketch_fg)
elif mode == 2: # inpaint
image, mask = init_img_with_mask["image"], init_img_with_mask["mask"]
mask = processing.create_binary_mask(mask)
image = init_img_with_mask
mask = init_img_with_mask_fg.getchannel('A').convert('L')
mask = Image.merge('RGBA', (mask, mask, mask, Image.new('L', mask.size, 255)))
elif mode == 3: # inpaint sketch
image = inpaint_color_sketch
orig = inpaint_color_sketch_orig or inpaint_color_sketch
pred = np.any(np.array(image) != np.array(orig), axis=-1)
mask = Image.fromarray(pred.astype(np.uint8) * 255, "L")
mask = ImageEnhance.Brightness(mask).enhance(1 - mask_alpha / 100)
blur = ImageFilter.GaussianBlur(mask_blur)
image = Image.composite(image.filter(blur), orig, mask.filter(blur))
image = Image.alpha_composite(inpaint_color_sketch, inpaint_color_sketch_fg)
mask = inpaint_color_sketch_fg.getchannel('A').convert('L')
short_side = min(mask.size)
dilation_size = int(0.015 * short_side) * 2 + 1
mask = mask.filter(ImageFilter.MaxFilter(dilation_size))
mask = Image.merge('RGBA', (mask, mask, mask, Image.new('L', mask.size, 255)))
elif mode == 4: # inpaint upload mask
image = init_img_inpaint
mask = init_mask_inpaint
else:
image = None
mask = None
# Use the EXIF orientation of photos taken by smartphones.
if image is not None:
image = ImageOps.exif_transpose(image)
if mask and isinstance(mask, Image.Image):
mask = mask.point(lambda v: 255 if v > 128 else 0)
image = images.fix_image(image)
mask = images.fix_image(mask)
if selected_scale_tab == 1 and not is_batch:
assert image, "Can't scale by because no image is selected"
@@ -195,10 +199,8 @@ def img2img_function(id_task: str, mode: int, prompt: str, negative_prompt: str,
prompt=prompt,
negative_prompt=negative_prompt,
styles=prompt_styles,
sampler_name=sampler_name,
batch_size=batch_size,
n_iter=n_iter,
steps=steps,
cfg_scale=cfg_scale,
width=width,
height=height,
@@ -225,8 +227,15 @@ def img2img_function(id_task: str, mode: int, prompt: str, negative_prompt: str,
with closing(p):
if is_batch:
assert not shared.cmd_opts.hide_ui_dir_config, "Launched with --hide-ui-dir-config, batch img2img disabled"
processed = process_batch(p, img2img_batch_input_dir, img2img_batch_output_dir, img2img_batch_inpaint_mask_dir, args, to_scale=selected_scale_tab == 1, scale_by=scale_by, use_png_info=img2img_batch_use_png_info, png_info_props=img2img_batch_png_info_props, png_info_dir=img2img_batch_png_info_dir)
if img2img_batch_source_type == "upload":
assert isinstance(img2img_batch_upload, list) and img2img_batch_upload
output_dir = ""
inpaint_mask_dir = ""
png_info_dir = img2img_batch_png_info_dir if not shared.cmd_opts.hide_ui_dir_config else ""
processed = process_batch(p, img2img_batch_upload, output_dir, inpaint_mask_dir, args, to_scale=selected_scale_tab == 1, scale_by=scale_by, use_png_info=img2img_batch_use_png_info, png_info_props=img2img_batch_png_info_props, png_info_dir=png_info_dir)
else: # "from dir"
assert not shared.cmd_opts.hide_ui_dir_config, "Launched with --hide-ui-dir-config, batch img2img disabled"
processed = process_batch(p, img2img_batch_input_dir, img2img_batch_output_dir, img2img_batch_inpaint_mask_dir, args, to_scale=selected_scale_tab == 1, scale_by=scale_by, use_png_info=img2img_batch_use_png_info, png_info_props=img2img_batch_png_info_props, png_info_dir=img2img_batch_png_info_dir)
if processed is None:
processed = Processed(p, [], p.seed, "")
@@ -247,5 +256,5 @@ def img2img_function(id_task: str, mode: int, prompt: str, negative_prompt: str,
return processed.images + processed.extra_images, generation_info_js, plaintext_to_html(processed.info), plaintext_to_html(processed.comments, classname="comments")
def img2img(id_task: str, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, init_img_with_mask, inpaint_color_sketch, inpaint_color_sketch_orig, init_img_inpaint, init_mask_inpaint, steps: int, sampler_name: str, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, request: gr.Request, *args):
return main_thread.run_and_wait_result(img2img_function, id_task, mode, prompt, negative_prompt, prompt_styles, init_img, sketch, init_img_with_mask, inpaint_color_sketch, inpaint_color_sketch_orig, init_img_inpaint, init_mask_inpaint, steps, sampler_name, mask_blur, mask_alpha, inpainting_fill, n_iter, batch_size, cfg_scale, image_cfg_scale, denoising_strength, selected_scale_tab, height, width, scale_by, resize_mode, inpaint_full_res, inpaint_full_res_padding, inpainting_mask_invert, img2img_batch_input_dir, img2img_batch_output_dir, img2img_batch_inpaint_mask_dir, override_settings_texts, img2img_batch_use_png_info, img2img_batch_png_info_props, img2img_batch_png_info_dir, request, *args)
def img2img(id_task: str, request: gr.Request, mode: int, prompt: str, negative_prompt: str, prompt_styles, init_img, sketch, sketch_fg, init_img_with_mask, init_img_with_mask_fg, inpaint_color_sketch, inpaint_color_sketch_fg, init_img_inpaint, init_mask_inpaint, mask_blur: int, mask_alpha: float, inpainting_fill: int, n_iter: int, batch_size: int, cfg_scale: float, image_cfg_scale: float, denoising_strength: float, selected_scale_tab: int, height: int, width: int, scale_by: float, resize_mode: int, inpaint_full_res: bool, inpaint_full_res_padding: int, inpainting_mask_invert: int, img2img_batch_input_dir: str, img2img_batch_output_dir: str, img2img_batch_inpaint_mask_dir: str, override_settings_texts, img2img_batch_use_png_info: bool, img2img_batch_png_info_props: list, img2img_batch_png_info_dir: str, img2img_batch_source_type: str, img2img_batch_upload: list, *args):
return main_thread.run_and_wait_result(img2img_function, id_task, request, mode, prompt, negative_prompt, prompt_styles, init_img, sketch, sketch_fg, init_img_with_mask, init_img_with_mask_fg, inpaint_color_sketch, inpaint_color_sketch_fg, init_img_inpaint, init_mask_inpaint, mask_blur, mask_alpha, inpainting_fill, n_iter, batch_size, cfg_scale, image_cfg_scale, denoising_strength, selected_scale_tab, height, width, scale_by, resize_mode, inpaint_full_res, inpaint_full_res_padding, inpainting_mask_invert, img2img_batch_input_dir, img2img_batch_output_dir, img2img_batch_inpaint_mask_dir, override_settings_texts, img2img_batch_use_png_info, img2img_batch_png_info_props, img2img_batch_png_info_dir, img2img_batch_source_type, img2img_batch_upload, *args)
modules/infotext_utils.py
+72 -37
View File
@@ -8,7 +8,7 @@ import sys
import gradio as gr
from modules.paths import data_path
from modules import shared, ui_tempdir, script_callbacks, processing, infotext_versions
from modules import shared, ui_tempdir, script_callbacks, processing, infotext_versions, images, prompt_parser, errors
from PIL import Image
sys.modules['modules.generation_parameters_copypaste'] = sys.modules[__name__] # alias for old name
@@ -74,29 +74,38 @@ def image_from_url_text(filedata):
if filedata is None:
return None
if type(filedata) == list and filedata and type(filedata[0]) == dict and filedata[0].get("is_file", False):
filedata = filedata[0]
if type(filedata) == dict and filedata.get("is_file", False):
filename = filedata["name"]
is_in_right_dir = ui_tempdir.check_tmp_file(shared.demo, filename)
assert is_in_right_dir, 'trying to open image file outside of allowed directories'
filename = filename.rsplit('?', 1)[0]
return Image.open(filename)
if type(filedata) == list:
if isinstance(filedata, list):
if len(filedata) == 0:
return None
filedata = filedata[0]
if filedata.startswith("data:image/png;base64,"):
filedata = filedata[len("data:image/png;base64,"):]
if isinstance(filedata, dict) and filedata.get("is_file", False):
filedata = filedata
filedata = base64.decodebytes(filedata.encode('utf-8'))
image = Image.open(io.BytesIO(filedata))
return image
filename = None
if type(filedata) == dict and filedata.get("is_file", False):
filename = filedata["name"]
elif isinstance(filedata, tuple) and len(filedata) == 2: # gradio 4.16 sends images from gallery as a list of tuples
return filedata[0]
if filename:
is_in_right_dir = ui_tempdir.check_tmp_file(shared.demo, filename)
assert is_in_right_dir, 'trying to open image file outside of allowed directories'
filename = filename.rsplit('?', 1)[0]
return images.read(filename)
if isinstance(filedata, str):
if filedata.startswith("data:image/png;base64,"):
filedata = filedata[len("data:image/png;base64,"):]
filedata = base64.decodebytes(filedata.encode('utf-8'))
image = images.read(io.BytesIO(filedata))
return image
return None
def add_paste_fields(tabname, init_img, fields, override_settings_component=None):
@@ -138,8 +147,6 @@ def register_paste_params_button(binding: ParamBinding):
def connect_paste_params_buttons():
for binding in registered_param_bindings:
if binding.tabname not in paste_fields:
continue
destination_image_component = paste_fields[binding.tabname]["init_img"]
fields = paste_fields[binding.tabname]["fields"]
override_settings_component = binding.override_settings_component or paste_fields[binding.tabname]["override_settings_component"]
@@ -148,18 +155,19 @@ def connect_paste_params_buttons():
destination_height_component = next(iter([field for field, name in fields if name == "Size-2"] if fields else []), None)
if binding.source_image_component and destination_image_component:
need_send_dementions = destination_width_component and binding.tabname != 'inpaint'
if isinstance(binding.source_image_component, gr.Gallery):
func = send_image_and_dimensions if destination_width_component else image_from_url_text
func = send_image_and_dimensions if need_send_dementions else image_from_url_text
jsfunc = "extract_image_from_gallery"
else:
func = send_image_and_dimensions if destination_width_component else lambda x: x
func = send_image_and_dimensions if need_send_dementions else lambda x: x
jsfunc = None
binding.paste_button.click(
fn=func,
_js=jsfunc,
inputs=[binding.source_image_component],
outputs=[destination_image_component, destination_width_component, destination_height_component] if destination_width_component else [destination_image_component],
outputs=[destination_image_component, destination_width_component, destination_height_component] if need_send_dementions else [destination_image_component],
show_progress=False,
)
@@ -187,6 +195,8 @@ def connect_paste_params_buttons():
def send_image_and_dimensions(x):
if isinstance(x, Image.Image):
img = x
elif isinstance(x, list) and isinstance(x[0], tuple):
img = x[0][0]
else:
img = image_from_url_text(x)
@@ -267,17 +277,6 @@ Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 965400086, Size: 512x512, Model
else:
prompt += ("" if prompt == "" else "\n") + line
if shared.opts.infotext_styles != "Ignore":
found_styles, prompt, negative_prompt = shared.prompt_styles.extract_styles_from_prompt(prompt, negative_prompt)
if shared.opts.infotext_styles == "Apply":
res["Styles array"] = found_styles
elif shared.opts.infotext_styles == "Apply if any" and found_styles:
res["Styles array"] = found_styles
res["Prompt"] = prompt
res["Negative prompt"] = negative_prompt
for k, v in re_param.findall(lastline):
try:
if v[0] == '"' and v[-1] == '"':
@@ -292,6 +291,26 @@ Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 965400086, Size: 512x512, Model
except Exception:
print(f"Error parsing \"{k}: {v}\"")
# Extract styles from prompt
if shared.opts.infotext_styles != "Ignore":
found_styles, prompt_no_styles, negative_prompt_no_styles = shared.prompt_styles.extract_styles_from_prompt(prompt, negative_prompt)
same_hr_styles = True
if ("Hires prompt" in res or "Hires negative prompt" in res) and (infotext_ver > infotext_versions.v180_hr_styles if (infotext_ver := infotext_versions.parse_version(res.get("Version"))) else True):
hr_prompt, hr_negative_prompt = res.get("Hires prompt", prompt), res.get("Hires negative prompt", negative_prompt)
hr_found_styles, hr_prompt_no_styles, hr_negative_prompt_no_styles = shared.prompt_styles.extract_styles_from_prompt(hr_prompt, hr_negative_prompt)
if same_hr_styles := found_styles == hr_found_styles:
res["Hires prompt"] = '' if hr_prompt_no_styles == prompt_no_styles else hr_prompt_no_styles
res['Hires negative prompt'] = '' if hr_negative_prompt_no_styles == negative_prompt_no_styles else hr_negative_prompt_no_styles
if same_hr_styles:
prompt, negative_prompt = prompt_no_styles, negative_prompt_no_styles
if (shared.opts.infotext_styles == "Apply if any" and found_styles) or shared.opts.infotext_styles == "Apply":
res['Styles array'] = found_styles
res["Prompt"] = prompt
res["Negative prompt"] = negative_prompt
# Missing CLIP skip means it was set to 1 (the default)
if "Clip skip" not in res:
res["Clip skip"] = "1"
@@ -307,6 +326,9 @@ Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 965400086, Size: 512x512, Model
if "Hires sampler" not in res:
res["Hires sampler"] = "Use same sampler"
if "Hires schedule type" not in res:
res["Hires schedule type"] = "Use same scheduler"
if "Hires checkpoint" not in res:
res["Hires checkpoint"] = "Use same checkpoint"
@@ -358,9 +380,15 @@ Steps: 20, Sampler: Euler a, CFG scale: 7, Seed: 965400086, Size: 512x512, Model
if "Cache FP16 weight for LoRA" not in res and res["FP8 weight"] != "Disable":
res["Cache FP16 weight for LoRA"] = False
if "Emphasis" not in res:
prompt_attention = prompt_parser.parse_prompt_attention(prompt)
prompt_attention += prompt_parser.parse_prompt_attention(negative_prompt)
prompt_uses_emphasis = len(prompt_attention) != len([p for p in prompt_attention if p[1] == 1.0 or p[0] == 'BREAK'])
if "Emphasis" not in res and prompt_uses_emphasis:
res["Emphasis"] = "Original"
if "Refiner switch by sampling steps" not in res:
res["Refiner switch by sampling steps"] = False
infotext_versions.backcompat(res)
for key in skip_fields:
@@ -396,6 +424,9 @@ def create_override_settings_dict(text_pairs):
res = {}
if not text_pairs:
return res
params = {}
for pair in text_pairs:
k, v = pair.split(":", maxsplit=1)
@@ -458,7 +489,7 @@ def get_override_settings(params, *, skip_fields=None):
def connect_paste(button, paste_fields, input_comp, override_settings_component, tabname):
def paste_func(prompt):
if not prompt and not shared.cmd_opts.hide_ui_dir_config:
if not prompt and not shared.cmd_opts.hide_ui_dir_config and not shared.cmd_opts.no_prompt_history:
filename = os.path.join(data_path, "params.txt")
try:
with open(filename, "r", encoding="utf8") as file:
@@ -472,7 +503,11 @@ def connect_paste(button, paste_fields, input_comp, override_settings_component,
for output, key in paste_fields:
if callable(key):
v = key(params)
try:
v = key(params)
except Exception:
errors.report(f"Error executing {key}", exc_info=True)
v = None
else:
v = params.get(key, None)

Some files were not shown because too many files have changed in this diff Show More