Stable Diffusion: RunPod model catalog - Configurable model catalog (name -> model filename -> download sources, incl. companion files) in the RunPod settings block. - When the ComfyUI URL points at the runpod-lazy proxy, the Model dropdown lists catalog entries instead of querying the backend; selecting one pushes the catalog + active model to the proxy, which downloads exactly that set (re-provisioning a running pod on change). - homelab/: proxy v3 (catalog-driven provisioning, workflow filename matching on wake, /lazy/catalog endpoint).

630785ad4c5344a76994566c0b7fc880a116fa74

permissionBRICK <40219477+permissionBRICK@users.noreply.github.com>

3 files changed, +265 -77Ignore whitespace
homelab/runpod-lazy-proxy.py+118 -77
@@ -1,23 +1,25 @@
1#!/usr/bin/env python31#!/usr/bin/env python3
2"""ComfyUI lazy proxy for a volumeless RunPod on-demand pod.2"""ComfyUI lazy proxy for a volumeless RunPod on-demand pod (catalog edition).
33
4SillyTavern's fallback chain points at this proxy as a normal ComfyUI server;4SillyTavern's fallback chain points at this proxy as a normal ComfyUI server.
5the pod downloads its models at boot (boot-models.py in the image), so no5Models come from a CATALOG pushed by the ST UI: entries of
6network volume or region lock is needed.6 {"name": display, "value": model filename, "files": [{"dest": "unet/x.gguf", "url": "..."}]}
77The pod downloads exactly the selected entry's files at boot (boot-models.py
8Control API (CORS-enabled, used by the ST warmup UI and the CLI script):8reads MODEL_MANIFEST). Changing the active model with a running pod recreates
9 GET /lazy/status -> {"state": "red"|"orange"|"green", ...}9it (fresh disk = old model gone, only the new one downloads).
10 POST /lazy/warmup -> start pod with MODELS=all (idempotent)10
11Control API (CORS-enabled; ST warmup UI + runpod-pod.sh CLI):
12 GET /lazy/status -> {"state": red|orange|green, "model": ..., ...}
13 POST /lazy/warmup -> start pod for the active catalog entry
11 POST /lazy/shutdown -> terminate pod now14 POST /lazy/shutdown -> terminate pod now
12 POST /lazy/ping -> keepalive (extends idle timer; never starts a pod)15 POST /lazy/ping -> keepalive (extends idle timer; never starts a pod)
16 POST /lazy/catalog -> {"models": [...], "active": "<value>"} store catalog;
17 re-provision the pod if the active model changed
1318
14Wake semantics: a ComfyUI /prompt POST arriving with no pod starts one that19Wake semantics: a ComfyUI /prompt arriving with no ready pod matches the
15downloads only the model group the workflow references (flux/qwen/all).20workflow against catalog filenames and provisions for that entry (falls back
16The idle timer lives HERE (NAS side): IDLE_SECONDS after the last activity21to the active entry). The idle timer lives here (NAS side): IDLE_SECONDS after
17signal (comfy traffic or keepalive ping) the pod is terminated - regardless22the last activity signal the pod is terminated, frontend or no frontend.
18of whether any frontend still exists.
19
20stdlib only; runs on python:3.12-alpine.
21"""23"""
22import json24import json
23import os25import os
@@ -30,7 +32,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
30RUNPOD_KEY = os.environ['RUNPOD_KEY']32RUNPOD_KEY = os.environ['RUNPOD_KEY']
31LISTEN_PORT = int(os.environ.get('LISTEN_PORT', '8189'))33LISTEN_PORT = int(os.environ.get('LISTEN_PORT', '8189'))
32IDLE_SECONDS = int(os.environ.get('IDLE_SECONDS', '900'))34IDLE_SECONDS = int(os.environ.get('IDLE_SECONDS', '900'))
33DATACENTERS = [d for d in os.environ.get('DATACENTERS', '').split(',') if d] # empty = any35DATACENTERS = [d for d in os.environ.get('DATACENTERS', '').split(',') if d]
34CLOUD_TYPE = os.environ.get('CLOUD_TYPE', 'SECURE')36CLOUD_TYPE = os.environ.get('CLOUD_TYPE', 'SECURE')
35IMAGE = os.environ.get('IMAGE', 'ghcr.io/permissionbrick/comfyui-runpod-worker:latest')37IMAGE = os.environ.get('IMAGE', 'ghcr.io/permissionbrick/comfyui-runpod-worker:latest')
36GPU_TYPES = os.environ.get('GPU_TYPES', 'NVIDIA RTX 6000 Ada Generation,NVIDIA L40S,NVIDIA L40,NVIDIA RTX A6000,NVIDIA A40').split(',')38GPU_TYPES = os.environ.get('GPU_TYPES', 'NVIDIA RTX 6000 Ada Generation,NVIDIA L40S,NVIDIA L40,NVIDIA RTX A6000,NVIDIA A40').split(',')
@@ -39,13 +41,11 @@ START_TIMEOUT = int(os.environ.get('START_TIMEOUT', '1500'))
39HF_TOKEN = os.environ.get('HF_TOKEN', '')41HF_TOKEN = os.environ.get('HF_TOKEN', '')
40COMFY_ARGS = os.environ.get('COMFY_ARGS', '--listen 0.0.0.0 --port 8188 --use-pytorch-cross-attention')42COMFY_ARGS = os.environ.get('COMFY_ARGS', '--listen 0.0.0.0 --port 8188 --use-pytorch-cross-attention')
41CACHE_FILE = os.environ.get('CACHE_FILE', '/app/system_stats.cache')43CACHE_FILE = os.environ.get('CACHE_FILE', '/app/system_stats.cache')
4244CATALOG_FILE = os.environ.get('CATALOG_FILE', '/app/catalog.json')
43# Model-group detection: filename fragment (lowercased) -> group
44GROUP_HINTS = {'flux-2-klein': 'flux', 'flux2-klein': 'flux', 'flux2-vae': 'flux', 'qwen-rapid': 'qwen'}
4545
46state = {46state = {
47 'pod_id': None, # current pod id (may still be booting)47 'pod_id': None,
48 'models': None, # MODELS value the pod was created with48 'model': None, # catalog `value` the pod was provisioned for
49 'phase': 'red', # red | orange | green49 'phase': 'red', # red | orange | green
50 'gpu': None,50 'gpu': None,
51 'since': time.time(),51 'since': time.time(),
@@ -58,6 +58,32 @@ def log(*args):
58 print(time.strftime('%H:%M:%S'), *args, flush=True)58 print(time.strftime('%H:%M:%S'), *args, flush=True)
5959
6060
61def load_catalog():
62 try:
63 with open(CATALOG_FILE) as f:
64 return json.load(f)
65 except Exception:
66 return {'models': [], 'active': None}
67
68
69def save_catalog(catalog):
70 with open(CATALOG_FILE, 'w') as f:
71 json.dump(catalog, f, indent=2)
72
73
74def catalog_entry(value):
75 if not value:
76 return None
77 for entry in load_catalog().get('models', []):
78 if entry.get('value') == value:
79 return entry
80 return None
81
82
83def active_value():
84 return load_catalog().get('active')
85
86
61def api(method, path, body=None):87def api(method, path, body=None):
62 req = urllib.request.Request(88 req = urllib.request.Request(
63 f'https://rest.runpod.io/v1{path}',89 f'https://rest.runpod.io/v1{path}',
@@ -73,13 +99,20 @@ def find_pod():
73 try:99 try:
74 for pod in api('GET', '/pods'):100 for pod in api('GET', '/pods'):
75 if pod.get('name') == POD_NAME and pod.get('desiredStatus') == 'RUNNING':101 if pod.get('name') == POD_NAME and pod.get('desiredStatus') == 'RUNNING':
76 return pod['id'], (pod.get('env') or {}).get('MODELS'), pod.get('machine', {}).get('gpuTypeId')102 env = pod.get('env') or {}
103 return pod['id'], env.get('MODEL_KEY'), pod.get('machine', {}).get('gpuTypeId')
77 except Exception as err:104 except Exception as err:
78 log('find_pod failed:', err)105 log('find_pod failed:', err)
79 return None, None, None106 return None, None, None
80107
81108
82def create_pod(models):109def create_pod(value):
110 entry = catalog_entry(value)
111 env = {'HF_TOKEN': HF_TOKEN, 'MODEL_KEY': value or ''}
112 if entry and entry.get('files'):
113 env['MODEL_MANIFEST'] = json.dumps(entry['files'])
114 else:
115 env['MODELS'] = 'all' # legacy fallback when no catalog is configured
83 body = {116 body = {
84 'name': POD_NAME,117 'name': POD_NAME,
85 'imageName': IMAGE,118 'imageName': IMAGE,
@@ -88,13 +121,13 @@ def create_pod(models):
88 'cloudType': CLOUD_TYPE,121 'cloudType': CLOUD_TYPE,
89 'containerDiskInGb': 80,122 'containerDiskInGb': 80,
90 'ports': ['8188/http'],123 'ports': ['8188/http'],
91 'env': {'MODELS': models, 'HF_TOKEN': HF_TOKEN},124 'env': env,
92 'dockerStartCmd': ['bash', '-c', f'python3 /boot-models.py && cd /comfyui && exec python main.py {COMFY_ARGS}'],125 'dockerStartCmd': ['bash', '-c', f'python3 /boot-models.py && cd /comfyui && exec python main.py {COMFY_ARGS}'],
93 }126 }
94 if DATACENTERS:127 if DATACENTERS:
95 body['dataCenterIds'] = DATACENTERS128 body['dataCenterIds'] = DATACENTERS
96 pod = api('POST', '/pods', body)129 pod = api('POST', '/pods', body)
97 log(f"pod created: {pod['id']} MODELS={models} {pod.get('machine', {}).get('gpuTypeId')} ${pod.get('costPerHr')}/hr")130 log(f"pod created: {pod['id']} model={value or 'legacy-all'} {pod.get('machine', {}).get('gpuTypeId')} ${pod.get('costPerHr')}/hr")
98 return pod['id'], pod.get('machine', {}).get('gpuTypeId')131 return pod['id'], pod.get('machine', {}).get('gpuTypeId')
99132
100133
@@ -115,35 +148,33 @@ def upstream_ready(pod_id):
115 return False148 return False
116149
117150
118def groups_cover(have, need):151def terminate(pod_id):
119 if not have:152 try:
120 return False153 api('DELETE', f'/pods/{pod_id}')
121 have_set = set(have.split(','))154 log('pod terminated:', pod_id)
122 need_set = set(need.split(','))155 except Exception as err:
123 return 'all' in have_set or need_set <= have_set156 log('terminate failed:', err)
124157
125158
126def ensure_pod(models='all', wait=True):159def ensure_pod(value=None, wait=True):
127 """Ensures a pod covering the requested model groups; optionally waits for readiness."""160 """Ensures a pod provisioned for catalog entry `value` (None = active entry)."""
161 value = value or active_value()
128 with state['lock']:162 with state['lock']:
129 pod_id, have, gpu = (state['pod_id'], state['models'], state['gpu'])163 pod_id, have, gpu = (state['pod_id'], state['model'], state['gpu'])
130 if not pod_id:164 if not pod_id:
131 pod_id, have, gpu = find_pod()165 pod_id, have, gpu = find_pod()
132 if pod_id and not groups_cover(have, models):166 if pod_id and value and have != value:
133 log(f'pod has MODELS={have}, need {models} - recreating with all')167 log(f'pod has model={have}, need {value} - recreating')
134 try:168 terminate(pod_id)
135 api('DELETE', f'/pods/{pod_id}')
136 except Exception as err:
137 log('delete failed:', err)
138 pod_id, have, gpu = None, None, None169 pod_id, have, gpu = None, None, None
139 models = 'all'
140 if not pod_id:170 if not pod_id:
141 state['phase'] = 'orange'171 state['phase'] = 'orange'
142 state['since'] = time.time()172 state['since'] = time.time()
143 last_err = None173 last_err = None
144 for attempt in range(3):174 for attempt in range(3):
145 try:175 try:
146 pod_id, gpu = create_pod(models)176 pod_id, gpu = create_pod(value)
177 have = value
147 break178 break
148 except urllib.error.HTTPError as err:179 except urllib.error.HTTPError as err:
149 last_err = err.read().decode()[:200]180 last_err = err.read().decode()[:200]
@@ -152,8 +183,7 @@ def ensure_pod(models='all', wait=True):
152 if not pod_id:183 if not pod_id:
153 state['phase'] = 'red'184 state['phase'] = 'red'
154 raise RuntimeError(f'could not create pod: {last_err}')185 raise RuntimeError(f'could not create pod: {last_err}')
155 have = models186 state['pod_id'], state['model'], state['gpu'] = pod_id, have, gpu
156 state['pod_id'], state['models'], state['gpu'] = pod_id, have, gpu
157 state['last'] = time.monotonic()187 state['last'] = time.monotonic()
158188
159 if state['phase'] != 'green':189 if state['phase'] != 'green':
@@ -162,12 +192,14 @@ def ensure_pod(models='all', wait=True):
162 return pod_id192 return pod_id
163 deadline = time.monotonic() + START_TIMEOUT193 deadline = time.monotonic() + START_TIMEOUT
164 while time.monotonic() < deadline:194 while time.monotonic() < deadline:
195 # Booting/downloading counts as activity, else the reaper would kill a
196 # warming pod mid-download.
197 state['last'] = time.monotonic()
165 if upstream_ready(pod_id):198 if upstream_ready(pod_id):
166 if state['phase'] != 'green':199 if state['phase'] != 'green':
167 state['phase'] = 'green'200 state['phase'] = 'green'
168 state['since'] = time.time()201 state['since'] = time.time()
169 log('pod ready:', pod_id)202 log('pod ready:', pod_id)
170 state['last'] = time.monotonic()
171 return pod_id203 return pod_id
172 if state['pod_id'] != pod_id:204 if state['pod_id'] != pod_id:
173 raise RuntimeError('pod replaced while waiting')205 raise RuntimeError('pod replaced while waiting')
@@ -175,15 +207,17 @@ def ensure_pod(models='all', wait=True):
175 raise RuntimeError('pod not ready within START_TIMEOUT')207 raise RuntimeError('pod not ready within START_TIMEOUT')
176208
177209
178def detect_groups(workflow_bytes):210def match_workflow_model(workflow_bytes):
211 """Returns the catalog value whose filename appears in the workflow, else None."""
179 try:212 try:
180 text = workflow_bytes.decode('utf-8', 'replace').lower()213 text = workflow_bytes.decode('utf-8', 'replace')
181 except Exception:214 except Exception:
182 return 'all'215 return None
183 found = {group for hint, group in GROUP_HINTS.items() if hint in text}216 for entry in load_catalog().get('models', []):
184 if len(found) == 1:217 value = entry.get('value')
185 return found.pop()218 if value and value in text:
186 return 'all'219 return value
220 return None
187221
188222
189def idle_reaper():223def idle_reaper():
@@ -194,16 +228,12 @@ def idle_reaper():
194 pod_id = state['pod_id'] or find_pod()[0]228 pod_id = state['pod_id'] or find_pod()[0]
195 if pod_id:229 if pod_id:
196 log(f'idle {int(idle)}s - terminating pod')230 log(f'idle {int(idle)}s - terminating pod')
197 try:231 terminate(pod_id)
198 api('DELETE', f'/pods/{pod_id}')
199 except Exception as err:
200 log('terminate failed:', err)
201 if pod_id or state['phase'] != 'red':232 if pod_id or state['phase'] != 'red':
202 state.update({'pod_id': None, 'models': None, 'gpu': None, 'phase': 'red', 'since': time.time()})233 state.update({'pod_id': None, 'model': None, 'gpu': None, 'phase': 'red', 'since': time.time()})
203234
204235
205def status_body():236def status_body():
206 # Refresh green-ness cheaply on demand
207 pod_id = state['pod_id']237 pod_id = state['pod_id']
208 if pod_id and state['phase'] != 'green' and upstream_ready(pod_id):238 if pod_id and state['phase'] != 'green' and upstream_ready(pod_id):
209 state['phase'] = 'green'239 state['phase'] = 'green'
@@ -211,14 +241,15 @@ def status_body():
211 if not pod_id:241 if not pod_id:
212 found, have, gpu = find_pod()242 found, have, gpu = find_pod()
213 if found:243 if found:
214 state.update({'pod_id': found, 'models': have, 'gpu': gpu})244 state.update({'pod_id': found, 'model': have, 'gpu': gpu})
215 state['phase'] = 'green' if upstream_ready(found) else 'orange'245 state['phase'] = 'green' if upstream_ready(found) else 'orange'
216 else:246 elif state['phase'] != 'orange':
217 state['phase'] = 'red'247 state['phase'] = 'red'
218 return json.dumps({248 return json.dumps({
219 'state': state['phase'],249 'state': state['phase'],
220 'pod_id': state['pod_id'],250 'pod_id': state['pod_id'],
221 'models': state['models'],251 'model': state['model'],
252 'active': active_value(),
222 'gpu': state['gpu'],253 'gpu': state['gpu'],
223 'since': state['since'],254 'since': state['since'],
224 'url': pod_url(state['pod_id']) if state['pod_id'] else None,255 'url': pod_url(state['pod_id']) if state['pod_id'] else None,
@@ -258,34 +289,44 @@ class Proxy(BaseHTTPRequestHandler):
258 state['last'] = time.monotonic()289 state['last'] = time.monotonic()
259 return self._reply(200, b'{"ok": true}')290 return self._reply(200, b'{"ok": true}')
260 if path == '/lazy/warmup':291 if path == '/lazy/warmup':
261 threading.Thread(target=self._safe_ensure, args=('all',), daemon=True).start()292 threading.Thread(target=self._safe_ensure, args=(None,), daemon=True).start()
262 state['phase'] = state['phase'] if state['phase'] == 'green' else 'orange'293 if state['phase'] != 'green':
294 state['phase'] = 'orange'
263 return self._reply(200, status_body())295 return self._reply(200, status_body())
264 if path == '/lazy/shutdown':296 if path == '/lazy/shutdown':
265 pod_id = state['pod_id'] or find_pod()[0]297 pod_id = state['pod_id'] or find_pod()[0]
266 if pod_id:298 if pod_id:
267 try:299 terminate(pod_id)
268 api('DELETE', f'/pods/{pod_id}')300 state.update({'pod_id': None, 'model': None, 'gpu': None, 'phase': 'red', 'since': time.time()})
269 except Exception as err:
270 return self._reply(500, json.dumps({'error': str(err)}).encode())
271 state.update({'pod_id': None, 'models': None, 'gpu': None, 'phase': 'red', 'since': time.time()})
272 return self._reply(200, status_body())301 return self._reply(200, status_body())
302 if path == '/lazy/catalog':
303 try:
304 payload = json.loads(body or b'{}')
305 catalog = {'models': payload.get('models', []), 'active': payload.get('active')}
306 save_catalog(catalog)
307 log(f"catalog updated: {len(catalog['models'])} models, active={catalog['active']}")
308 # Re-provision proactively when the active model changed under a live pod.
309 if state['pod_id'] and catalog['active'] and state['model'] != catalog['active']:
310 log('active model changed - re-provisioning pod')
311 threading.Thread(target=self._safe_ensure, args=(catalog['active'],), daemon=True).start()
312 return self._reply(200, status_body())
313 except Exception as err:
314 return self._reply(400, json.dumps({'error': str(err)}).encode())
273315
274 # ---- comfy proxying ----316 # ---- comfy proxying ----
275 state['last'] = time.monotonic()317 state['last'] = time.monotonic()
276 pod_id = state['pod_id']318 pod_id = state['pod_id']
277 ready = pod_id and upstream_ready(pod_id)319 ready = pod_id and upstream_ready(pod_id)
278 if not ready and self.command == 'GET' and path == '/system_stats':320 if not ready and self.command == 'GET' and path == '/system_stats':
279 # Reachability probe while cold: answer from cache; do NOT wake.
280 try:321 try:
281 with open(CACHE_FILE, 'rb') as f:322 with open(CACHE_FILE, 'rb') as f:
282 return self._reply(200, f.read())323 return self._reply(200, f.read())
283 except FileNotFoundError:324 except FileNotFoundError:
284 return self._reply(200, json.dumps({'system': {'comfyui_version': 'cold'}, 'devices': []}).encode())325 return self._reply(200, json.dumps({'system': {'comfyui_version': 'cold'}, 'devices': []}).encode())
285 if not ready:326 if not ready or (self.command == 'POST' and path == '/prompt'):
286 models = detect_groups(body) if (self.command == 'POST' and path == '/prompt' and body) else 'all'327 value = match_workflow_model(body) if (self.command == 'POST' and path == '/prompt' and body) else None
287 try:328 try:
288 pod_id = ensure_pod(models)329 pod_id = ensure_pod(value)
289 except Exception as err:330 except Exception as err:
290 log('ensure_pod failed:', err)331 log('ensure_pod failed:', err)
291 return self._reply(502, json.dumps({'error': str(err)}).encode())332 return self._reply(502, json.dumps({'error': str(err)}).encode())
@@ -304,11 +345,11 @@ class Proxy(BaseHTTPRequestHandler):
304 finally:345 finally:
305 state['last'] = time.monotonic()346 state['last'] = time.monotonic()
306347
307 def _safe_ensure(self, models):348 def _safe_ensure(self, value):
308 try:349 try:
309 ensure_pod(models)350 ensure_pod(value)
310 except Exception as err:351 except Exception as err:
311 log('warmup failed:', err)352 log('provision failed:', err)
312353
313 do_GET = _handle354 do_GET = _handle
314 do_POST = _handle355 do_POST = _handle
@@ -317,7 +358,7 @@ class Proxy(BaseHTTPRequestHandler):
317def main():358def main():
318 threading.Thread(target=idle_reaper, daemon=True).start()359 threading.Thread(target=idle_reaper, daemon=True).start()
319 server = ThreadingHTTPServer(('0.0.0.0', LISTEN_PORT), Proxy)360 server = ThreadingHTTPServer(('0.0.0.0', LISTEN_PORT), Proxy)
320 log(f'runpod-lazy v2 on :{LISTEN_PORT} (pod "{POD_NAME}", DCs {DATACENTERS or "any"}, idle {IDLE_SECONDS}s)')361 log(f'runpod-lazy v3 (catalog) on :{LISTEN_PORT} (pod "{POD_NAME}", DCs {DATACENTERS or "any"}, idle {IDLE_SECONDS}s)')
321 server.serve_forever()362 server.serve_forever()
322363
323364
public/scripts/extensions/stable-diffusion/index.js+138 -0
@@ -376,6 +376,9 @@ const defaultSettings = {
376 // RunPod lazy-pod proxy base URL ('' = feature off)376 // RunPod lazy-pod proxy base URL ('' = feature off)
377 runpod_lazy_url: '',377 runpod_lazy_url: '',
378378
379 // RunPod model catalog ({ name, value, downloads } entries)
380 runpod_models: [],
381
379 // Dedicated LLM connection profile for image-prompt generation ('' = use active model)382 // Dedicated LLM connection profile for image-prompt generation ('' = use active model)
380 prompt_generation_profile: '',383 prompt_generation_profile: '',
381384
@@ -396,6 +399,7 @@ const PRESET_EXCLUDE_KEYS = [
396 'ref_images_enabled',399 'ref_images_enabled',
397 'ref_images',400 'ref_images',
398 'runpod_lazy_url',401 'runpod_lazy_url',
402 'runpod_models',
399 // The image-prompt LLM profile is independent of the image backend, so it must403 // The image-prompt LLM profile is independent of the image backend, so it must
400 // never be captured/swapped by image-generation presets or the fallback retry.404 // never be captured/swapped by image-generation presets or the fallback retry.
401 'prompt_generation_profile',405 'prompt_generation_profile',
@@ -713,6 +717,10 @@ async function loadSettings() {
713 extension_settings.sd.ref_images = [];717 extension_settings.sd.ref_images = [];
714 }718 }
715719
720 if (!Array.isArray(extension_settings.sd.runpod_models)) {
721 extension_settings.sd.runpod_models = [];
722 }
723
716 if (!Array.isArray(extension_settings.sd.custom_entries)) {724 if (!Array.isArray(extension_settings.sd.custom_entries)) {
717 extension_settings.sd.custom_entries = [];725 extension_settings.sd.custom_entries = [];
718 }726 }
@@ -787,7 +795,9 @@ async function loadSettings() {
787 $('#sd_runpod_lazy_url').val(extension_settings.sd.runpod_lazy_url ?? '');795 $('#sd_runpod_lazy_url').val(extension_settings.sd.runpod_lazy_url ?? '');
788 renderPresetChain();796 renderPresetChain();
789 renderRefImages();797 renderRefImages();
798 renderRunpodModels();
790 setupRunpodLoops();799 setupRunpodLoops();
800 pushRunpodCatalog();
791801
792 for (const style of extension_settings.sd.styles) {802 for (const style of extension_settings.sd.styles) {
793 const option = document.createElement('option');803 const option = document.createElement('option');
@@ -1594,6 +1604,122 @@ function onRunpodLazyUrlInput() {
1594 setupRunpodLoops();1604 setupRunpodLoops();
1595}1605}
15961606
1607/**
1608 * Whether a comfy URL points at the runpod-lazy proxy.
1609 * @param {string} url ComfyUI URL to test.
1610 * @returns {boolean} True when it equals the configured proxy URL.
1611 */
1612function isRunpodProxyUrl(url) {
1613 const lazy = getRunpodLazyUrl();
1614 return !!lazy && String(url ?? '').trim().replace(/\/$/, '') === lazy;
1615}
1616
1617/**
1618 * Returns the configured RunPod model catalog entries that have a filename.
1619 * @returns {{name: string, value: string, downloads: string}[]} Catalog entries.
1620 */
1621function getRunpodCatalog() {
1622 const models = Array.isArray(extension_settings.sd.runpod_models) ? extension_settings.sd.runpod_models : [];
1623 return models.filter(m => m && String(m.value ?? '').trim());
1624}
1625
1626/**
1627 * Parses a catalog entry's download lines ("dest-subpath url" per line).
1628 * @param {string} text Multiline downloads definition.
1629 * @returns {{dest: string, url: string}[]} Parsed file list.
1630 */
1631function parseRunpodFiles(text) {
1632 return String(text ?? '').split('\n')
1633 .map(line => line.trim())
1634 .filter(Boolean)
1635 .map(line => {
1636 const space = line.indexOf(' ');
1637 return space > 0 ? { dest: line.slice(0, space).trim(), url: line.slice(space + 1).trim() } : null;
1638 })
1639 .filter(x => x && x.dest && x.url);
1640}
1641
1642/**
1643 * The model filename saved for whichever config uses the proxy: the live
1644 * settings if they point at it, else the chain entry that does.
1645 * @returns {string|null} Active catalog value.
1646 */
1647function getRunpodActiveModel() {
1648 if (isRunpodProxyUrl(extension_settings.sd.comfy_url)) {
1649 return extension_settings.sd.model || null;
1650 }
1651 const chain = Array.isArray(extension_settings.sd.settings_preset_chain) ? extension_settings.sd.settings_preset_chain : [];
1652 const entry = chain.find(e => isRunpodProxyUrl(e?.preset?.comfy_url));
1653 return entry?.preset?.model ?? null;
1654}
1655
1656/** Pushes the model catalog + active selection to the proxy (fire-and-forget). */
1657async function pushRunpodCatalog() {
1658 const url = getRunpodLazyUrl();
1659 if (!url) {
1660 return;
1661 }
1662 const models = getRunpodCatalog().map(m => ({
1663 name: m.name || m.value,
1664 value: m.value,
1665 files: parseRunpodFiles(m.downloads),
1666 }));
1667 try {
1668 await fetch(`${url}/lazy/catalog`, {
1669 method: 'POST',
1670 headers: { 'Content-Type': 'application/json' },
1671 signal: AbortSignal.timeout(8000),
1672 body: JSON.stringify({ models, active: getRunpodActiveModel() }),
1673 });
1674 } catch (error) {
1675 console.warn('SD: runpod catalog push failed', error);
1676 }
1677}
1678
1679/**
1680 * Rebuilds the RunPod model catalog editor in the settings UI.
1681 */
1682function renderRunpodModels() {
1683 const container = $('#sd_runpod_models_list');
1684 if (!container.length) {
1685 return;
1686 }
1687 container.empty();
1688 const models = Array.isArray(extension_settings.sd.runpod_models) ? extension_settings.sd.runpod_models : [];
1689 if (models.length === 0) {
1690 container.append($('<small></small>').text('No models configured.'));
1691 return;
1692 }
1693 models.forEach((model, index) => {
1694 const nameInput = $('<input>').addClass('text_pole flex1').attr({ type: 'text', placeholder: 'Display name' })
1695 .val(model.name || '')
1696 .on('change', function () { model.name = String($(this).val() ?? '').trim(); saveSettingsDebounced(); pushRunpodCatalog(); });
1697 const valueInput = $('<input>').addClass('text_pole flex1').attr({ type: 'text', placeholder: 'Model filename (used by %model%)' })
1698 .val(model.value || '')
1699 .on('change', function () { model.value = String($(this).val() ?? '').trim(); saveSettingsDebounced(); pushRunpodCatalog(); });
1700 const deleteButton = $('<div></div>').addClass('menu_button menu_button_icon fa-solid fa-trash-can')
1701 .attr('title', 'Remove model')
1702 .on('click', () => { models.splice(index, 1); saveSettingsDebounced(); renderRunpodModels(); pushRunpodCatalog(); });
1703 const downloadsInput = $('<textarea></textarea>').addClass('text_pole textarea_compact')
1704 .attr({ rows: 2, placeholder: 'One file per line: <models-subpath> <url>\ne.g. unet/model.gguf https://huggingface.co/...' })
1705 .val(model.downloads || '')
1706 .on('change', function () { model.downloads = String($(this).val() ?? ''); saveSettingsDebounced(); pushRunpodCatalog(); });
1707 container.append(
1708 $('<div></div>').addClass('flexFlowColumn marginTopBot5')
1709 .append($('<div></div>').addClass('flex-container alignItemsCenter').append(nameInput).append(valueInput).append(deleteButton))
1710 .append(downloadsInput));
1711 });
1712}
1713
1714function onRunpodModelsAddClick() {
1715 if (!Array.isArray(extension_settings.sd.runpod_models)) {
1716 extension_settings.sd.runpod_models = [];
1717 }
1718 extension_settings.sd.runpod_models.push({ name: '', value: '', downloads: '' });
1719 saveSettingsDebounced();
1720 renderRunpodModels();
1721}
1722
1597// #endregion1723// #endregion
15981724
1599/**1725/**
@@ -2452,6 +2578,12 @@ async function onModelChange() {
2452 extension_settings.sd.model = selectedModel.val();2578 extension_settings.sd.model = selectedModel.val();
2453 saveSettingsDebounced();2579 saveSettingsDebounced();
24542580
2581 // Selecting a catalog model while pointed at the RunPod proxy kicks off the
2582 // download / re-provision for it right away.
2583 if (isRunpodProxyUrl(extension_settings.sd.comfy_url)) {
2584 pushRunpodCatalog();
2585 }
2586
2455 if (extension_settings.sd.model && extension_settings.sd.source === sources.electronhub) {2587 if (extension_settings.sd.model && extension_settings.sd.source === sources.electronhub) {
2456 const cachedModel = selectedModel.data('model');2588 const cachedModel = selectedModel.data('model');
2457 const models = cachedModel ? [cachedModel] : await loadElectronHubModels();2589 const models = cachedModel ? [cachedModel] : await loadElectronHubModels();
@@ -3502,6 +3634,11 @@ function loadNovelSchedulers() {
3502}3634}
35033635
3504async function loadComfyModels() {3636async function loadComfyModels() {
3637 if (isRunpodProxyUrl(extension_settings.sd.comfy_url)) {
3638 // The RunPod pod downloads models on demand: list the configured catalog
3639 // instead of asking the (possibly cold) backend what it has on disk.
3640 return getRunpodCatalog().map(m => ({ value: m.value, text: m.name || m.value }));
3641 }
3505 if (extension_settings.sd.comfy_type === comfyTypes.runpod_serverless) {3642 if (extension_settings.sd.comfy_type === comfyTypes.runpod_serverless) {
3506 $('#sd_runpod_key').toggleClass('success', !!secret_state[SECRET_KEYS.COMFY_RUNPOD]);3643 $('#sd_runpod_key').toggleClass('success', !!secret_state[SECRET_KEYS.COMFY_RUNPOD]);
3507 return [3644 return [
@@ -7045,6 +7182,7 @@ export async function init() {
7045 $('#sd_runpod_lazy_url').on('input', onRunpodLazyUrlInput);7182 $('#sd_runpod_lazy_url').on('input', onRunpodLazyUrlInput);
7046 $('#sd_runpod_warmup').on('click', () => runpodControl('warmup'));7183 $('#sd_runpod_warmup').on('click', () => runpodControl('warmup'));
7047 $('#sd_runpod_shutdown').on('click', () => runpodControl('shutdown'));7184 $('#sd_runpod_shutdown').on('click', () => runpodControl('shutdown'));
7185 $('#sd_runpod_models_add').on('click', onRunpodModelsAddClick);
7048 $('#sd_custom_entry_add').on('click', onAddCustomEntryClick);7186 $('#sd_custom_entry_add').on('click', onAddCustomEntryClick);
7049 $('#sd_custom_entries_list').on('click', '[data-action]', function () {7187 $('#sd_custom_entries_list').on('click', '[data-action]', function () {
7050 const id = $(this).attr('data-entry-id');7188 const id = $(this).attr('data-entry-id');
public/scripts/extensions/stable-diffusion/settings.html+9 -0
@@ -94,6 +94,15 @@
94 </a>94 </a>
95 </div>95 </div>
96 <small data-i18n="sd_runpod_small">While a SillyTavern tab is open, a keepalive ping keeps the pod alive; it shuts down 15 minutes after the last activity either way. Red = off, orange = starting/downloading models, green = ready.</small>96 <small data-i18n="sd_runpod_small">While a SillyTavern tab is open, a keepalive ping keeps the pod alive; it shuts down 15 minutes after the last activity either way. Red = off, orange = starting/downloading models, green = ready.</small>
97 <h5 data-i18n="Model catalog">Model catalog</h5>
98 <small data-i18n="sd_runpod_models_small">When the ComfyUI URL points at the proxy, the Model dropdown lists these entries instead of asking the backend. The pod downloads exactly the selected entry's files at boot; changing the selection re-provisions with the new model. Filename = what &quot;%model%&quot; resolves to; downloads = one &quot;subpath url&quot; per line, including companion files (text encoder, VAE, LoRA).</small>
99 <div id="sd_runpod_models_list" class="flex-container flexFlowColumn marginTopBot5"></div>
100 <div class="flex-container marginTopBot5">
101 <div id="sd_runpod_models_add" class="menu_button menu_button_icon">
102 <i class="fa-solid fa-plus"></i>
103 <span data-i18n="Add model">Add model</span>
104 </div>
105 </div>
97 <hr>106 <hr>
98 <label for="sd_source" data-i18n="Source">Source</label>107 <label for="sd_source" data-i18n="Source">Source</label>
99 <select id="sd_source" class="text_pole">108 <select id="sd_source" class="text_pole">