Nodes for messing with a model's layers in ComfyUI.

For CLIP, T5, and Flux! ❗🤖🔀🤖❓

V2: Add "skip" (whole) layers in single / double blocks
Separate folder to avoid conflicts with existing workflows
Right click -> "repair node" if using existing v1 workflows with this
⚠️ User reported: For mobile / portable ComfyUI, copy content of my ComfyUI* folder (i.e. both .py files) to ComfyUI/custom_nodes directly.✅
Example of skipping 5 vs. a third of total (center) layers in single + double blocks:

CLI script (Diffusers) that allows you to shuffle layer components (attention, MLP, ...) around.
Agonizing CPU offload, minutes to image - but runs on 4 GB VRAM. Insert your own Flux pipe to optimize it.
Check out the example-images and corresponding settings txt to get started!
ComfyUI nodes: Put the folder "ComfyUI_CLIPFluxShuffle" into "ComfyUI/custom_nodes". Launch Comfy.
Right click -> Add Node -> CLIP-Flux-Shuffle. Or use workflows from 'workflows' folder.
You can use the CLIP + T5 nodes to see what each AI contributes (see "hierarchical" image for an idea)!
You probably can't use the Flux node. :/ It peaks at ~70 GB RAM use. Yes, in ADDITION to OS stuff.
TO-DO: Need to figure out memory / model management and find a better way.
I am currently creating a deepcopy of a deepcopy (of the full model), as it appears buffers etc also get messed up during layer shuffle. Restoring from state_dict (deepcopy thereof) didn't seem to work. Re-loading / instantiating the model from file every time is also pretty unfeasible. Pull requests or even simple hint drops welcome (please open an issue)! Thank you!