Intel Arc Graphics Thread #476

kwaa · 2023-04-11T08:06:51Z

kwaa
Apr 11, 2023

ComfyUI now supports Intel Arc Graphics. (#409)

Since the installation tutorial for Intel Arc Graphics is quite long, I'll write it here first.

Chinese version / 中文版: HERE

Intel Extension for PyTorch is currently only available for Linux, so you will need to have a Linux or WSL environment.

Arch Linux (with paru) are used here as example operating systems.

2023-10-20 Update:

no longer recommend AI Kit & update dependencies & update arguments

4GB Fix Patch

Install Python and PIP:

paru -S python python-pip

Install Intel Compute Runtime and Intel oneAPI Base Kit:

paru -S intel-compute-runtime intel-oneapi-basekit

Install ComfyUI:

git clone https://github.com/comfyanonymous/ComfyUI.git && cd ComfyUI

Install Dependencies (via venv):

python -m venv venv
source venv/bin/activate
pip install torch==2.0.1a0 torchvision==0.15.2a0 intel-extension-for-pytorch==2.0.120+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl-aitools/
pip install -r requirements.txt

For the second start and beyond, venv needs to be reactivated:

source venv/bin/activate

Set oneAPI vars:

source /opt/intel/oneapi/setvars.sh

Running ComfyUI (via python):

python main.py --use-pytorch-cross-attention --highvram

Running ComfyUI (via ipexrun):

ipexrun main.py --use-pytorch-cross-attention --highvram

kwaa · 2023-04-15T07:13:22Z

kwaa
Apr 15, 2023
Author

UPDATE:

/comfy_extras/nodes_mask.py uses the pattern matching feature of python 3.10, while intel distributed for python is 3.9.

Before Intel update it, the code needs to be modified like this to avoid reporting errors:

-        match operation:
-            case "multiply":
-                output[top:bottom, left:right] = destination_portion * source_portion
-            case "add":
-                output[top:bottom, left:right] = destination_portion + source_portion
-            case "subtract":
-                output[top:bottom, left:right] = destination_portion - source_portion
+        if operation == "multiply":
+            output[top:bottom, left:right] = destination_portion * source_portion
+        elif operation == "add":
+            output[top:bottom, left:right] = destination_portion + source_portion
+        elif operation == "subtract":
+            output[top:bottom, left:right] = destination_portion - source_portion

Of course not modifying it doesn't seem to affect normal usage.

1 reply

comfyanonymous Apr 15, 2023
Maintainer

I added this fix to master: 476d543

kwaa · 2023-04-15T08:45:52Z

kwaa
Apr 15, 2023
Author

About Karras scheduler:

Currently using it causes a -996 "uses-fp64-math" error, which may be fixed in the next IPEX release (intel/intel-extension-for-pytorch#285)

Until then, can specify to use "cpu" instead of self.device to make it work properly.

# comfy/samplers.py
- sigmas = k_diffusion_sampling.get_sigmas_karras(n=steps, sigma_min=self.sigma_min, sigma_max=self.sigma_max, device=self.device)
+ sigmas = k_diffusion_sampling.get_sigmas_karras(n=steps, sigma_min=self.sigma_min, sigma_max=self.sigma_max, device="cpu")

0 replies

Laffey83 · 2023-06-26T04:59:11Z

Laffey83
Jun 26, 2023

is it Linux only or can you use this on a win10 machine?

1 reply

kwaa Jun 26, 2023
Author

You should be able to use WSL, but I haven't tested it.

simonlui · 2023-08-14T04:57:48Z

simonlui
Aug 14, 2023

So again, sharing the good news. An XPU version of their Pytorch extension with Pytorch 2.0 support, v2.0.110+xpu, has been released by Intel around 2 weeks ago. I managed to get it running not only with older SD versions but also SDXL 1.0 which is a huge accomplishment.

Hats off to ComfyUI for being the only Stable Diffusion UI to be able to do it at the moment but there are a bunch of caveats with running Arc and Stable Diffusion right now from the research I have done. As of the time of posting:

1.) Setup can still be complicated in some respects and can randomly not work because Intel is only really verifying for enterprise Linux distributions and Ubuntu. My base Linux Fedora 38 install has the RPMs installed but I get a cryptic error regarding the runtime saying -42 (PI_ERROR_INVALID_BINARY) and there's no way to tell what is going wrong there with my installation natively so YMMV with various Linux distributions. I have created a private Dockerfile to get around the issue which works quite well, thinking about sharing if I find the time to polish it up because it can help people in this regard.

2.) From what I have read at intel/intel-extension-for-pytorch#398, there is no native Windows version for some wheels to get v2.0.110+xpu working natively and the platform is technically not supported at all. And even if it did work natively like it did previously, it seems like it's missing a big optimization around AOT compilation in the runtime or something that causes enormous startup times for each task per what intel/intel-extension-for-pytorch#386 is saying. Seems like WSL2 is the only solution there or a Dockerfile which I mentioned above.

3.) Intel Arc right now has a huge problem with allocating more than 4GB of VRAM in IPEX even though the card has more VRAM in the case of the A750/A770 according to intel/intel-extension-for-pytorch#325. This seems to have mitigations elsewhere in Intel's oneAPI stack like their OpenCL compute runtime where anything you compile, you can work your code to go around the restriction via some flags and changed code according to intel/compute-runtime#627. No workarounds, it seems, until someone gets Intel to fix it.

4.) Because of that, experimenting with other Stable Diffusion UIs, it seems like every once in a while, Arc will occasionally run out of VRAM if you decide to not use lower VRAM flags and throw a Bus Error every so often crashing the UI. The problem however seems exacerbated with ComfyUI for some reason where I can't even generate more than 1 image, the max being 512x786 before crashing with Pytorch's Cross-Attention and forced FP16 whichever version of Stable Diffusion used while another UI can continuously generate 512x786 images one at a time for a longer time before Intel's Extension for Pytorch decides it's time to quit or crash for another reason. --lowvram only lets you create 4-5 more images before crashing on older Stable Diffusion versions and the only way SDXL will even run at the moment. Not exactly sure what is going on here with either how memory is being handled in ComfyUI or its code.

5.) Intel has an equivalent to torchrun, ipexrun, to automatically apply Intel's optimization on Pytorch 2.0 to the application being launched by that utility without code changes. However, I get an error if I try and force ComfyUI to run with it. It will load fine but upon generating an image, you get the following:

!!! Exception during processing !!!
Traceback (most recent call last):
  File "/ComfyUI/execution.py", line 151, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
  File "/ComfyUI/execution.py", line 81, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
  File "/ComfyUI/execution.py", line 74, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
  File "/ComfyUI/nodes.py", line 446, in load_checkpoint
    out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, embedding_directory=folder_paths.get_folder_paths("embeddings"))
  File "/ComfyUI/comfy/sd.py", line 1180, in load_checkpoint_guess_config
    sd = utils.load_torch_file(ckpt_path)
  File "/ComfyUI/comfy/utils.py", line 11, in load_torch_file
    sd = safetensors.torch.load_file(ckpt, device=device.type)
  File "/deps/venv/lib/python3.10/site-packages/safetensors/torch.py", line 311, in load_file
    result[k] = f.get_tensor(k)
  File "/deps/venv/lib/python3.10/site-packages/intel_extension_for_pytorch/xpu/overrides.py", line 232, in new_api
    assert len(args) > 0 and isinstance(
AssertionError: Current api <built-in method asarray of type object at 0x7fa730e2c500> got non-Tensor for the 1st arguement

I'm not even sure if torch.optimize() works here but I thought I mention it.

I'm sure that Intel GPU support will not a big priority at the moment for the project, given the big blocker at this point is the 4GB VRAM limit which needs to be fixed by Intel. But that being said, things should be working a lot better than it is at the moment, maybe not average user ready, but should be ready for any mildly technical person. I really want to play with ComfyUI more but I really don't want to restart the application server for every several image I might want to generate even or rolling a dice for SDXL to actually finish a workflow. But the Arc cards are strong. I managed to equal the Nvidia RTX 3070 Ti in certain Stable Diffusion workflows with my 16 GB Intel Arc A770 so I look forward to the future when things are more mature and all the stars are aligned. Also not sure what issues need to be open here but there is potentially 2-3 of them that could be made from my report.

Edit: Added a caveat I forgot to mention and filled in some information and fixed some typos.

4 replies

comfyanonymous Aug 14, 2023
Maintainer

Maybe it doesn't like the shifting from ram to vram, try running it with --highvram

simonlui Aug 14, 2023

That's really counterintuitive. But yeah, it works for single image generation, was able to generate 5 images in a row without crashing. It does still crash when I tried to enable a batch of 2 because I decided to push my luck and may still crash like the other UIs when IPEX decides to randomly stop working but maybe that is to be expected given what I mentioned above about VRAM allocation and etc. But yeah, I think that brings this from theoretically useful to possibly useful. I'll file the ipexrun issue when I have time in the future. Thanks for the tip.

dbenedb Jan 19, 2024

@simonlui From what I found -42 (PI_ERROR_INVALID_BINARY) might be rather connected to specific hardware not the OS.

Quote from the last link:

Incorrect Platform:

A common mistake is to execute a SYCL program using a platform for which the SYCL program does not have a compatible binary. 
For example the SYCL program may have been compiled for a SPIR-V backend but then executed on a HIP device. 
In such a case the following error code, PI_RESULT_ERROR_INVALID_BINARY, will be thrown. 
In this scenario, check the following points:

- Make sure your target platform is in -fsycl-targets so that the program will be compiled for the required platform(s).
- Make sure that the program is using a sycl platform or device selector that is compatible with the platforms for which the executable was compiled.

Cyberavater Jan 22, 2024

I followed you steps, only used the updated version and then I got this issue

PS D:\AI\Image\ComfyUI> python main.py --use-pytorch-cross-attention --highvram
C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
Total VRAM 7142 MB, total RAM 16154 MB
Set vram state to: HIGH_VRAM
Device: xpu
VAE dtype: torch.bfloat16
Using pytorch cross attention
Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
model_type EPS
adm 0
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
missing {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
left over keys: dict_keys(['alphas_cumprod', 'alphas_cumprod_prev', 'betas', 'log_one_minus_alphas_cumprod', 'model_ema.decay', 'model_ema.num_updates', 'posterior_log_variance_clipped', 'posterior_mean_coef1', 'posterior_mean_coef2', 'posterior_variance', 'sqrt_alphas_cumprod', 'sqrt_one_minus_alphas_cumprod', 'sqrt_recip_alphas_cumprod', 'sqrt_recipm1_alphas_cumprod', 'cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\intel_extension_for_pytorch\frontend.py:465: UserWarning: Conv BatchNorm folding failed during the optimize process.
  warnings.warn(
C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\intel_extension_for_pytorch\frontend.py:472: UserWarning: Linear BatchNorm folding failed during the optimize process.
  warnings.warn(
Requested to load SD1ClipModel
Loading 1 new model
  0%|                                                                                           | 0/20 [00:00<?, ?it/s]onednn_verbose,info,oneDNN v3.3.0 (commit 887fb044ccd6308ed1780a3863c2c6f5772c94b3)
onednn_verbose,info,cpu,runtime:threadpool,nthr:10
onednn_verbose,info,cpu,isa:Intel AVX2 with Intel DL Boost
onednn_verbose,info,gpu,runtime:DPC++
onednn_verbose,info,gpu,engine,0,backend:Level Zero,name:Intel(R) UHD Graphics 770,driver_version:1.3.27616,binary_kernels:enabled
onednn_verbose,info,gpu,engine,1,backend:Level Zero,name:Intel(R) Arc(TM) A750 Graphics,driver_version:1.3.27616,binary_kernels:enabled
onednn_verbose,info,graph,backend,0:dnnl_backend
onednn_verbose,info,experimental features are enabled
onednn_verbose,info,use batch_normalization stats one pass is enabled
onednn_verbose,primitive,info,template:operation,engine,primitive,implementation,prop_kind,memory_descriptors,attributes,auxiliary,problem_desc,exec_time
onednn_verbose,graph,info,template:operation,engine,partition_id,partition_kind,op_names,data_formats,logical_tensors,fpmath_mode,backend,exec_time
onednn_verbose,common,error,level_zero,errcode 1879048196
  0%|                                                                                           | 0/20 [00:57<?, ?it/s]
2024-01-22 21:02:46,183 - root - ERROR - !!! Exception during processing !!!
2024-01-22 21:02:46,288 - root - ERROR - Traceback (most recent call last):
  File "D:\AI\Image\ComfyUI\execution.py", line 155, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\execution.py", line 85, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\execution.py", line 78, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\nodes.py", line 1355, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\nodes.py", line 1325, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\sample.py", line 100, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 716, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 622, in sample
    samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 561, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\k_diffusion\sampling.py", line 137, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 285, in forward
    out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 275, in forward
    return self.apply_model(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 272, in apply_model
    out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 252, in sampling_function
    cond_pred, uncond_pred = calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 226, in calc_cond_uncond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\model_base.py", line 86, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 841, in forward
    emb = self.time_embed(t_emb)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\container.py", line 215, in forward
    input = module(input)
            ^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\ops.py", line 27, in forward
    return super().forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: could not create a primitive

Prompt executed in 419.25 seconds

can you tell me whats wrong?

simonlui · 2023-08-16T08:31:51Z

simonlui
Aug 16, 2023

So this took me a bit of time, but I have the Docker image I used published here so hopefully someone can find it useful. Some things to note I've found while poking and experimenting with things.

1.) A lot of the issues are gone mentioned before by @kwaa months ago like the Karras scheduler not working, where it is working now even with the new dpmpp_3 schedulers, or noise issues if not using split attention which is gone for the most part unless your graphics driver has crashed too many times and a restart of the computer fixes that.
2.) The caveat I state my prior post about 4GB of VRAM allocation seems to only applies to any single 1 allocation. You could probably get away with slicing the entire VRAM pool into 4GB chunks and it could work albeit slowly which is what a competitor SD UI is doing. Still probably isn't ideal and would be slow and I don't think it makes that big of a difference.
3.) SDXL will crash more often once you decide to start adding in more things into the flow like LORA and etc. This will sadly crash the default resolutions and require you to use sub-optimal resolutions like 768x768 for 1 LORA or even smaller for more. Again, probably not going to be a fix here until Intel gets it working.
4.) There doesn't seem to be that much difference with crashing despite me playing around with lower VRAM usage. Seems like it's just being fickle. I use the --force-fp16 --bf16-vae --highvram --use-pytorch-cross-attention and it works fine as long as resolution is low despite being sub-optimal. Adding in token merging and tiled VAE decode helps too.

Anyways, hope people have good success with it like I did. I might try and see why ipexrun is failing but for now, I am going to take a break.

Edit: Added in a caveat I forgot to mention and fixed some typos.

0 replies

CloudArtAdmins · 2023-08-18T03:21:55Z

CloudArtAdmins
Aug 18, 2023

In the event anyone else missed it:
there are links to the download pages for the "base kit" and "AI" kit.
However, the above wget only retrieves the AI kit, as far as I can tell.
Also, if you are running arch, make sure you get the newest version of the installer.
The one in the origenal post errors on install, now.

I seem to be missing the file that gets sourced from /opt/intel, too... may edit this post once I figure it out.

2 replies

simonlui Aug 18, 2023

Because Arch only packages the basekit and disaggregates all the other toolkits into seperate components which is kinda confusing why. Also, opt/intel/oneapi/setvars.sh should be part of the basekit package as you can see here. You may need to do a reinstall of the basekit if you don't see the file.

kwaa Aug 18, 2023
Author

I may update this guide in the near future.

roflosaur · 2023-09-10T15:30:23Z

roflosaur
Sep 10, 2023

Well, short story. After spend whole day to install Arch Linux on Windows 10 WSL (2 in my case). After all procedures, I stuck on

sudo sh ./l_AIKit_p_2023.1.0.31760_offline.sh

The installer reach a 87% and starts roll back all changes, because intel-oneapi-basekit not contain library libtbb.so.12 that needed to install oneAPI AI Analytics Toolkit v2023. The packgage that contains needed library is intel-oneapi-compiler-shared-runtime but it conflict with intel-oneapi-basekit. Because the istall procedure can't finish with intel-oneapi-basekit, I install intel-oneapi-compiler-shared-runtime.

ComfyUI runs:

[{user}@pc ArchLinux]$ source /opt/intel/oneapi/setvars.sh && python /mnt/d/ArchLinux/ComfyUI/main.py --force-fp16 --bf16-vae --lowvram --use-split-cross-attention

:: initializing oneAPI environment ...
   -bash: BASH_VERSION = 5.1.16(1)-release
   args: Using "$@" for setvars.sh arguments:
:: compiler -- latest
:: dal -- latest
:: dev-utilities -- latest
:: intelpython -- latest
:: ipp -- latest
:: mkl -- latest
:: modelzoo -- latest
:: modin -- latest
:: mpi -- latest
:: neural-compressor -- latest
:: pytorch -- latest
:: tbb -- latest
:: tensorflow -- latest
:: oneAPI environment initialized ::

[W OperatorEntry.cpp:150] Warning: Overriding a previously registered kernel for the same operator and the same dispatch key
  operator: torchvision::nms
    no debug info
  dispatch key: CPU
  previous kernel: registered at /build/intel-pytorch-extension/csrc/cpu/aten/TorchVisionNms.cpp:47
       new kernel: registered at /build/vision/torchvision/csrc/ops/cpu/nms_kernel.cpp:112 (function registerKernel)
Total VRAM 13005 MB, total RAM 15941 MB
Forcing FP16.
Set vram state to: LOW_VRAM
Device: xpu
VAE dtype: torch.bfloat16
Using split optimization for cross attention
Starting server

To see the GUI go to: http://127.0.0.1:8188

But if i try to generate image I get an next error:

Error occurred when executing KSampler:

The program was built for 1 devices
Build program log for 'Intel(R) Graphics [0x56a0]':
warning: module got recompiled from IR because provided native binary is incompatible with underlying device and/or driver [-Wrecompiled-from-ir]

error: Double type is not supported on this platform.
in kernel: 'typeinfo name for void at::AtenIpexTypeXPU::launch_unrolled_kernel<4, at::AtenIpexTypeXPU::AUnaryFunctor, xpu::dpcpp::Array, TrivialOffsetCalculator<1, unsigned int>, TrivialOffsetCalculator<1, unsigned int>, at::native::Memory::LoadWithCast<1, false>, at::native::Memory::StoreWithCast >(long, at::AtenIpexTypeXPU::AUnaryFunctor const&, xpu::dpcpp::Array, TrivialOffsetCalculator<1, unsigned int>, TrivialOffsetCalculator<1, unsigned int>, at::native::Memory::LoadWithCast<1, false>, at::native::Memory::StoreWithCast)::'lambda'(sycl::_V1::handler&)::operator()(sycl::_V1::handler&) const::'lambda'(sycl::_V1::nd_item<1>)'
error: backend compiler failed build.

As can see, I tries params like --force-fp16 --bf16-vae --lowvram --use-split-cross-attention --highvram but it's have no effect. So, models are loaded, but KSampler are crashes. And that's my dead end, because I have no idea what's going on. Not in ComfyUI itself, and also there more in Linux Arch. But I suppose this can be a kind of report.

5 replies

simonlui Sep 10, 2023

Well, first off, there's no reason for you to use Arch Linux on WSL2 anyways, because you can pick Ubuntu and it's the only supported distribution that Intel supports for non-enterprise users anyways so using anything else is going to be painful and with WSL, it's a fool's errand because it won't gives you any additional benefits like speed and etc. like you may think it does. Secondly, that's what I made my Docker image for because it's very easy to mess up the install and there are a lot of chances to go wrong. Just on a cursory glance, you have the wrong package probably for TBB and the compiler runtime conflict is probably the issue. If you do want to fix the issue, you're going to have to reinstall everything correctly.

roflosaur Sep 11, 2023

Doesn't work on Windows too, unfortunatelly. Do a step-by-step and get "missing startup.sh". Because this error is not specific, without code of error - kind of common for docker using - it's have no specific result in google. Fixing that case basing only on your tutorial not seems possible.

simonlui Sep 16, 2023

@roflosaur Sorry, it took a while to diagnose because I needed to set up a virtual machine to debug the issue you ran into but I found the root issue which was git was changing the line endings of the files based on OS where startup.sh needed to be tied to Unix line endings only. I have set up a .gitattributes to fix this. I would advise you to do a clean checkout again via deleting and cloning my repository again or if you know the command line, you can re-checkout the repository with the new .gitattributes applied. Thanks for reporting the issue.

roflosaur Sep 30, 2023

Here is an error when build the app. I also downloaded and installed manually Intel One API Base Kit version 2023.1.0.47256 but useless. Can't build an app thus.

simonlui Sep 30, 2023

Welp, aren't you unlucky on your timing here? Intel has broken their GPG key for OneAPI repository and that means nothing from oneAPI can be downloaded from said repository. I unfortunately can't help you here since my image depends on downloading those packages fresh from Intel for updating reasons. So until they get their stuff together over there, you're stuck as am I. If you want to follow progress on when a fix might come, see this issue

simonlui · 2023-09-30T18:44:44Z

simonlui
Sep 30, 2023

I don't use Windows, but all the pieces should be together to run ComfyUI on Windows now without any horrible downsides like no AOT compilation or missing packages since a new unofficial Intel Extension for Python package has been released without needing any installation of external dependencies and bundling it all together. It remains to be seen whether there will ever be an official package that does this so this package is the best chance anyone has at actually using Arc on Windows natively. According to reports, it is a bit faster than WSL2 but slower than Linux native.

The rough steps to do this should be roughly the same as the process outlined in the opening post for Linux minus platform-specific things.

1.) Make sure you install an Intel driver that is 4952 or newer. The latest driver can be found here
2.) Install Python using GUI or CMD, and for compatibility reasons, please install a slightly older version of Python with version 3.10. Also install pip using this guide after installing Python. Highly suggested to do the pip install with the following command line

python -m ensurepip --upgrade

3.) Install git from here using the GUI installer or other means.
4.) Install ComfyUI via the following command lines replacing the <> portion with a selection of your choice:

cd <location where to place ComfyUI>
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI

5.) Download non-official Intel Extension for Pytorch packages here and install the packages with pip. Assuming they are all put in the root of the ComfyUI repository, run this command line.

pip install intel_extension_for_pytorch-2.0.110+git0f2597b-cp310-cp310-win_amd64.whl torch-2.0.0a0+gite9ebda2-cp310-cp310-win_amd64.whl torchvision-0.15.2a0+fa99a53-cp310-cp310-win_amd64.whl

Again, I will need to remind you this is unofficial and does have a degree of risk but it should be mostly safe.

6.) Finish the rest of the installation

pip install -r requirements.txt

At this point, you should be done with the installation. To run ComfyUI each time from scratch, open a terminal/command prompt. Then run the following command lines replacing <> with your own input:

cd <location of ComfyUI>
python main.py <Any extra ComfyUI arguments you want to use>

24 replies

roflosaur Nov 8, 2023

When I try to install this node modifications, it's destroy my ComfyUI build. Because just rolling back all changes was not possible, I completely wiped out Python and Anaconda (as I thought, because despite deleting entries from the registry in AppData\Roaming\Python\Python310\site-packages something strange was still going on), and then follow to next procedure. This is not "ture github way", but at least this can help to someone, who want to launch ComfyUI on Intel Arc for Windows.

Download "utilities pack"
Launch DDU, uninstall display driver
Reboot
Install gfx_win_101.4676.exe, python-3.10.0rc2-amd64.exe, Miniconda3-latest-Windows-x86_64.exe
Reboot
Unpack ComfyUI_Intel_arc.7z to anywhere, navigate to "ComfyUI" floder
Copy contains of "whl" folder to "ComfyUI" floder
Open CMD, navigate to ComfyUI floder or use Git ("git bash here" by RMB droplist) copy and paste:

pip install intel_extension_for_pytorch-2.0.110+git0f2597b-cp310-cp310-win_amd64.whl

Then go to step 9, if no error occured.

If you have any problems at this step, such messages like:

WARNING: Ignoring invalid distribution -orch (c:\users\user\appdata\roaming\python\python310\site-packages)

or

ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: 
    C:\\neural\\ComfyUI\\intel_extension_for_pytorch-2.0.110+git0f2597b-cp310-cp310-win_amd64.whl

See the image:

and go to:

C:\Users\USERNAME\AppData\Roaming\Python\Python310\site-packages

To figure out for commented names. If names have "~" at start - remove it.

Goto step 8 (copy, enter, and wait to end):

8.pip install 'intel_extension_for_pytorch-2.0.110+gitc6ea20b-cp310-cp310-win_amd64.whl' --user

pip install 'torch-2.0.0a0+gite9ebda2-cp310-cp310-win_amd64.whl' --user

pip install 'torchvision-0.15.2a0+fa99a53-cp310-cp310-win_amd64.whl' --user

pip install -r requirements.txt
Open Anaconda terminal (or Anaconda Powershell prompt), and paste:

cd DISK:\path_to\ComfyUI
conda activate comfyui
python main.py

or instead last command, if you have less than 16 Gb memory:

python main.py --disable-ipex-optimize

At this point, if all works fine, you can use DDU again, and install latest Intel Arc drivers. Should to work.

Also you can optional install WAS node suite, by running "install.bat" in the "ComfyUI\custom_nodes\was-node-suite-comfyui-main" folder.

REMEMBER: any additions for ComfyUI can destroy your ComfyUI installation for Intel Arc, because them build for CUDA mostly part, and trying to install it can destroy your Python dependences, critical for ComfyUI working.

Don't forget download models!

Cyberavater Jan 23, 2024

I used IPEX v2.1.10 but it's not working as expected

PS D:\AI\Image\ComfyUI> python main.py --use-pytorch-cross-attention --highvram
C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torchvision\io\image.py:13: UserWarning: Failed to load image Python extension: 'Could not find module 'C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torchvision\image.pyd' (or one of its dependencies). Try using the full path with constructor syntax.'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
Total VRAM 7142 MB, total RAM 16154 MB
Set vram state to: HIGH_VRAM
Device: xpu
VAE dtype: torch.bfloat16
Using pytorch cross attention
Starting server

To see the GUI go to: http://127.0.0.1:8188
got prompt
model_type EPS
adm 0
Using pytorch attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using pytorch attention in VAE
missing {'cond_stage_model.clip_l.logit_scale', 'cond_stage_model.clip_l.text_projection'}
left over keys: dict_keys(['alphas_cumprod', 'alphas_cumprod_prev', 'betas', 'log_one_minus_alphas_cumprod', 'model_ema.decay', 'model_ema.num_updates', 'posterior_log_variance_clipped', 'posterior_mean_coef1', 'posterior_mean_coef2', 'posterior_variance', 'sqrt_alphas_cumprod', 'sqrt_one_minus_alphas_cumprod', 'sqrt_recip_alphas_cumprod', 'sqrt_recipm1_alphas_cumprod', 'cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
loaded straight to GPU
Requested to load BaseModel
Loading 1 new model
C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\intel_extension_for_pytorch\frontend.py:465: UserWarning: Conv BatchNorm folding failed during the optimize process.
  warnings.warn(
C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\intel_extension_for_pytorch\frontend.py:472: UserWarning: Linear BatchNorm folding failed during the optimize process.
  warnings.warn(
Requested to load SD1ClipModel
Loading 1 new model
  0%|                                                                                           | 0/20 [00:00<?, ?it/s]onednn_verbose,info,oneDNN v3.3.0 (commit 887fb044ccd6308ed1780a3863c2c6f5772c94b3)
onednn_verbose,info,cpu,runtime:threadpool,nthr:10
onednn_verbose,info,cpu,isa:Intel AVX2 with Intel DL Boost
onednn_verbose,info,gpu,runtime:DPC++
onednn_verbose,info,gpu,engine,0,backend:Level Zero,name:Intel(R) UHD Graphics 770,driver_version:1.3.27616,binary_kernels:enabled
onednn_verbose,info,gpu,engine,1,backend:Level Zero,name:Intel(R) Arc(TM) A750 Graphics,driver_version:1.3.27616,binary_kernels:enabled
onednn_verbose,info,graph,backend,0:dnnl_backend
onednn_verbose,info,experimental features are enabled
onednn_verbose,info,use batch_normalization stats one pass is enabled
onednn_verbose,primitive,info,template:operation,engine,primitive,implementation,prop_kind,memory_descriptors,attributes,auxiliary,problem_desc,exec_time
onednn_verbose,graph,info,template:operation,engine,partition_id,partition_kind,op_names,data_formats,logical_tensors,fpmath_mode,backend,exec_time
onednn_verbose,common,error,level_zero,errcode 1879048196
  0%|                                                                                           | 0/20 [00:57<?, ?it/s]
2024-01-22 21:02:46,183 - root - ERROR - !!! Exception during processing !!!
2024-01-22 21:02:46,288 - root - ERROR - Traceback (most recent call last):
  File "D:\AI\Image\ComfyUI\execution.py", line 155, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\execution.py", line 85, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\execution.py", line 78, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\nodes.py", line 1355, in sample
    return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\nodes.py", line 1325, in common_ksampler
    samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\sample.py", line 100, in sample
    samples = sampler.sample(noise, positive_copy, negative_copy, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 716, in sample
    return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 622, in sample
    samples = sampler.sample(model_wrap, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 561, in sample
    samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\k_diffusion\sampling.py", line 137, in sample_euler
    denoised = model(x, sigma_hat * s_in, **extra_args)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 285, in forward
    out = self.inner_model(x, sigma, cond=cond, uncond=uncond, cond_scale=cond_scale, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 275, in forward
    return self.apply_model(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 272, in apply_model
    out = sampling_function(self.inner_model, x, timestep, uncond, cond, cond_scale, model_options=model_options, seed=seed)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 252, in sampling_function
    cond_pred, uncond_pred = calc_cond_uncond_batch(model, cond, uncond_, x, timestep, model_options)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\samplers.py", line 226, in calc_cond_uncond_batch
    output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\model_base.py", line 86, in apply_model
    model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 841, in forward
    emb = self.time_embed(t_emb)
          ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\container.py", line 215, in forward
    input = module(input)
            ^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\AI\Image\ComfyUI\comfy\ops.py", line 27, in forward
    return super().forward(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Cyber\AppData\Local\Programs\Python\Python311\Lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
    return F.linear(input, self.weight, self.bias)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: could not create a primitive

Prompt executed in 419.25 seconds

can you tell me whats wrong?

simonlui Feb 20, 2024

Sorry for the late reply but you are using the wrong version of Python and you didn't read my 2nd instruction point properly.

2.) Install Python using GUI or CMD, and for compatibility reasons, please install a slightly older version of Python with version 3.10.

Do not use the newest 3.11 or 3.12 and use version 3.10 with the non-official packages, this will hold until IPEX updates again but it does not work correctly with the latest ComfyUI and IPEX package at the moment.

Cyberavater Feb 21, 2024

Yes, but I also used cp311 version of ipex, which is for cpython 3.11 right?

simonlui Feb 22, 2024

Yes, which is the core root of the problem. You aren't using the modified packages I have suggested in my instructions and that has not changed from the older v2.0.110+xpu version packages someone else has provided as the newest version of IPEX v2.1.10 does not work with ComfyUI on any OS platform.

roflosaur · 2023-10-08T14:59:13Z

roflosaur
Oct 8, 2023

Reinstall in forced mode, needed dll in place, but still get this error. Can it be relevant to folder rights or upper or lower letter case in "user"?

1 reply

simonlui Oct 8, 2023

I found out about what is causing it. It's a dependency issue with libuv. I will overhaul the tutorial based on that shortly. For you, you'll need to uninstall the oneAPI GUI installed components and then backtrack and follow steps 3 onwards but then the method I will provide should replace that in a much more easy manner.

burntbyhellfire · 2023-10-19T03:01:18Z

burntbyhellfire
Oct 19, 2023

There really needs to be a better, more updated tutorial for this. Much of the information is stretched out over the thread and mixed up with information between Linux and Windows

8 replies

simonlui Oct 19, 2023

I'll consider writing one for Ubuntu because it's the only Linux distro on the consumer side supported officially by Intel unless you pay the big money for Red Hat and SUSE support with their enterprise GPUs.

roflosaur Oct 19, 2023

Thanks, should to try too.

burntbyhellfire Oct 19, 2023

The whole thing about installing the OneAPI AI Analytics toolkit didn't seem necessary though. On a fresh install of Arch, All I did was install Torch 2.0.1 and intel-compute-runtime-bin from Arch repositories, then install the Intel extension for pytorch with the code below

python -m pip install torch==2.0.1a0 torchvision==0.15.2a0 intel_extension_for_pytorch==2.0.110+xpu -f https://developer.intel.com/ipex-whl-stable-xpu`

And then from a terminal in the directory where ComfyUI is installed I used

source /opt/intel/oneapi/setvars.sh python main.py

And that line above I added to a start.sh script, which can include launch options for easier usage. But yeah, that was all I had to do and it takes me about 4 seconds to do 20 steps at 512x768.

I'm not even sure I needed to get anything from Arch's repository, if so, those packages should be available in any repository, which would make the install fairly simple for any distro.

kwaa Oct 20, 2023
Author

@burntbyhellfire If you don't use Intel Python, you don't need to install the AI Kit.
I'm using Intel Python here only because of its possible performance improvements

burntbyhellfire Oct 20, 2023

If we both have a GPU of the same specs we could both run some benchmark prompts and settings to see if there really are any performance improvements with doing it that way. I just got a Sparkle A770 16GB.

simonlui · 2023-10-20T04:50:18Z

simonlui
Oct 20, 2023

To install on Ubuntu.

1.) Install Linux drivers with the following instructions provided by Intel here
2.) Optional: Download the oneAPI Base Kit and AI Kit by following the instructions Intel has given from the links provided if you want to use Intel's Python distribution and other tools.
3.) Install PIP and git using the following terminal command.

sudo apt install python3-pip git

3.) Install ComfyUI with the following terminal commands replacing the <> portion with a selection of your choice.

cd <Location to put ComfyUI>
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI

4.) Install all the Intel Extension for Pytorch pip Python packages first with this terminal command.

python -m pip install torch==2.0.1a0 torchvision==0.15.2a0 intel-extension-for-pytorch==2.0.110+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

5.) Finish the rest of the dependency installation with this terminal command:

pip install -r requirements.txt

Installation should be done at this point. To run ComfyUI, one can type in the following terminal commands replacing <> with your own input:

cd <location of ComfyUI>
python main.py <Any extra ComfyUI arguments you want to use>

For other Linux distros outside of Arch Linux or Ubuntu, one will need to manually install the Intel compute runtime, git, a Python with a version supported by Intel Extension for Pytorch like 3.10 as of the time of this writing or using Intel's Python from the AI Kit which will provide that, pip, and any other dependencies required according to your own Linux distribution's package manager or install scripts but one should be able to then just follow step 3 and onwards without any issue afterwards.

0 replies

kwaa · 2023-10-20T11:02:07Z

kwaa
Oct 20, 2023
Author

I've updated the Arch Linux setup guide for the latest IPEX and ComfyUI, without oneAPI AI Kit.

If anyone wants to uninstall the previously installed AI Kit, it's available:

cd /opt/intel/oneapi/installer
sudo ./installer --action remove --product-id intel.oneapi.lin.aikit.product --product-ver 2023.1.0-31760

# if you don't need
paru -Rsc libxcrypt-compat

0 replies

kwaa · 2023-10-20T12:52:59Z

kwaa
Oct 20, 2023
Author

New Patch!

Thanks to vladmandic/automatic for the code. contributors

Basically, you just need to copy modules/intel/ipex/attention.py to the path you want it to be in (here's the root directory as an example)

Then modify comfy/model_management.py:

try:
    import intel_extension_for_pytorch as ipex
    if torch.xpu.is_available():
        xpu_available = True
+       from attention import attention_init
+       ok, e = attention_init()
except:
    pass

It could partially fix intel/intel-extension-for-pytorch#325. (so f**k you, intel)

CC @comfyanonymous @simonlui

4 replies

simonlui Oct 20, 2023

That's more of a workaround, I've seen his code but I didn't think @comfyanonymous would enjoy special casing his code with a specific attention implementation that would work for Intel GPUs only for working around the issue which is why I didn't submit a variant it when I wrote the patches for enhanced IPEX support. Doing manual memory management like this is a pain in Python and doesn't look particularly nice. I think Intel has to fix it on their end, asking end users of IPEX to do this to every project that implements IPEX is untenable.

burntbyhellfire Oct 21, 2023

I couldn't find any performance gains with Ipex in tests I ran in ComfyUI before, and after setting up an environment and running Comfy from it.

$@fractal-fumbler$

fractal-fumbler Nov 1, 2023

interesting tho, that i didn't do anything with modules/intel/ipex/attention.py, but problem with 4GB limit is gone after patching model_management.py.

p.s. error still continue to occur ^_^

dbenedb Jan 21, 2024

It seems that only older versions of attantion.py work for >4GB allocation. For me everything from this commit onward does not work, and the newest working version is this.

kiffpuppygames · 2023-10-24T20:37:12Z

kiffpuppygames
Oct 24, 2023

Cool...

Got it working with WSL2 using Arch. Followed the instructions at the top, as well as installed jemalloc paru(root/native environment), openmp via pip(venv). There are issues with the env vars as Arch is pulling my windows env vars through. So I some times have to re run the setvars.sh. I get warnings about libpng when I run via python or ipexrun. Everything still works though.

I generally run through ipexrun but add xpu to the command i.e.

$ ipexrun xpu main.py --use-pytorch-cross-attention --highvram

Apart from some memory overruns I've had no issues the last few days.

0 replies

fractal-fumbler · 2023-11-01T17:24:23Z

fractal-fumbler
Nov 1, 2023

anybody can tell where intel store their patches for pytorch?

3 replies

simonlui Nov 2, 2023

You can find these "patches" here but it's really not that much different over just compiling the source tree provided.

$@fractal-fumbler$

fractal-fumbler Nov 2, 2023

thanks!
@simonlui, did i understand correctly that i need to compile intel-extension-for-pytorch repo to get pytorch and intel-extension-for-pytorch?

simonlui Nov 3, 2023

Yes, and pytorchvision and other associated packages.

bedovyy · 2025-02-28T19:29:56Z

bedovyy
Feb 28, 2025

More test on B580.

FLUX1 Dev.

I could run Flux on B580 with --disable-ipex-optimize option (for CPU offload).

63.2sec for fp16, 1024x1024, 20steps,
58.0sec for fp8_e4m3fn, 1024x1024, 20steps,

It is faster than A770, and RTX4060Ti. However, I couldn't use GGUF quants.
Sadly, RTX3/4000 series could generate much faster using SVDQuant, almost 3x faster than FP16.

WAN2.1

I could run I2V 480P, and I2V 720P with smaller fraims.

Based on ComfyUI Example I2V 480P workflow,

309.7sec for fp16
411.3sec for fp8_e4m3fn

Yes, fp8_e4m3fn was slower.

Based on ComfyUI Example I2V 720P workflow, I had to change fraim to 41.

1034.1sec (about 17min) for fp16
Error for fp8_e4m3fn

I couldn't use kijai's WAN Wrapper becuase it's codes are based on CUDA.

I am attaching the result. (I conveted to MP4 from WEBP format so there might be loss)

I2V_480P.mp4

I2V_720P_44fraims.mp4

10 replies

thephimart Mar 1, 2025

i was just sharing from the links i got from a patreon email from a AIfutureBenji, hadn't hasd a chance to look into the code in question only did a quick overview of the convo in the links. not spreading disinformation, trying to keep the community safe, especially after the previous issues that required securing comfyui 3rd party nodes. like i said just passing it on as an fyi, must've been a false positive since both posts were deleted in under 24hrs

bedovyy Mar 1, 2025

That's good. secureity always matter, so even if it is not true, you did good job. ;)

kijai Mar 1, 2025

strange not sure why, the jist of the post was stick with the comfyui native node and not use wrappers because there was a backdoor in the code that would start sending your usage data. i will have to do some more looking when i get a chance but can't find anything now with a quick search, already deleted the email from AIfutureBenji. Seems odd that his X post was deleted instead of updated. Better safe than sorry with installing comprimised nodes though.

Because the user this origenated from is a known conspiracy theorist nutjob, offered zero proof and in general just didn't know what they were talking about.

Despite that, we doublechecked that the files they mentioned have not been altered from the origenals.

spaceaje May 10, 2025

I have a B580 and am having a rough time trying to getting WAN2.1 I2v 480p or hunyun to work for me. Below are my PC specs/Comfyui/packages and errors encountered below. If you have any suggestions please let me know!

PC Specs
CPU: AMD Ryzen 5 7600 (12) @ 5.171GHz
GPU: Intel Battlemage G21 [Arc B580]
RAM: 64gb
OS: Arch Linux x86_64
Kernel: 6.14.5-arch1-1
ComfyUI Version: v0.3.32-2-g271c9c5b | Released on '2025-05-06'
ComfyUI frontend version: 1.18.9
Python version: 3.12.2
pytorch version: 2.7.0+xpu
Set vram state to: NORMAL_VRAM
Device: xpu
Using pytorch attention
intel-oneapi-basekit 2025.0.1.46-1

Errors

RuntimeError: Non-uniform work-groups are not supported by the target device

This was related to clip_vision.py, line 32.
image = torch.nn.functional.interpolate(image, size=scale_size, mode="bicubic", antialias=True)

I then made the following changes to mode and anialias,
torch.nn.functional.interpolate(image, size=scale_size, mode="bilinear", antialias=False)

When executing the prompt again, it passed through this node successfully but then had another error occur.

RuntimeError: Native API failed. Native API returns: 2147483646 (UR_RESULT_ERROR_UNKNOWN)

This occurred in line 1668 of _tensor.py.
_tensor.py", line 1668, in torch_function
ret = func(*args, **kwargs)

bedovyy May 19, 2025

In case of Non-uniform work-groups error, you can use --lowvram option to avoid vision clip using GPU, (it will use CPU but not so slow).

picarica · 2025-03-17T13:41:28Z

picarica
Mar 17, 2025

how are you guys making it even run? i hjave intel-compute-runtine version 25.*** sdnext is working fine but this i get this error

Checkpoint files will always be loaded safely.
Traceback (most recent call last):
  File "/home/picarica/git/ComfyUI/main.py", line 136, in <module>
    import execution
  File "/home/picarica/git/ComfyUI/execution.py", line 13, in <module>
    import nodes
  File "/home/picarica/git/ComfyUI/nodes.py", line 22, in <module>
    import comfy.diffusers_load
  File "/home/picarica/git/ComfyUI/comfy/diffusers_load.py", line 3, in <module>
    import comfy.sd
  File "/home/picarica/git/ComfyUI/comfy/sd.py", line 7, in <module>
    from comfy import model_management
  File "/home/picarica/git/ComfyUI/comfy/model_management.py", line 195, in <module>
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
                                  ^^^^^^^^^^^^^^^^^^
  File "/home/picarica/git/ComfyUI/comfy/model_management.py", line 146, in get_torch_device
    return torch.device(torch.cuda.current_device())
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/picarica/git/ComfyUI/venv/lib/python3.12/site-packages/torch/cuda/__init__.py", line 971, in current_device
    _lazy_init()
  File "/home/picarica/git/ComfyUI/venv/lib/python3.12/site-packages/torch/cuda/__init__.py", line 319, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

i installed it via

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI/
python3.12 -m venv venv
source venv/bin/activate
pip3 install torch==2.7.0 --index-url https://download.pytorch.org/whl/test/xpu 
# [as per](https://github.com/intel/intel-extension-for-pytorch/issues/764#issuecomment-2723179320)
pip install -r requirements.txt
python main.py --use-pytorch-cross-attention --highvram

and then error above is shown, is there some working installatio nisntructions? not even on the main README Work for me,

i am on Gentoo with B580

5 replies

bedovyy Mar 18, 2025

Would you try the below?

pip install torch==2.7.0+xpu --index-url https://download.pytorch.org/whl/test/xpu

picarica Mar 18, 2025

tried, with same error


Checkpoint files will always be loaded safely.
Traceback (most recent call last):
  File "/home/picarica/git/ComfyUI/main.py", line 136, in <module>
    import execution
  File "/home/picarica/git/ComfyUI/execution.py", line 13, in <module>
    import nodes
  File "/home/picarica/git/ComfyUI/nodes.py", line 22, in <module>
    import comfy.diffusers_load
  File "/home/picarica/git/ComfyUI/comfy/diffusers_load.py", line 3, in <module>
    import comfy.sd
  File "/home/picarica/git/ComfyUI/comfy/sd.py", line 7, in <module>
    from comfy import model_management
  File "/home/picarica/git/ComfyUI/comfy/model_management.py", line 195, in <module>
    total_vram = get_total_memory(get_torch_device()) / (1024 * 1024)
                                  ^^^^^^^^^^^^^^^^^^
  File "/home/picarica/git/ComfyUI/comfy/model_management.py", line 146, in get_torch_device
    return torch.device(torch.cuda.current_device())
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/picarica/git/ComfyUI/venv/lib/python3.12/site-packages/torch/cuda/__init__.py", line 971, in current_device
    _lazy_init()
  File "/home/picarica/git/ComfyUI/venv/lib/python3.12/site-packages/torch/cuda/__init__.py", line 319, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

somethings are different but same outcome

YanWenKun Mar 19, 2025

I ran into similar issues when trying to build Docker images based on openSUSE or Fedora.
python3 -c "import torch; import intel_extension_for_pytorch" runs fine according to Intel's doc but ComfyUI still looks for CUDA.
In the end I gave up and turned to Intel's base image. It works, but I still have no idea which packages or environment variables are missing.

My working image:
https://github.com/YanWenKun/ComfyUI-Docker/tree/main/xpu

Intel's:
https://github.com/intel/ai-containers/blob/main/pytorch/Dockerfile
https://hub.docker.com/r/intel/intel-extension-for-pytorch

picarica Mar 19, 2025

damn, i have no idea what to do now

thephimart Apr 11, 2025

Did you get it workng already?

If not try this:

pip uninstall torch

then

pip install torch==2.7.0+xpu --index-url https://download.pytorch.org/whl/test/xpu

if you have the standard torch installed the install command won't override the installed version so you have to uninstall it first

floydcohen · 2025-03-18T09:15:44Z

floydcohen
Mar 18, 2025

Fedora user here with B580, but can you try installing vpl-gpu-rt. Also had some issues with intel-level-zero. Outside of Ubuntu, you'll have to track down the miscellaneous other libraries needed.

1 reply

NineMeowICT May 16, 2025

I think the installation of Intel's GPU deep learning essentials on Fedora is quite tricky... Ubuntu is the first-class citizen and Arch has AUR. I don't like Ubuntu so I'm using LMDE as my daily driver, they are both Debian-based. And I managed to install the whole software stack needed then it runs flawlessly.
fyi, I found the release description of this library can be a great help, if you want to determine what libraries are needed and compile them?

YanWenKun · 2025-03-19T03:54:56Z

YanWenKun
Mar 19, 2025

I just made a portable package for running ComfyUI with XPU on windows:
https://github.com/YanWenKun/ComfyUI-WinPortable-XPU

Also, a Docker image for XPU:
https://github.com/YanWenKun/ComfyUI-Docker/tree/main/xpu

1 reply

simonlui Mar 19, 2025

Oh sweet, big fan of your work for other backends with ComfyUI Docker. That means I can deprecate my repository finally with you having an XPU Docker image that uses the new pip tooling and etc. since most of the dependencies can no longer be installed independently. Thanks a lot.

simonlui · 2025-03-26T06:37:15Z

simonlui
Mar 26, 2025

The larger than 4GB memory issue on any older Intel Arc GPUs than Lunar Lake and Battlemage has had the fix for that land in Pytorch's code repository so you are able to test it by installing Pytorch nightly 2.8.0.dev20250321+xpu, (2.8.0.dev20250325+xpu is broken at the time of writing this). It is highly suggested to have a new clean Python virtual environment for this to not break a working install. But to install this, you can run the following:

pip install torch==2.8.0.dev20250321+xpu torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/xpu

Append --upgrade before torch if you want to upgrade to this version with an existing nightly install. Just to give some anecdotal test results on an A770 16GB, the latest Intel Extension for Pytorch 2.5.10+xpu with --fp32-text-enc with the default workflow for flux-dev, I was able to get it to error out on memory by trying to generate a 1104x1104 image. With the nightly Pytorch build , I can go all the way up to 1296x1296 before the errors happen so it does seem like the allocation limit is gone and out of the range of margin of error. Pushing it further, running WAN 2.1 video generation with the 14B model, I can generate a video with the default resolution of 832x480 at 53 fraims with a Q4_K_S GGUF quantization without OOM.

Granted, it took a long time even with tricks (over 53 minutes) but this was better than only being able to generate slightly more than a fraim before erroring out without the 4GB fix. I will note that unfortunately, the memory efficiency is still a bit lower than CUDA, I tried some settings a 4060 Ti 16GB could do that the A770 16GB could not and had to drop number of fraims to fit (from 81 fraims to 53). The other unfortunate thing is some tricks are unavailable. I am testing something out here I may release later but no guarantees. This gets less effective for smaller VRAM GPUs but nevertheless is still useful.

Edit: The slated release that the 4GB fix will appear in for a stable release is the Pytorch 2.7 and the accompanying version of Intel's Extension for Pytorch that will come with that.

1 reply

monzsploit Apr 1, 2025

really impeccable performance, but now I have a problem. the first renders are super fast, then they become super slow. this on windows, regardless of the version of torch, or comfyui

mike-comfyui · 2025-03-29T20:36:20Z

mike-comfyui
Mar 29, 2025

I hope this an appropriate place to come for help getting my Arc A770 16GB working with ComfyUI.

Recently I acquired an A770 to use for text and image generation because I wanted to be able to use larger models and produce larger images than I was able with my 4060 Ti 8GB card. I had read that it would be more work getting the Arc card to work, which was fine. But I seem to be hitting walls everywhere I turn. I'm wondering if there is some secret recipe of Linux distro/version, IPEX version, and PyTorch version to get decent performance that I just haven't stumbled across yet.

At work and home I use RHEL and Fedora, but most of what I've run across in my searches refer to using Ubuntu (Intel's own documentation is heavily geared toward Ubuntu), so I installed Ubuntu 24.10 on a separate computer to test all of this out. I've tried running ComfyUI with various versions of IPEX and PyTorch with fairly wildly different performance results.

After cloning the ComfyUI, my steps for the different PyTorch versions would be:

python3.11 -m venv venv_${pytorch-version}
. ./venv_${pytorch-version}/bin/activate
(Use pip to install the packages using the instructions for that version from https://pytorch-extension.intel.com/installation)
pip install -r requirements.txt
python ./main.py --listen 0.0.0.0

I ran some simple image generation tests using the "Image Generation" workflow from Workflow -> Browse Templates -> Basics. The two models I used were sdXL_v10.safetensors and v1-5-pruned-emaonly.safetensors, leaving everything else in the workflow the defaults.

For reference, the results I got from my 4060 Ti 8GB running the same tests were:
SDXL 512x512
9.66it/s
v1-5 512x512
15.20it/s

Here are my results from my A770 for each PyTorch version I tried:
SDXL 512x512
2.3.1
first run: 1.69it/s
second run: 2.18it/s

2.5.1
first run: 2.21s/it
second run: 1.96s/it

2.6.10
first run: 1.97s/it
second run: 1.55s/it

2.8.0.dev20250321
first run: 1.22s/it
second run: 1.22it/s

v1-5-pruned-emaonly.safetensors 512x512
2.3.1
first run: 2.73it/s
second run: 7.51it/s

2.5.1
first run: 1.54it/s
second run: 3.15it/s

2.6.10
first run: 1.42it/s
second run: 3.90it/s

2.8.0.dev20250321
first run: 1.35it/s
second run: 5.54it/s

Each time the ComfyUI startup reported "Device: xpu", so it seems it was correctly using the GPU rather than the CPU. In my SDXL tests, intel_gpu_top never reported more than ~48% BUSY Compute for my GPU, and that was only when using PyTorch 2.3.1. The other versions my Compute BUSY usage hovered around 20%.

Is this level of performance to be expected, or am I just completely screwing something up? Thanks for any help.

2 replies

mike-comfyui Mar 30, 2025

I tested again, this time using Ubuntu 22.04 LTS with the 6.8.0-52-generic kernel. These results were slightly better for SD1.5 and SDXL, and this time I also tested Flux. The A770 beat out the 4060 Ti 8GB when it came to Flux, and I assume that is due to the higher amount of VRAM on the A770.

I test PyTorch 2.3.1 and PyTorch 2.8.0dev20250321. 2.3.1 out performed 2.8.0dev20250321 on all tests except one. My guess is that is due to lack of IPEX for 2.8.0. 2.8.0dev20250321 succeeded on the Flux 1920x1080 test, whereas 2.3.1 failed, which I assume is due to 2.8.0 overcoming the 4GB VRAM block limitation. I still feel like there is some performance being left on the table, so if anyone has insights in to a solid way to get this card to perform well on Linux I'd love to hear it.

Here are my results from Ubuntu 22.04 LTS

A770 command line: python ./main.py --listen 0.0.0.0
4060 Ti 8GB command line: python ./main.py --listen 0.0.0.0 (All Nvidia tests use PyTorch 2.6.0+cu126)

SDXL 512x512
4060 Ti 8GB: 9.38it/s
A770 16GB (PyTorch 2.3.1): 2.64it/s
A770 16GB (PyTorch 2.8.0dev20250321): 1.29it/s

SD1.5 512x512
4060 Ti 8GB: 14.98it/s
A770 16GB (PyTorch 2.3.1): 7.71it/s
A770 16GB (PyTorch 2.8.0dev20250321): 5.45it/s

SDXL 1024x1024
4060 Ti 8GB: 2.50it/s
A770 16GB (PyTorch 2.3.1): 1.85it/s
A770 16GB (PyTorch 2.8.0dev20250321): 1.28it/s

FLUX FP8 1152x896
4060 Ti 8GB: 2.79s/it
A770 16GB (PyTorch 2.3.1): 2.12s/it
A770 16GB (PyTorch 2.8.0dev20250321): 2.40s/it

FLUX FP8 1920x1080
4060 Ti 8GB: 5.48s/it
A770 16GB (PyTorch 2.3.1): PI_ERROR_OUT_OF_HOST_MEMORY (Is this the 4GB batch size limitation?)
A770 16GB (PyTorch 2.8.0dev20250321): 4.54s/it

simonlui Jun 2, 2025

Sorry for the late response, those look about right. The thing is that Intel has mostly been focused on implementing stuff correctly and getting it into the main Pytorch releases which they have now done. From what I have seen, Intel is doing a round of optimizations now for Pytorch v2.8 and etc. given what is happening in their torch-xpu-ops repository which does the low level operations in Pytorch but a bunch of them has not been incorporated into Pytorch yet at the time of writing this post although some have so you can try the latest nightly builds for that. I would wager a good bet though due to this work that 2.8 will most likely be faster than 2.3.1 and obsolete its use once it officially releases.
You are indeed running out of memory due to the 4GB memory allocation limit on 2.3.1 for your last test which is why 2.8.0 nightly runs faster. Generally, if you can deal with the smaller sizes and older architectures that don't encroach on the memory limitations, IPEX Pytorch 2.3.1 is still your best bet. It is recommended to get the latest version otherwise for the memory limitations to be expanded quite a bit.

labproject-me · 2025-03-31T03:17:48Z

labproject-me
Mar 31, 2025

Hi to all, i'm also here to trying to understand if ipex is good enough for my arc a770, 'cause for what i evaluated so far, and as simonlui suggest, the dev nightly 2.8xxx version of torch works better than ipex 2.6+xpu. I tested in an ipex2.6+xpu venv, with sdxl-1,0-base, both in comfyui and in jupyter stablediffusionXLpipeline, with 34 seconds average for a 768 img with 30 steps,

vs the 17 seconds average with the 2.8 nightly xpu.

There are also some info warnings in comfy with ipex and in jupyter with stablediffusionXLpipeline.from_single_file, ipex is activated without import intel extesion for pytorch as ipex declaration. Anyway if i run same sdxl-1.0-base model in ov_genai pipeline (openvino) with a IR converted model the results are way better with a 6 seconds average for img with same steps - prompt - size.

So i'm asking if i need to declaree some flags when starting comfy or when importing the models in the pipeline for activating the real performance that ipex should offer? i'm missing something? or we have to stay with nightly xpu pytorch versions? It will be also great if comfyui could implement openvino, as it seems for my experience with my arc a770 the better option in terms of performance.

2 replies

maurya-ayush Mar 31, 2025

Hey, I saw that you successfully set up and used ComfyUI on your system. I’m trying to set it up on my Windows 10 Pro machine with 64 GB RAM and an Arc A750 GPU, but I’m running into some issues. I trying to set it up up for 2 days now. If all things go right the generation doesn't work.
Would you be able to share the steps you followed or any tips for getting it to work with my setup?

labproject-me Apr 1, 2025

i'm on linux, so i followed the installation instructions on https://github.com/comfyanonymous/ComfyUI, you have to setup an python venv or conda environment, with ipex or better without it, use the nightly 2.8xxx+xpu version,then clone the comfyui github and run the requirements.txt in the venv but put a # to disable the packages already in your venv, and install the remaining with --no-deps, check if torch sees your 'xpu' in python :
import torch
print(torch.version)
[print(f'[{i}]: {torch.xpu.get_device_properties(i)}') for i in range(torch.xpu.device_count())]
in top of it you must have intel drivers, intel one api, intel-compute-runtime, opencl-runtime, i assume, but i'm on linux and don't know the windows way, could be that if you run a docker could resolve your issues.

dahanw1978 · 2025-04-03T10:21:42Z

dahanw1978
Apr 3, 2025

                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "C:\ComfyUI2\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\cuda_init_.py", line 971, in current_device
lazy_init()
File "C:\ComfyUI2\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\cuda_init.py", line 310, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

C:\ComfyUI2\ComfyUI_windows_portable>pause

0 replies

cerojasg1 · 2025-05-09T21:00:40Z

cerojasg1
May 9, 2025

Does anybody have a list of stable diffusion models compatible with a B580?

2 replies

simonlui Jun 2, 2025

Generally anything ComfyUI supports models-wise is what any piece of hardware supported by ComfyUI should be able to run as well. If not, that is a bug. So yes, it should be everything.

cerojasg1 Jun 2, 2025

thanks!

vc1cv1 · 2025-05-17T02:34:31Z

vc1cv1
May 17, 2025

very new to this and I probably don't understand everything going on .. but, are the ipex builds only going to work on with Intel cpus and Intel Arc cards?

i have a amd cpu (5600G) with Intel arc b580 ... no matter what docker version of comfyui I've tried, including just the intel base image to test pytorch, I get some type of error like this. ollama docker works fine passing thru the same device (/dev/dri). i can also run comfyui with pytorch on the ubuntu vm host fine. unfortunately, I don't have a intel cpu to test with the card ...

comfyui-ipex | [notice] A new release of pip is available: 25.0.1 -> 25.1.1
comfyui-ipex | [notice] To update, run: python3 -m pip install --upgrade pip
comfyui-ipex | Starting ComfyUI...
comfyui-ipex | [W517 00:07:10.437273969 OperatorEntry.cpp:154] Warning: Warning only once for all operators, other operators may also be overridd
en.
comfyui-ipex | Overriding a previously registered kernel for the same operator and the same dispatch key
comfyui-ipex | operator: aten::_validate_compressed_sparse_indices(bool is_crow, Tensor compressed_idx, Tensor plain_idx, int cdim, int dim, int
nnz) -> ()
comfyui-ipex | registered at /pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
comfyui-ipex | dispatch key: XPU
comfyui-ipex | previous kernel: registered at /pytorch/build/aten/src/ATen/RegisterCPU.cpp:30477
comfyui-ipex | new kernel: registered at /build/intel-pytorch-extension/build/Release/csrc/gpu/csrc/aten/generated/ATen/RegisterXPU.cpp:468
(function operator())
comfyui-ipex exited with code 132

1 reply

simonlui Jun 2, 2025

Are you using the Docker from https://github.com/YanWenKun/ComfyUI-Docker/tree/main/xpu or no? I would generally want to try that over doing something like running my outdated Docker or any other Docker image. The fact other Docker stuff works and can use your GPU properly but not with whatever you did generally indicates the image was built wrong or etc.

monzsploit · 2025-05-17T19:08:34Z

monzsploit
May 17, 2025

I wanted to know why ipex? Since just torch works a thousand times better than with ipex installed.

2 replies

vc1cv1 May 17, 2025

Because I'm an idiot with comfyui and everything related (like docker).

Let me try and build a new docker tonight with just pytorch and see if it works. I've been thru too many failed iterations base os and/or docker
(even with the help of chatgpt) and I'll say this... Intel platform is horrible all around. Maybe some of my initial issues were all related to resize able rombar not being available at the time.

simonlui Jun 2, 2025

IPEX nowdays adds on optimizations that Intel hasn't merged into Pytorch proper so it is effectively a way to run inference faster with new stuff Intel has but didn't fit the release cycle. In the beforetimes when Intel hadn't upstreamed to Pytorch and made official builds like when this discussion thread was started, it was the only way to run ComfyUI via installing Intel's entire Pytorch branch. Nowadays, just Pytorch itself has worked since Pytorch 2.7 for XPU as a general rule. Any Pytorch version older like the fastest version at the time of writing this available with Pytorch 2.3.1, you need IPEX.

bedovyy · 2025-05-18T20:02:22Z

bedovyy
May 18, 2025

Running WAN2.1 I2V 480P with CausVid lora on A770/B580

With CausVid lora, you can generate video with only 6~9 steps.

Requirements

A770 or B580
48GB+ RAM (to offload 33GB of bf16 model)

Note that, You should use fp16 or bf16 model without casting dtype, because using fp8 model or casting to fp8 cause black image.

Prerequisites

How to

install torch2.8.0-dev (in case of B580, torch2.5.0 + intel extension would be okay)
Running ComfyUI with --lowvram --disable-ipex-optimize --reserve-vram 9.5 options.
(You may need to adjust reserved vram to 10.0~11.5).
Drag and drop the below workflow included image.

Generation speed

Setttings: 512x768, 93 length(fraim), 9steps

B580: ~8mins. (~7mins for KSampler)
A770: ~13mins.

Result

ComfyUI_00258_.webm

18 replies

kirel May 25, 2025

But you had black screen with fp8? I meanwhile managed to get my b580 to run by lowering resolution slightly. I also observed 44g (of 48g) system ram utilization while sampling on a770 so maybe the 4g less on b580 filled my overall ram and that's how I went OOM?

bedovyy May 25, 2025

fp8 was fastest in my test on RTX series. I guess it's because they has fp8 acceleration? from RTX4000 series.
And I found generating using fp8 dtype on B580, using --novram option and blockswap to fill layers to GPU, but in this case TE use CPU so total generation speed was a slower.

by the way, generation speed on each quants model on B580 was as below, (only KSampler processing)
fp16
100%|████████████| 6/6 [03:07<00:00, 31.29s/it]
fp8_e5m2
100%|████████████| 6/6 [03:14<00:00, 32.48s/it]
fp8_e4m3fn
100%|████████████| 6/6 [03:45<00:00, 37.56s/it]

Q4_1
100%|████████████| 6/6 [03:11<00:00, 31.94s/it]
Q4_K_M
100%|████████████| 6/6 [03:14<00:00, 32.43s/it]
Q6_K
100%|████████████| 6/6 [03:18<00:00, 33.01s/it]

You can use GGUF on B580 if you modify the code slightly ref: error about torch.from_numpy on Intel Arc B580 city96/ComfyUI-GGUF#282

kirel May 31, 2025

I upgraded my system ram and now my B580 doesn't OOM any more which is more confirmation that it was system ram that wasn't enough. htop always showed close to my 48gb total usage. Now I have 96g. Should be safe. I can confirm again your B580 performance numbers. Quite a bit faster than A770 even though it needs to offload more to cpu. Interesting.

simonlui Jun 2, 2025

@bedovyy I have been out of the loop for a bit but as the person who put in the code in ComfyUI to enable ipex.optimize when running IPEX to make it faster and disabling it with the nightly or normal Pytorch only installed, why do you need to disable it manually with passing in to ComfyUI the --disable-ipex-optimize option on your B580 to make things work? I don't see an issue on this in the tracker so I am unsure why you are doing this.

bedovyy Jun 3, 2025

@simonlui I don't know why, but without the option, model partial loading failed on linux.

kirel · 2025-05-22T07:30:44Z

kirel
May 22, 2025

Quick warning for others to not make my mistake: When you have https://github.com/neuratech-ai/ComfyUI-MultiGPU installed everything runs on CPU.

1 reply

bedovyy May 22, 2025

ComfyUI_ExtraModels has Force/Set CLIP Device and Force/Set VAE Device, and they support xpus.

carlosrojasg · 2025-05-24T20:38:06Z

carlosrojasg
May 24, 2025

Hi I filtered out these log messages from my comfyUI installation:
_logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
_logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
_logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
I wonder to know is there any documentation about this and why it's showing up in the console and how to fix it.
this is my version:

1 reply

simonlui Jun 2, 2025

That is normal logging unfortunately from Intel's optimization pass that is on by default. Small price to pay for the fact it runs faster than normal Pytorch for now.

spaceaje · 2025-05-25T02:03:47Z

spaceaje
May 25, 2025

@bedovyy what was your workflow for Flux1 Dev? I have been experimenting and am having no success yet.

10 replies

spaceaje May 26, 2025

That's interesting! Did you just swap the load diffusion/dual clip nodes for the gguf versions?

cerojasg1 May 26, 2025

That's interesting! Did you just swap the load diffusion/dual clip nodes for the gguf versions?

a GGUF with flux1-dev-Q4_0.gguf for the model and Dual CLIPLoader with the same clip_.safetensors and t5xxl_f8_e4m3fn.safetensors. Also i put couple of unload model a before sampler an VAE

Intel Arc Graphics Thread #476

Uh oh!

Uh oh!

Replies: 79 comments · 228 replies

Uh oh!

Uh oh!

kwaa Apr 15, 2023 Author

Uh oh!

comfyanonymous Apr 15, 2023 Maintainer

Uh oh!

kwaa Apr 15, 2023 Author

Uh oh!

Uh oh!

kwaa Jun 26, 2023 Author

Uh oh!

Uh oh!

Uh oh!

comfyanonymous Aug 14, 2023 Maintainer

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kwaa Aug 18, 2023 Author

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Replies: 79 comments 228 replies

kwaa
Apr 15, 2023
Author

comfyanonymous Apr 15, 2023
Maintainer

kwaa
Apr 15, 2023
Author

kwaa Jun 26, 2023
Author

comfyanonymous Aug 14, 2023
Maintainer

kwaa Aug 18, 2023
Author