3

I have encountered a particular problem while executing a function from the transformers library of huggingface on an Intel GPU wheel of torch. Since I am doing something I normally shouldn't be doing as Intel GPU's have minimal support for transformers it seems. I ran some very basic diagnostics and found the incompatibility that results in:

Traceback (most recent call last):
  File "/home/cemc/Desktop/python.py.tar/Python Stuff/Group Policy Gradient and Relative Policy Optimization.py", line 22, in <module>
    sequences = model.generate(**inputs, max_new_tokens=32)
  File "/home/cemc/.local/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 120, in decorate_context
    return func(*args, **kwargs)
  File "/home/cemc/.local/lib/python3.10/site-packages/transformers/generation/utils.py", line 2564, in generate
    result = decoding_method(
  File "/home/cemc/.local/lib/python3.10/site-packages/transformers/generation/utils.py", line 2784, in _sample
    outputs = self(**model_inputs, return_dict=True)
  File "/home/cemc/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/cemc/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/cemc/.local/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 1068, in forward
    transformer_outputs = self.transformer(
  File "/home/cemc/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1775, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/cemc/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1786, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/cemc/.local/lib/python3.10/site-packages/transformers/models/gpt2/modeling_gpt2.py", line 873, in forward
    causal_mask = create_causal_mask(
  File "/home/cemc/.local/lib/python3.10/site-packages/transformers/masking_utils.py", line 788, in create_causal_mask
    early_exit, attention_mask, packed_sequence_mask, kv_length, kv_offset = _preprocess_mask_arguments(
  File "/home/cemc/.local/lib/python3.10/site-packages/transformers/masking_utils.py", line 723, in _preprocess_mask_arguments
    attention_mask = attention_mask.to(device=cache_position.device, dtype=torch.bool)
RuntimeError: UR error: 45 (UR_RESULT_ERROR_INVALID_ARGUMENT)

Was in fact not caused by data type incompatibility. So I thought it was because the

attention_mask = attention_mask.to(device=cache_position.device, dtype=torch.bool)

method is attempting to order my GPU to do something it can not do. Which is why I wanted to log what this function is asking my GPU to do so I can hand patch or alter what its doing. I do not know where to look. I asked ChatGPT for a solution to the exact problem in the title since I didn't want to bother real people with real jobs with this minor issue, it gave methods that do not work. I also asked some senior devs where I am interning. They told me there is a way to log CPU/GPU usage but that they didn't know how to log operations. I am running an intel iRISx^e and my operating system is Ubuntu. I am executing the code in terminal.

I checked if the error in question came from a datatype incompatibility by running:


device = torch.device("xpu:0")
dtypes = [torch.float32, torch.float16, torch.bfloat16, torch.int32, torch.bool]
for dt in dtypes:

    try:
        t = torch.zeros(1, dtype=dt, device=device)
        print(f"{dt} -> supported")
    except Exception as e:
        print(f"{dt} -> not supported ({e})")

This returned

torch.float32 -> supported
torch.float16 -> supported
torch.bfloat16 -> supported
torch.int32 -> supported 
torch.bool -> supported

I concluded no datatype incompatibility existed and the incompatibility in question was from operations ordered by the function.

Improve this question
6
  • 2
    "transformers" is not something any GPU is aware of; there's not one GPU architecture that supports or not supports it. What might or might not be supported are some specific mathematical operations, but pytorch will simply not be able to compile the necessary GPU code if you ask it to do something that's impossible (but that's also not how you write torch code, so); so this is a dead end. Commented Oct 17 at 11:21
  • 2
    "is attempting to order my GPU to do something it can not do", no this is going wrong in software running on your CPU, not on the GPU. Commented Oct 17 at 11:23
  • Oh. Where do I need to look to see what is going wrong? And seeing as torch can't do what I want it to, what would I need to do? Commented Oct 17 at 11:25
  • I am precisely wanting to log which mathematical operations are attempting to be done on my GPU by particular libraries, hopefully by not having to dig through source code and where exactly in that chain I get RuntimeError: UR error: 45 (UR_RESULT_ERROR_INVALID_ARGUMENT). Since doing the source code thing would be torturous. Commented Oct 17 at 12:10
  • 2
    but there's nothing to be logged here – 1. pytorch code sets up an operational graph, that then 2. gets compiled to things that can run on a GPU, and then 3. these things get scheduled in which order and iterativeness to run on the GPU and finally, 4. the code is uploaded to the GPU and 5. executed. The libraries help defining that graph, and thus happen before 1., and logging can only happen in or after 2. Commented Oct 17 at 12:15
Related questions
13112
What does the "yield" keyword do in Python?
3982
How do I get the current time in Python?
3956
How can I catch multiple exceptions in one line? (in the "except" block)
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.