How do I debug a segmentation fault in Python when loading a BERT model?

Question

I am attempting to test out this KeyBERT model. I am new to Python (much experience in R). I'm not sure how to debug this issue since the error message is so short. My question is, how do I debug this with such little information? Where to start?

I have a fresh install of Anaconda with Python 3.9.13:

Python 3.9.13 (main, Aug 25 2022, 18:29:29) [Clang 12.0.0 ] :: Anaconda, Inc. on darwin

When attempting to initialize a KeyBERT model I get a segmentation fault error. I'm not sure what the next steps would be to debug this. I'm well-versed with R, but still new to Python so I don't have a toolbox of debug steps. Any thoughts about where to look first?

>>> from keybert import KeyBERT
>>> kw_model = KeyBERT()
zsh: segmentation fault  python

I've run it in a Jupyter notebook with the same results. The first time I used it, the models were downloaded (I expect succesfully?). But when attempting to initialize the model, I get this segfault.

Edited to add, I used faulthandler to get this output:

(base) user@place ~ % python -q -X faulthandler
>>> from keybert import KeyBERT
>>> kw_model = KeyBERT()
Fatal Python error: Fatal Python error: Segmentation fault

Segmentation faultThread 0x

00000002054742c0 (most recent call first):
  File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1568 in _load_from_state_dict
  File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/transformers/modeling_utils.py", line 469 in load
  File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/transformers/modeling_utils.py", line 473 in load
  File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/transformers/modeling_utils.py", line 473 in load
  File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/transformers/modeling_utils.py", line 475 in _load_state_dict_into_model
  File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2517 in _load_pretrained_model
  File "/Users/user/opt/anaconda3/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2326 in from_pretrained
zsh: segmentation fault  python -q -X faulthandler

Jesse · Accepted Answer · 2022-12-05 20:33:10Z

3

I have solved this issue. Steps

Reinstalled PyTorch using conda
I imported torch before importing KeyBERT

Runs successfully, no errors, output is as expected. Thanks for anyone that read this.

answered Dec 5, 2022 at 20:33

Jesse

4088 silver badges18 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

How do I debug a segmentation fault in Python when loading a BERT model?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related