While using GPU for PyTorch models, getting the CUDA error: Unknown error?

I am trying to use a pre-trained model using PyTorch. While loading the model to GPU, it is giving the following error:

Traceback (most recent call last):
  File "model\vgg_model.py", line 45, in <module>
    vgg_model1 = VGGFeatureExtractor(True).double().to(device)
  File "C:\Users\myidi\Anaconda3\envs\openpose\lib\site-packages\torch\nn\modules\module.py", line 386, in to
    return self._apply(convert)
  File "C:\Users\myidi\Anaconda3\envs\openpose\lib\site-packages\torch\nn\modules\module.py", line 193, in _apply
    module._apply(fn)
  File "C:\Users\myidi\Anaconda3\envs\openpose\lib\site-packages\torch\nn\modules\module.py", line 193, in _apply
    module._apply(fn)
  File "C:\Users\myidi\Anaconda3\envs\openpose\lib\site-packages\torch\nn\modules\module.py", line 199, in _apply
    param.data = fn(param.data)
  File "C:\Users\myidi\Anaconda3\envs\openpose\lib\site-packages\torch\nn\modules\module.py", line 384, in convert
    return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
  File "C:\Users\myidi\Anaconda3\envs\openpose\lib\site-packages\torch\cuda\__init__.py", line 163, in _lazy_init
    torch._C._cuda_init()
RuntimeError: CUDA error: unknown error

I have a Windows 10 Laptop, Nvidia 940m GPU, Latest Pytorch and CUDA Toolkit 9.0 (Also tried on 10.0).

I have tried re-installing the GPU drivers, restarted my machine, re-installed PyTorch, Torchvision and CUDA Toolkit.

While using the following to see if PyTorch is detecting a GPU:

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

I am getting the following output: device(type='cuda').

What could be the possible issues? I have tried the solution mentioned here: https://github.com/pytorch/pytorch/issues/20990 and the issue still persists.

I simply put the torch.cuda.current_device() after import torch but the issue still persists.