Once I am getting next batch from DataLoader, there is an error message:
E tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:267] failed call to cuInit: CUDA_ERROR_NOT_INITIALIZED: initialization error
The program finally crashes with RuntimeError: cuDNN error: CUDNN_STATUS_INTERNAL_ERROR once the code reaches neural net. I have recently updated tensorflow to 2.11.0, PyTorch is 1.13.0+cu117, and I have NVIDIA TITAN V GPU with nvidia-smi:
I am on conda environment, I have
cudnn 7.6.5.32 and nvidia-cudnn-cu11 8.5.0.96 installed. Any help would be appreciated!
