RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS

[1245994042]

tomtop5@tomtop5-B360-M-AORUS-PRO:~/download/clevr-iep-master$ python3 scripts/run_model.py --program_generator models/CLEVR/program_generator_700k.pt --execution_engine models/CLEVR/execution_engine_700k_strong.pt --image img/CLEVR_val_000013.png --question "Does the small sphere have the same color as the cube left of the gray cube?"

Loading program generator from models/CLEVR/program_generator_700k.pt
Loading execution engine from models/CLEVR/execution_engine_700k_strong.pt
Loading CNN for feature extraction
scripts/run_model.py:133: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
img_var = Variable(torch.FloatTensor(img).type(dtype), volatile=True)
THCudaCheck FAIL file=/pytorch/aten/src/THC/THCGeneral.cpp line=663 error=11 : invalid argument
scripts/run_model.py:146: UserWarning: volatile was removed and now has no effect. Use with torch.no_grad(): instead.
question_var = Variable(question_encoded, volatile=True)
Running the model

Traceback (most recent call last):
File "scripts/run_model.py", line 301, in
main(args)
File "scripts/run_model.py", line 85, in main
run_single_example(args, model)
File "scripts/run_model.py", line 159, in run_single_example
argmax=(args.sample_argmax == 1))
File "/home/tomtop5/download/clevr-iep-master/scripts/iep/models/seq2seq.py", line 162, in reinforce_sample
encoded = self.encoder(x)
File "/home/tomtop5/download/clevr-iep-master/scripts/iep/models/seq2seq.py", line 86, in encoder
out, _ = self.encoder_rnn(embed, (h0, c0))
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/rnn.py", line 192, in forward
output, hidden = func(input, self.all_weights, hx, batch_sizes)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 324, in forward
return func(input, *fargs, **fkwargs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 288, in forward
dropout_ts)
RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS

if you have same problem，plz help me to solve it ! Thx!

i use pytorch 0.4 , python 3.5.2 ; GPU:1660 ; CUDA: 10.1 ; Cudnn:7.5.0

[1245994042]

When i train the Program Generator ,i typed it in step by step.(python scripts/train_model.py
--model_type PG
--num_train_samples 18000
--num_iterations 20000
--checkpoint_every 1000
--checkpoint_path data/program_generator.pt)
I also get this error:RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS

The details are as follows:

python3 scripts/train_model.py \

--model_type PG
--num_train_samples 18000
--num_iterations 20000
--checkpoint_every 1000
--checkpoint_path data/program_generator.pt
Reading features from data/train_features.h5
Reading questions from data/train_questions.h5
Reading question data into memory
Reading features from data/val_features.h5
Reading questions from data/val_questions.h5
Reading question data into memory
Traceback (most recent call last):
File "scripts/train_model.py", line 490, in
main(args)
File "scripts/train_model.py", line 151, in main
train_loop(args, train_loader, val_loader)
File "scripts/train_model.py", line 171, in train_loop
program_generator, pg_kwargs = get_program_generator(args)
File "scripts/train_model.py", line 343, in get_program_generator
pg.cuda()
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 258, in cuda
return self._apply(lambda t: t.cuda(device))
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 185, in _apply
module._apply(fn)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/rnn.py", line 112, in _apply
self.flatten_parameters()
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/rnn.py", line 105, in flatten_parameters
self.batch_first, bool(self.bidirectional))
RuntimeError: CuDNN error: CUDNN_STATUS_SUCCESS