Resume sweep and edit batch size

[NataliaAlves13]

My job was interrupted while running the sweep command. I would like to resume the sweep without overwriting all files already completed. Is this possible?
Also, how can we increase the batch size manually? I tried changing the plans.pkl file but it just gets overwritten to 4 when training is resumed.

[mibaumgartner]

Dear @NataliaAlves13 ,

there is a nndet_sweep command which will run the sweep - it is also possible to skip the inference step if all cases were already predicted successfully.

You can only overwrite the batch size for new trainings, modifying parameters while training is not supported.

Best,
Michael

[NataliaAlves13]

Thank you for the reply Michael. The nndet_sweep command was interrupted after 1 day but not all cases were predicted yet. Is it possible to resume the inference only for cases that haven't been predicted? In any case, how can the inference step be skipped if all cases are predicted successfully?
I do want to overwrite the batch size for a new training, how can I do that?

[mibaumgartner]

Hi,

1. It is currently not implemented to only predict cases which are missing in the sweep step.
2. Sorry, just noticed that we haven't introduced the flag in the github version yet .. You can simply modify this line:

   nnDetection/scripts/train.py

   Line 377 in b0504dc

   run_prediction=True, # TODO: add commmand line arg

3. The most robust way is to change the plan file.