Why Batch inference is not support currently？

[LovingThresh]

AssertionError: Batch inference is not support currently, as the image size might be different in a batch.

It is Confusing.

[xiexinch]

Hi @LovingThresh,
Since the validation or test set images have different shapes, support batch inference at evaluation might affect the result.

[edsml-hmc122]

Hi @LovingThresh, Since the validation or test set images have different shapes, support batch inference at evaluation might affect the result.

See #2965
It would be nice to at least have support for batched inference when all images have the same dimensions.

[louan1998]

I followed the official documentation to run the "unet_s5-d16_deeplabv3_4xb4-40k_chase-db1-128x128.py" module, and reported an error "AssertionError: Batch inference is not support currently, as the image size might be different in a batch" when iterating to 4000 times, and then I use the --resume command to continue training on the weight of 4000 iterations, and report this error again when it reaches 8000 times. This data set is also supported by mmseg. I saw that the size of the pictures inside is the same. why is that?

[edsml-hmc122]

I followed the official documentation to run the "unet_s5-d16_deeplabv3_4xb4-40k_chase-db1-128x128.py" module, and reported an error "AssertionError: Batch inference is not support currently, as the image size might be different in a batch" when iterating to 4000 times, and then I use the --resume command to continue training on the weight of 4000 iterations, and report this error again when it reaches 8000 times. This data set is also supported by mmseg. I saw that the size of the pictures inside is the same. why is that?

@louan1998 I think right now it doesn't matter what the image sizes actually are. MMSegmentation will reject the inference if batch_size != 1, even if the images are all the same size. It's just not implemented yet, from what I understand. :(

[louan1998]

@edsml-hmc122 That's right! I noticed that too! If batch_size = 1 in val_dataloader， there is no problem! Anyway, batch_size doesn't play that big role in validation and testing, right?

[chenhuagg]

@louan1998 How did you solve it in the end?

[louan1998]

@louan1998 How did you solve it in the end?

Set the batch_size in the val_dataloader to 1
val_dataloader = dict( batch_size=1, num_workers=16, persistent_workers=True, sampler=dict(type='DefaultSampler', shuffle=False), dataset=dict( type=dataset_type, data_root=data_root, data_prefix=dict( img_path='images/validation', seg_map_path='annotations/validation'), pipeline=test_pipeline))

[edsml-hmc122]

@edsml-hmc122 That's right! I noticed that too! If batch_size = 1 in val_dataloader， there is no problem! Anyway, batch_size doesn't play that big role in validation and testing, right?

It plays a huge role if you have a lot of validation/testing data. The process is slowed down by a massive amount, and you are under-utilising your GPU. I hope the developers will be able to implement batched inference asap, it's the biggest downside of MMSegmentation in my opinion, compared to other libraries. Also, MMDetection supports batched inference from what I understand, maybe they can port the code.

[xiexinch]

Hi @edsml-hmc122,
I removed this limitation, you might have a try. If there are any problems, feel free to create an issue, and we'll fix it asap.
https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/data_preprocessor.py#L135

[edsml-hmc122]

Hi @edsml-hmc122,
I removed this limitation, you might have a try. If there are any problems, feel free to create an issue, and we'll fix it asap.
https://github.com/open-mmlab/mmsegmentation/blob/main/mmseg/models/data_preprocessor.py#L135

Thank you, I will try to test it when I have time! Also, might be worth setting this issue to "open" again.