Bug of demo.py

[leftthomas]

According to your code, the input image is in RGB format:

SparseInst/configs/Base-SparseInst.yaml

Line 33 in fd6fb38

FORMAT: "RGB"

and the given mean and std is also in RGB format:

SparseInst/configs/Base-SparseInst.yaml

Lines 4 to 5 in fd6fb38

	PIXEL_MEAN: [123.675, 116.280, 103.530]
	PIXEL_STD: [58.395, 57.120, 57.375]

In demo.py, the image is read as RGB format:

SparseInst/demo.py

Line 93 in fd6fb38

img = read_image(path, format="RGB")

and then passed to DefaultPredictor:

SparseInst/sparseinst/d2_predictor.py

Lines 36 to 49 in fd6fb38

	self.predictor = DefaultPredictor(cfg)
	def run_on_image(self, image, confidence_threshold):
	"""
	Args:
	image (np.ndarray): an image of shape (H, W, C) (in BGR order).
	This is the format used by OpenCV.
	Returns:
	predictions (dict): the output of the model.
	vis_output (VisImage): the visualized image output.
	"""
	vis_output = None
	predictions = self.predictor(image)

In DefaultPredictor, the image is firstly converted into BGR format:
https://github.com/facebookresearch/detectron2/blob/96c752ce821a3340e27edd51c28a00665dd32a30/detectron2/engine/defaults.py#L309
and finally be normalized according to the given mean and std:

SparseInst/sparseinst/sparseinst.py

Line 61 in fd6fb38

image = (image - self.pixel_mean) / self.pixel_std

As the given mean and std is in RGB format and the image is in BGR format, so there remains a mismatch.

[wondervictor]

Hi @leftthomas, I'm so sorry for the late reply.
Thank you for reminding me of this problem and I'll check and fix it soon~

[leftthomas]

@wondervictor it still haven't been fixed now.

[nai-kon]

@wondervictor
Thank you for the wonderful project. I also encountered the same problem, when will it be fixed?