ImageSharp.Compare

[UkooLabs]

@JimBobSquarePants

Thought id let you know I've started to extract the image comparing code and make it its own library, as wanting to have something I can use for ImageSharp.Textures and other projects. So thought its time I start making it a nuget.

Currently doing the unit tests then will be doing the build scripts, thought id mention here incase once done if you wanted to pull it in as a SixLabors Library.

I'm open to suggestions for a decent name :)

Here is the repo for now...

https://github.com/UkooLabs/ImageSharp.Compare

[antonfirsov]

@UkooLabs ideally, it would be the best to expose it from the core library. We had quite many requests for this feature, so I opened #1129 to track this request.

In any case the hard thing is that the comparer code has a big flaw now:
The ImageSimilarityReport thing is just terrible, and useless as a debugger tool. It would be much more helpful if the API would return difference images in case of mismatch.

I don't like the idea of exposing the API in any form before this weakness is fixed. (Sidenote: If you need a good reason why are we so conservative about public API-s, see the complaints in #1283 :) )

Briefly:
I discourage a separate library, especially if based on the current code, but if you file a PR dealing with #1129 and the difference image thing, I'm more than happy to review it, and get it merged.

[UkooLabs]

@antonfirsov

My plan is to spit out a image showing differences for my use case but have not been sure how to show the differences, the idea of the black/white/grey has given me a few ideas. So plan is not just to do a straight copy and paste and improve based on some of my own image comparison needs and what I think may help.

For now ill keep as a separate project as find it easier to develop/test that way until its ready to be pulled in, hopefully that is ok.

[JimBobSquarePants]

Yep, works for me. Work separately for now so you can thrash out ideas then we can look to merge.

[UkooLabs]

@JimBobSquarePants @antonfirsov

I've created my first pass of library complete with tests.

FYI I found a bug in ImageComparerExtensions.cs the using a region check width and height of 0 still checks a pixel, the code should be...

IEnumerable<PixelDifference> outsideChanges = r.Differences.Where(
    x =>
    !(ignoredRegion.X <= x.Position.X
    && x.Position.X < ignoredRegion.Right
    && ignoredRegion.Y <= x.Position.Y
    && x.Position.Y < ignoredRegion.Bottom));

I added a method to help aid calculating the Manhattan Per Pixel value (which I feel is useful if you want to do a tolerant exact compare but with slight color drift)

https://github.com/UkooLabs/ImageSharp.Compare/blob/56651267f1e0aa502726103d709c285375b3807a/src/ImageSharp.Compare/ImageComparer.cs#L41

I also added a create difference image method which creates the black, white image as suggested, see for code...

https://github.com/UkooLabs/ImageSharp.Compare/blob/56651267f1e0aa502726103d709c285375b3807a/src/ImageSharp.Compare/Reports/ImageSimilarityReport.cs#L28

Not sure if i should provide a way of overlaying the greyscale image on-top of the actual image, thoughts welcome...

I now need to do create build pipeline...

Please have a look and let me know any thoughts...

[antonfirsov]

Functionally it's fine and very useful now! The API needs some changes before exposing it publicly. We need to hide less relevant details to make it thinner, and to maintain flexibility of implementation without the risk of breaking changes:

• The class should be pixel-agnostic, taking/returning Image instead of Image<T> when possible. (Hint: use IImageVistitor and Image.AcceptVisitor to double dispatch into the pixel-specific implementation.)
  • CompareImagesOrFrames should take non-generic images and return a non-generic ImageSimilarityReport
  • CreateDifferenceImage() should return Image instead of Image<Rgba32>
• The PixelDifference type looks like an arbitrary implementation detail, than anything else. It's also inefficient to collect Width * Height instances into a list. Since we may want to alter the way the final sums and difference images are calculated, I suggest to make it internal. If users want to observe the points where pixels differ, they can use the difference image now.

[UkooLabs]

Here is example output showing differences (I did small changes on odd squares and big changes on even squares) and used tolerant compare. Grey is same, White / Black is where actual color is above or below expected

https://github.com/UkooLabs/ImageSharp.Compare/blob/master/tests/Images/ExpectedDifference.png

[UkooLabs]

@antonfirsov @JimBobSquarePants question for either of you since I'm saving ImageFrames in the report is there a way of either converting a frame to image so I could provide a expected image with difference over the top or similar for actual.

[JimBobSquarePants]

ImageFrameCollection.ExportFrame(index) allows you to do this.

[UkooLabs]

@antonfirsov @JimBobSquarePants sorry to ping again i dont see a way of doing an equiv double dispatch with a non generic ImageFrame, should I change the process to convert the ImageFrames to Images instead?

[JimBobSquarePants]

I would export to an image instead.