Research Blog

the weighted Procrustes Local Shape Distance (where we use a weighted sum of the entire local shape vs. neighborhood size curve instead of just the min value) performed much better for finding correspondences between shapes with missing/extra parts, and i was able to generate the mean shapes for a few examples of pairs of shapes with extra parts.



matching between two birds correctly skips over left foot of red bird.


matching (left) and mean shape (right)

unfortunately, even with the weighted PLSD the quality of the match depended on the order of matching--matching the shape with the extra part to the shape without the extra part did horribly while the other way around did perfectly. this is because we were not allowing the correspondence to skip over any points on the first shape (i.e. the lambda spacing penalties were only on the horizontal edges in the COPAP graph).



good match (left) and bad match (right) due to asymmetry in the correpondence graph.



good match (left) and bad match (right) due to asymmetry in the correpondence graph.

because of this problem, tonight i tried using lambda on both the horizontal and vertical edges of the COPAP graph, where the vertical edges now allow us to skip points on the matching (first) shape. with this small change, i'm now getting correspondences that look quite resonable for almost all the hard classes of shapes in the MPEG shape database (i'll post some picture on my blog when i wake up later today). the only caveat is that now if we turn the lambda parameter down too low, we will get an empty correspondence, since it will low cost to skip over all the points on both shapes. i've been playing around with the lambda parameter, and it seems like a good choice of lambda depends on 1) the complexity of the shape, and 2) the number of points used to represent the shape. i think regression seems like the right way to learn the lambda parameter, although we may also need to adaptively change lambda on the fly if we get a bad match.

here are some examples of correspondences with the symmetric COPAP graph (with the weighted PLSD):




one interesting thing we can see from these pictures is that when one shape has two legs very close together where the other one only has one leg, the correspondence often skips over the concavity between the two legs rather than skipping over one of the legs. a priori, this makes sense--there is no reason to bias the correspondence towards positive curvature portions of the curve. also, from the horse matching, we can see that the tail of the blue horse is matched up to the back leg of the red horse. but notice that the back leg and tail do look strikingly similar! it is only due to prior knowledge of horses that i as a human am able to distinguish the tail from the leg on the blue horse.

finally, flies and lizards are simply extremely hard to match well with contour-based shape models--lizards deform too much, and flies have too many pointy appendages. still--the correspondences are resonable for the lizard at least, and if we didn't know the fly was a fly then the fly correspondences may not be that bad either.