Research Blog

Here are the effects of the first 5 eigenvectors on the mean shape for a model of a wire-cutter tool:







Here is a contour of two tools occluding eachother:



Segmenting this contour by hand yields the following two partial contours (shown in dashed-red):


With hand-labelled correspondences between partial contours and the mean shape, 50 (scale, orientation) pairs were sampled. Here are some of the resulting sample completed shapes (in green) for the first partial contour (on the left):






Clearly, either of the first two completions is to be preferred over the latter two; however, the maximum likelihood shape completion was closer to the bottom two:



The situation is slightly better for the second partial shape (on the right). Here are a few sample completions (in green):






and here is the best completion:



Here is a brief discussion (from an email to my advisor) of the challenges of shape completion as pertains to this experiment:

-------------------------------------------

i've just completed some experiments with known correspondences, and one
issue that stuck out as being problematic is that the shape completion is
highly sensitive to the eigenvalues of the gaussian eigenspace model. for
example, if the model thinks that opening and closing of a tool (a
wire-cutter) is much more likely than changes in perspective, the completed
shape will incorporate very little perspective change, even if the resulting
completed shape is quite ridiculous (e.g. makes the tool look like one of
the handles is crooked).

clearly this is a modelling problem, and not a flaw in the shape completion
algorithm. one way this issue is dealt with in some of the active contour
literature is to use the original dataset of complete contours from which
the model is generated to simply discover modes of transformation
(eigenvectors) while ignoring the relative eigenvalues associated with the
transformations. thus, such an discovery algorithm for the wire-cutter
should discover the modes of opening/closing, and one or two perspective
transformations. this approach makes sense when the changes in contour
appearance have more to do with external factors, such as camera positioning
or deformation by a human (opening/closing), rather than to internal changes
such as the change in shape between two different wire-cutters (as was the
case for the fish classification problem).

--------------------------------------------