<H1>4. A Metric for Matching Simulation to Video</H1>
        We use a perceptually motivated metric to compare the motion of cloth in simulation with a video sequence of real fabric motion. Our algorithm compares the two sequences frame by frame and computes an average error across the entire sequence. Real fabrics exhibit a wide variety of motion ranging from soft and flowing (satin) to stiff (linen). Our metric captures the complex dynamics of cloth motion and also helps to distinguish between different fabrics. Researchers in computational neurobiology hypothesize that the human perceptual system is sensitive to moving edges in video 
11 , 12 , 36 . Studies have shown that the receptive fields of simple cells in the macaque cortex act as edge or line detectors, responding to oriented edges or lines in natural scenes 
19 , 35 , 10 . In cloth, these edges correspond to folds, which are regions of high variation in shape. Hence, our perceptually motivated metric for cloth compares two video sequences, one from simulation and one from the real world, and returns a number that measures the differences in their folds. The metric also penalizes the silhouette mismatch between the two sequences. Fold Detection and Representation: Folds appear as soft edges in video whose appearance is dependent on material properties and lighting. Haddon and Forsyth 
15 , 16 describe a learning approach for detecting and grouping folds (and grooves) in images of fabrics. Their technique can handle lighting effects caused by diffuse inter-reflections in cloth. However, most fabrics have very complicated reflectance properties. In our experiments, we normalize the effects of lighting and material reflectance by projecting a structured light pattern of horizontal stripes onto the fabric. From the light-striped video sequence, we compute the dominant orientation for each edge pixel by convolving it with a steerable filter bank 
13 . In our implementation, we use the G2/H2 quadrature pair with kernel size 12 as the basis filters. Details of computing the dominant orientation from the coefficients of filter bank response are given in Appendix I of Freeman and Adelson 
13 . We convolve the image with the filter bank, compute the filter coefficient responses, blur the coefficients using a gaussian kernel, and compute the dominant orientation from these coefficients. We define the resulting orientation image as an angle map, shown in Fig. 1 . The angle map, which measures the local orientation of the projected pattern, has a constant value when the surface is planar and varies at folds. We threshold the gradient of the angle map to get a gradient mask M k for each frame of video ( Fig. 1 ). M k (i, j) = 1, 0, δ(i, δ(i, j) j) ≥ &lt; τ τ ( 4 ) where τ is a user defined threshold and δ(i, j) is the magnitude of the gradient of the angle map at (i, j). The gradient mask is non-zero at regions of high gradients, corresponding to folds, and zero at planar regions. Fold Comparison: Our metric computes the frame by frame sum of squared differences (SSD) between masked angle maps in simulation with video. We preprocess the input video sequence to compute the angle map at each frame. Similarly, in simulation, we render the cloth shape using the current parameter values and project the same striped pattern, to get a striped simulation sequence. We compute the angle map at every frame in simulation from this sequence. We then compute the SSD of the angle values for all overlapping points in the two angle maps. We pre-multiply this difference with the gradient mask, which helps to emphasize the differences in fold regions over planar regions ( Fig. 2 ). We sum the error across all frames to compute the overall error across the entire sequence. The error at any particular frame k along the sequence is S x S y E k f old = ∑ ∑ M k (i, j) · (θ real k (i, j) − θ sim k (i, j)) 2 ( 5 ) i=0 j=0 where (S x , S y ) is the size of the angle maps and θ real , θ sim are the angle values from real and simulation angle maps respectively. Silhouette Comparison: In addition to the angle map error, we penalize the silhouette mismatch between the simulation and the video of real cloth. This penalty is proportional to the difference between the two silhouettes, i.e., the number of mismatched pixels. S x S y E k silh = ∑ ∑ | A k real (i, j) − A k sim (i, j) | ( 6 ) i=0 j=0 where 1, inside silhouette A k (i, j) = 0, otherwise ( 7 ) The total error in frame k is E k = E k f old + αE k silh ( 8 ) where α is a user-defined weight that controls the relative contribution of the two terms. We used a value of 0.1 for α in our experiments. The error across the entire sequence of length N frames is given by N E = ∑ E k ( 9 ) k=1
        c The Eurographics Association 2003.
        Bhat et al. / Estimating Cloth Simulation Parameters from Video
        
          
          Figure 1: Top Row: Input light striped image. Bottom Row (left to right): angle map and gradient mask.
        
        
          
          
          Figure 2: The stages in the metric pipeline. Top row (left to right): Angle map from video, angle map from simulation. Bottom row (left to right): angle map difference, final metric value for this frame (angle map difference multiplied by gradient mask from video).
        
        c The Eurographics Association 2003.
        Bhat et al. / Estimating Cloth Simulation Parameters from Video
        
          
          Figure 3: This plot shows angle map error as a function of bend and stretch stiffness parameters. Dark areas indicate regions of small error and bright areas correspond to large errors. Note that the space is fairly noisy. The minimum found by the optimizer is contained in the large dark region in the lower portion of the plot.
        
      
      
        