Statistical Map: for edge-segment background modeling

The first problem I tackled since I came to the lab was background modeling through edges. I worked on this problem for a couple of years, and a few papers on the road, I think it’s time to talk a little about it. It was a fun experience, being new to research and all that. However, it was a tough one as it is not a “hot topic” or even approved by the community who works on an opposite direction. Although, there are many possibilities by working with this type of information, we need further research to make it viable for real time applications.

Edge-based modeling

In the literature, the most common methods are pixel-based. That means, that the information used to model the scene is that contained in each pixel. Most methods use the intensity and its changes to create a model of the entire scene.

On the other hand, we are exploring (or should I say explored…) the use of edges as a reliable source of modeling. As the edges reveal the structure of the scene, we can model it with less information and less bulk. However, due to the lack of other non-structural information (pixel intensities) it may be harder to differentiate some objects with similar structure (about this later). Moreover, because we are using less pixels (as the edges in a scene are sparse) we can use more expressive models to represent it.

However, this approach is not mainstream, and is it hard to produce models on it, because you need to pre-process the images to obtain the edges, and then post-process it to obtain the regions. But if you are not interested in the regions and only need higher information, e.g., behavior information, the isolated structures are goodies that you can’t refuse.

Statistical Map

Fig. 1. Overall flow

The main idea behind the edge modeling is the use of a frequency map of the appearance of the edges, that we called Statistical Map (SM). The SM models the edge behavior over time. Ideally, it would create wide distributions for high movement edges, and narrow ones for edges with no movement. From the temporal point of view, the distributions will be high if the edge belongs to background and small if the edge is a moving one. Fig. 1 illustrates this idea.

The use of the SM is the basic model which can be extended by using color, intensity, gradient and other information, as the main problem with edges happens when moving edges get close to the borders of background. Thus, we explore most of these options to enhance the detection, as objects with similar structure will present different color (or other properties).

A more robust model

We increase the robustness of the model by adding memory to the model. We keep a set of lists of the edges that we saw before, as well as their shape, color, gradient, and other information, to build a more discriminative model. Therefore, we can use these lists to match and learn new edges as they appear in the sequence.

Publications

For more information about these methods you can refer to my papers:

Ramirez Rivera, A.; Murhsed, M.; Kim, J.; Chae, O., “Background Modeling through Statistical Edge-Segment Distributions,” Transactions on Circuits and Systems for Video Technology, 2012 (to appear). Preprint link

Ramirez Rivera, A.; Murshed, M.; Chae, O., “Object detection through edge behavior modeling,” AVSS, 2011. Paper link

or for more information check the other related papers at here.

For the BibTeX form, you can use this:

@ARTICLE{RamirezR2012,
author={Ramirez Rivera, A. and Murshed, M. and Kim, J. and Chae, O.},
journal={Circuits and Systems for Video Technology, IEEE Transactions on},
title={Background Modeling through Statistical Edge-Segment Distributions},
year={2012}
}

@INPROCEEDINGS{RamirezR2011,
title={Object detection through edge behavior modeling},
author={Ramirez Rivera, A. and Murshed, M. and Chae, O.},
booktitle={Advanced Video and Signal-Based Surveillance ({AVSS}), 2011 8th IEEE International Conference on},
pages={273--278},
year={2011},
organization={IEEE}
}