New paper accepted

We got a paper accepted on the IEEE Trans. on Image Processing 🙂 . It is about a new micro pattern descriptor that relies on directional numbers (direction indexes in a local neighborhood), and we tested it on face analysis tasks, namely, face identification and expression recognition. We’ve been working on these ideas for some time, and sadly the revision was delayed some time, but finally it’s over.

I hope to give more details next week when I have more time, and give some insights on the ideas behind. Meanwhile, I leave you with the abstract and the pre-print at our site or at the IEEEXplore

Abstract—This paper proposes a novel local feature descriptor, Local Directional Number Pattern (LDN), for face analysis: face and expression recognition. LDN encodes the directional information of the face’s textures (i.e., the texture’s structure) in a compact way, producing a more discriminative code than current methods. We compute the structure of each micro-pattern with the aid of a compass mask, that extracts directional information, and we encode such information using the prominent direction indexes (directional numbers) and sign—which allows us to distinguish among similar structural patterns that have different intensity transitions. We divide the face into several regions, and extract the distribution of the LDN features from them. Then, we concatenate these features into a feature vector, and we use it as a face descriptor. We perform several experiments in which our descriptor performs consistently under illumination, noise, expression, and time lapse variations. Moreover, we test our descriptor with different masks to analyze its performance in different face analysis tasks.

Statistical Map: for edge-segment background modeling

The first problem I tackled since I came to the lab was background modeling through edges. I worked on this problem for a couple of years, and a few papers on the road, I think it’s time to talk a little about it. It was a fun experience, being new to research and all that. However, it was a tough one as it is not a “hot topic” or even approved by the community who works on an opposite direction. Although, there are many possibilities by working with this type of information, we need further research to make it viable for real time applications.

Edge-based modeling

In the literature, the most common methods are pixel-based. That means, that the information used to model the scene is that contained in each pixel. Most methods use the intensity and its changes to create a model of the entire scene.

On the other hand, we are exploring (or should I say explored…) the use of edges as a reliable source of modeling. As the edges reveal the structure of the scene, we can model it with less information and less bulk. However, due to the lack of other non-structural information (pixel intensities) it may be harder to differentiate some objects with similar structure (about this later). Moreover, because we are using less pixels (as the edges in a scene are sparse) we can use more expressive models to represent it.

However, this approach is not mainstream, and is it hard to produce models on it, because you need to pre-process the images to obtain the edges, and then post-process it to obtain the regions. But if you are not interested in the regions and only need higher information, e.g., behavior information, the isolated structures are goodies that you can’t refuse.

Statistical Map

Fig. 1. Overall flow

The main idea behind the edge modeling is the use of a frequency map of the appearance of the edges, that we called Statistical Map (SM). The SM models the edge behavior over time. Ideally, it would create wide distributions for high movement edges, and narrow ones for edges with no movement. From the temporal point of view, the distributions will be high if the edge belongs to background and small if the edge is a moving one. Fig. 1 illustrates this idea.

The use of the SM is the basic model which can be extended by using color, intensity, gradient and other information, as the main problem with edges happens when moving edges get close to the borders of background. Thus, we explore most of these options to enhance the detection, as objects with similar structure will present different color (or other properties).

A more robust model

We increase the robustness of the model by adding memory to the model. We keep a set of lists of the edges that we saw before, as well as their shape, color, gradient, and other information, to build a more discriminative model. Therefore, we can use these lists to match and learn new edges as they appear in the sequence.

Publications

For more information about these methods you can refer to my papers:

Ramirez Rivera, A.; Murhsed, M.; Kim, J.; Chae, O., “Background Modeling through Statistical Edge-Segment Distributions,” Transactions on Circuits and Systems for Video Technology, 2012 (to appear). Preprint link

Ramirez Rivera, A.; Murshed, M.; Chae, O., “Object detection through edge behavior modeling,” AVSS, 2011. Paper link

or for more information check the other related papers at here.

For the BibTeX form, you can use this:

@ARTICLE{RamirezR2012,
author={Ramirez Rivera, A. and Murshed, M. and Kim, J. and Chae, O.},
journal={Circuits and Systems for Video Technology, IEEE Transactions on},
title={Background Modeling through Statistical Edge-Segment Distributions},
year={2012}
}

@INPROCEEDINGS{RamirezR2011,
title={Object detection through edge behavior modeling},
author={Ramirez Rivera, A. and Murshed, M. and Chae, O.},
booktitle={Advanced Video and Signal-Based Surveillance ({AVSS}), 2011 8th IEEE International Conference on},
pages={273--278},
year={2011},
organization={IEEE}
}

Dark Image Enhancement: Channel Division

I’ve been busy for a while working in some research topics. One of those produced some results, and the final product is a paper! (Ok, it is exciting for me )

Channel Division

I designed and created the Channel Division algorithm for dark image enhancement. So, the main idea of the algorithm is to process the image, check for its contents, and build a transformation function according to such context. Thus, the algorithm is context-aware, in the sense that it adapts itself to each image.

Some results:

Original	Channel Division

Note that the enhancement method adapts to different images, and reveals the details in the dark regions. At the same time, it maintain the colors and tries to avoid artifacts.

Paper information

Abstract—Current contrast enhancement algorithms occasionally result in artifacts, over-enhancement, and unnatural effects in the processed images. These drawbacks increase for images taken under poor illumination conditions. In this paper, we propose a content-aware algorithm that enhances dark images, sharpens edges, reveals details in textured regions, and preserves the smoothness of flat regions. The algorithm produces an ad hoc transformation for each image, adapting the mapping functions to each image’s characteristics to produce the maximum enhancement. We analyze the contrast of the image in the boundary and textured regions, and group the information with common characteristics. These groups model the relations within the image, from which we extract the transformation functions. The results are then adaptively mixed, by considering the human vision system characteristics, to boost the details in the image. Results show that the algorithm can automatically process a wide range of images—e.g., mixed shadow and bright areas, outdoor and indoor lighting, and face images—without introducing artifacts, which is an improvement over many existing methods.

The bibliography (human readable):

Ramirez Rivera, A.; Ryu, B.; Chae, O.; , “Content-Aware Dark Image Enhancement through Channel Division,” Image Processing, IEEE Transactions on , vol.21, no.9, pp.3967-3980, Sept. 2012
doi: 10.1109/TIP.2012.2198667 (IEEExplore)

And BibTeX for those who may need it:

@ARTICLE{RamirezRivera2012,
 author={Ramirez Rivera, A. and Ryu, B. and Chae, O.},
 journal={Image Processing, IEEE Transactions on},
 title={Content-Aware Dark Image Enhancement through Channel Division},
 year={2012},
 month={sept},
 volume={21},
 number={9},
 pages={3967--3980},
 keywords={},
 doi={10.1109/TIP.2012.2198667},
 ISSN={1057-7149},}

Database

Also, we created a small database for the algorithm testing. It contains 10 different images in difficult lighting conditions. If you are interested, you can download it here.