H.R. 3699, the Research Works Act, was introduced Dec. 23, 2011, by Rep. Darrell Issa (R-Calif.), chairman of the Committee on Oversight and Government Reform, and committee member Rep. Carolyn Maloney (D-NY). According to the Association of American Publishers (AAP) website, “The legislation is aimed at preventing regulatory interference with private-sector research publishers in the production, peer review and publication of scientific, medical, technical, humanities, legal and scholarly journal articles.” Put another way, it is designed to thwart activities such as the National Institutes of Health (NIH) Public Access Policy, which requires scientists to submit final peer-reviewed journal manuscripts that arise from NIH funds to the digital archive PubMed Central upon acceptance for publication.

In the harsh words of The Guardian, “This is the moment academic publishers gave up all pretence of being on the side of scientists.” I, for myself, never believed their good intentions.

I you are a resident of USA, please take the time to sign the petition against RWS.

Bossa: Extended BoW Formalism for Image Classification
In image classification, the most powerful statistical learning approaches are based on the Bag-of-Words paradigm. In this article, we propose an extension of this formalism. Considering the Bag-of-features, dictionary coding and pooling steps, we propose to focus on the pooling step. Instead of using the classical sum or max pooling strategies, we introduced a density function-based pooling strategy. This flexible formalism allows us to better represent the links between dictionary codewords and local descriptors in the resulting image signature. We evaluate our approach in two very challenging tasks of video and image classification, involving very high level semantic categories with large and nuanced visual diversity.

I’ve also had two papers accepted on our counterpart national conference, SIBGRAPI. The first is the work of the Ph.D. student Ana Lopes and her R.A. Elerson Santos (supervised by Prof. Arnaldo Araújo and co-supervised by Prof. Jussara Almeida; I give her some technical and nontechnical support every now and then). It concerns the use of transfer learning of concepts from (static) image datasets to video datasets in order to recognize human actions. We show that learning the concepts present on the Caltech256 dataset allow a classifier to obtain improved results on the challenging “in the wild” human action Hollywood2 dataset.

Transfer Learning for Human Action Recognition
To manually collect action samples from realistic videos is a time-consuming and error-prone task. This is a serious bottleneck to research related to video understanding, since the large intra-class variations of such videos demand training sets large enough to properly encompass those variations. Most authors dealing with this issue rely on (semi-) automated procedures to collect additional, generally noisy, examples. In this paper, we exploit a different approach, based on a Transfer Learning (TL) technique, to address the target task of action recognition. More specifically, we propose a framework that transfers the knowledge about concepts from a previously labeled still image database to the target action video database. It is assumed that, once identified in the target action database, these concepts provide some contextual clues to the action classifier. Our experiments with Caltech256 and Hollywood2 databases indicate: a) the feasibility of successfully using transfer learningtechniques to detect concepts and, b) that it is indeed possible to enhance action recognition with the transferred knowledge of even a few concepts. In our case, only four concepts were enough to obtain statistically significant improvements for most actions.

The second is the work of my Ph.D. student Marcelo Coelho and his R.A. Cássio dos Santos Jr. (again his main supervisor is Prof. Arnaldo de Araújo). It concerns the clean-up of noisy SIFT features of street-view images (urban façades). We have found out that subspace clustering, a non-supervised technique, is able to isolate clusters of useful and non-useful SIFT features for the task of retrieving a target image. The challenge is identifying a priori which cluster is the relevant one. This work compares and contrasts two subspace clustering techniques: FINDIT (based on dimension voting) and MSSC (based on a fuzzy mean-shift).

Subspace Clustering for Information Retrieval in Urban Scene Databases
We present a comprehensive study of two important subspace clustering algorithms and their contribution to enhance results from the difficult task of matching images taken of the same object using different devices at different conditions. Our experiments were done on two distinct databases containing urban scenes which were tested using state-of-the-art matching algorithms. After initial evaluation of both datasets by that procedure, clustering algorithms were applied to them. An exhaustive comparison was performed in every cluster found and a significant amelioration in the results was obtained.

I’ll put a link to the preprints as soon as I have they become available.

An interesting difficulty I have faced for reporting on this work was showing representative images, without hurting the sensibilities of reviewers and readers. So far, my (admittedly coward) choice has been taking the tamest images that are still representative of the phenomena I want to illustrate. For example: to illustrate that the dataset is ethnically diverse, I would chose frames where only the faces of the actors are shown; to illustrate that the dataset contains gay porn as well as straight porn, I would show a frame with the actors kissing instead of having sex; etc.

But recently, I had a tough choice to make. A student was to submit his Master disstertation to the viva-voce committee, and, as it usually happens in Brazil, he has sent me a draft for corrections and suggestions. His “Results” chapter contained, among cold graphs and tables, several very explicit images, illustrating in detail the cases of success and failure of our algorithm. The only thing is: all images contained censor bars.

I returned the draft with several corrections, among which, a note begging him to remove the bars:

“Don’t censor the images — it’s extremely distasteful: this is a scientific work for an adult audience. Either remove the images entirely (if they are not needed), either keep them uncensored (don’t mess up with the data !). In the worst case, put them in an Annex or in a separate suplement.”

In the end, he’s decided to keep the images uncensored, which I feel was the right scientific decision.

Nevertheless, everytime I open his “Experimental Results” chapter I cringe a little bit. Againg admittedly cowardly I am looking forward for the defense, when I’ll be able to share the responsibility for the final decision — keeping or taking away the images from the definitive version — with the rest of the committee.

* * *

Taking a (superficial) look in the literature, I noticed that many authors (including myself) practice a form of “partial self-censorship”: choosing “tame” images, making them tiny in the page, or using washed out grayscale reproductions — a compromise between scientific truth and respect to the taboo ? Or just plain cowardice ? Most authors simply don’t include images, and a few choose to employ the censor bars. The full-fledged honesty of my student is rare.

The censor bars, IMHO, are the worst choice — at once hypocritical and unscientific. Hypocritical, because the reader can perfectly imagine what is behind them, so any of the “dirtiness” from which they would be supposedly “protecting” the reader is still being created in his or her mind. The effect is exactly the same as when using euphemisms like “f-word”: the  correct word is still created in the listener mind. Unscientific, because they count on the reader imagination (with its distortions, imprecisions, and, often, amplifications) instead of depicting precisely the phenomena at study.

Interestingly, in one paper, the authors censor the faces of the actors (by pixelization). This is an interesting choice and raises a question I have not considered: since we collect our dataset from pornography sharing social networks, we cannot assume that everyone in the video is a professional actor. I hope that none of our examples have Computer Vision scientists unaware that their amateur videos have escaped to the net !

* * *

In the end of the day, this is 2011 — 64 years since the first Kinsey report ! Shouldn’t science have got some guts by now ?

As I browse through more and more advanced material, the small inconsistences accumulate and the whole building starts  showing dangerous crevices. For example: as the emphasis progresses towards solid contructive concepts like computability and decidability, why do we continue to put up with explosive can of worms like supertasks, cardinals, diagonalization and the axiom of choice ?

Take that famous proof of the existence of uncountable infinite sets, by Cantor, the (in)famous diagonal argument (I will be quoting heavily from that Wikipedia article). I love that proof, and I remember how wonderful I found it when I saw it for the first time. But the more I think about it, the less watertight it starts to look.

Basically it ask us to put all the infinite binary sequences in one-to-one correspondence to the natural numbers, like below, and invites us to build another sequence D, by taking the ith bit of the ith sequence (in boldface) and negating it:

s₀ = (0, 0, 0, 0, 0, 0, 0, …)
s₁ = (1, 1, 1, 1, 1, 1, 1, …)
s₂ = (0, 1, 0, 1, 0, 1, 0, …)
s₃ = (1, 0, 1, 0, 1, 0, 1, …)
s₄ = (1, 1, 0, 1, 0, 1, 1, …)
s₅ = (0, 0, 1, 1, 0, 1, 1, …)
s₆ = (1, 0, 0, 0, 1, 0, 0, …)
(…)
D  = (1, 0, 1, 1, 1, 0, 1, …)

As it goes, the arguments says, D clearly is not in the list, which pretended to contain all sequences. So there can be no such list, and therefore no bijection between the sequences and the natural numbers. Consequently, the set of those sequences in uncountable. QED. *applause*

But, of course, I am not so happy, otherwise I would be seeping my tea and watching reruns of ‘Sex and the city’ instead of puzzling over established 19th century math. The crux of my insatisfaction is that the proof is not “computationally effective”, because building D is a supertask.

Here’s a troubling though experiment. Take the same list above, and now sort it in reverse-lexicographic order (with 0 before 1). Now, we have obviously:

s₀ = (0, 0, 0, 0, 0, 0, 0, …)
s₁ = (1, 0, 0, 0, 0, 0, 0, …)
s₂ = (0, 1, 0, 0, 0, 0, 0, …)
s₃ = (1, 1, 0, 0, 0, 0, 0, …)
s₄ = (0, 0, 1, 0, 0, 0, 0, …)
s₅ = (1, 0, 1, 0, 0, 0, 0, …)
s₆ = (0, 1, 1, 0, 0, 0, 0, …)

Now, given any “recipe” for a sequence and using the hierarchy of ordinals, you should be able to tell me the position of the sequence in the list. For exemple, all sequences whose last 1 occurs at a finite position correspond to very predictable finite ordinals. The trouble (for me) starts with the less well-behaved sequences. Which ordinal, for example corresponds to 01010101…01… ? And which corresponds to 10101010…01… ? Those are sequences with very simple “recipes”, finding them shouldn’t be so troublesome.

Now for an even more disquieting idea: the last sequence in this list will be, by construction:

s_something = (1, 1, 1, 1, 1, 1, 1, …)

But which ordinal is something?

Now, a handful of paradoxes isn’t what one gets when playing with infinites without defining the limiting proccesses ? At least that is what E. T. Jaynes keeps telling me…

* * *

The problem, as I see it, admittedly extremely naïvely (to the point of deserving three consecutive adverbs), is that the theory of ordinals lets you always express the first infinite thing you don’t have, but never the last infinite thing you have.

Imagine I had a handle for the “last” infinite natural number — let’s be creative and call it ∞. It would have to correspond to the position of the “last” bit in my sequences (because each sequence has countably many bits, numbered 1, 2, 3…). Observing that the first sequence to have a 1 in the ith position is sequence # 2^i-1, I would know, for example that x of the sequence…

s_x = (0, 0, 0, 0, 0, 0, 0, … ) ( 1 ] at ∞

…could be no other than x = 2^∞-1. And as for my something above, it would be simply 2^∞-1. Now, that starts to look like uncountably many !

Isn’t a notation like…

s_y = (1, 0, 1, 0, 1, 0, 1, … ) ( 0, 1 ] at ∞
s_z = (1, 0, 1, 0, 1, 0, 1, … ) ( 1, 0 ] at ∞

…just a fancy way of grasping limiting behaviors at infinity and thus avoiding Jaynes’ dreaded paradoxes ? With the specifications above, y and z are clearly defined. Without them, they are irretrievably lost.

This paper presents a survey of human action recognition approaches based on visual data recorded from a single video camera. We propose an organizing framework which puts in evidence the evolution of the area, with techniques moving from heavily constrained motion capture scenarios towards more challenging, realistic, “in the wild” videos. The proposed organization is based on the representation used as input for the recognition task, emphasizing the hypothesis assumed and thus, the constraints imposed on the type of video that each technique is able to address. Expliciting the hypothesis and constraints makes the framework particularly useful to select a method, given an application. Another advantage of the proposed organization is that it allows categorizing newest approaches seamlessly with traditional ones, while providing an insightful perspective of the evolution of the action recognition task up to now. That perspective is the basis for the discussion in the end of the paper, where we also present the main open issues in the area.

The survey was submitted for peer review at the CVIU, and is available as a preprint at arxiv.org.

We propose MONORAIL, an indexing scheme for very large multimedia descriptor databases. Our index is based on the Hilbert curve, which is able to map the high-dimensional space of those descriptors to a single dimension. Instead of using several curves to mitigate boundary effects, we use a single curve with several surrogate points for each descriptor. Thus, we are able to reduce the random accesses to the bare minimum. In a rigorous empirical comparison with another method based on multiple surrogates, ours shows a significant improvement, due to our careful choice of the surrogate points.

I am particularly proud of this paper, not only because of the method itself, but also because of the experimental design we propose for the validation. I have been studying for more than a year the topics of Design of Experiments, statistical tests and validation. This is the first of a crop of publications that are employing those rigorous evaluation tools, which, though commonplace in other fields, are still seldom used in Computer Sciences.

Instead, I was astonished by how much the presented selection was similar in terms of technical foundation: classification based on discriminant approach (almost always using SVM) and representation based on “bags of visual features”. It is not that those do not interest me — after all, our own work is sits squarely on those pillars — but I was very interested in hearing about, seeing other approaches: generative models based on latent or explicit semantics, representations based on constellation models — what do I know ? — perhaps something completely new, which I haven’t even heard about.

I was left wondering why those “competing theories” were so notably absent. Has the community decided that SVM + Bags of Features is so conspicuously better than everything else ? (If that is the case, I would like to know how they reached this conclusion — though I like the results given by the pair “bags + SVM”, I am far from considering the “case closed”).

Was it self-selection by the autors, who didn’t submit their works to this particularly community ?

Or — and this is obviously the worst scenario— have all the alternative works been retained at the peer review barrier, because ideological considerations have (maybe  unconsciously?) tainted the assessment of quality. I would like to quick dismiss this latter possibility, but the similarity between the works was really astounding.  My student Otávio Penatti, who is on his first months of Ph.D. (he was there presenting a demo of his M.Sc. work) remarked it immediately.

I was very glad, nevertheless, to have this opportunity to visit Philadelphia. It was a very moving experience for me, because it gave me a very concrete, very immediate realization of how strongly The Enlightenment was shining in America at that time.

* * *

Otávio and I have profited from our travel to the USA to visit Prof. Edward Fox in Virginia Tech, who was the former Ph.D. advisor of Otávio’s current Ph.D. avidsor and my Post-Doc advisor Prof. Ricardo Torres. We have an ongoing cooperation with Prof. Fox. In fact, while we were there, we have met a Brazilian colleague of ours, Nadia Kozievitch, who is spending an year of her Ph.D. with Prof. Fox.

While we were there, we gave a talk on our current work and got acquainted with several exciting projects Prof. Fox is conducting, on a broad array of applications of digital libraries,  including identification of fingerprints, biodiversity databases, e-Science, cooperation for crisis situations, and education.

We have also met Brazilian Prof. João Setúbal, who showed us the Virginia Bioinformatics Institute, and talked about his work in genomics, and the new field of transcriptonics.

We were very impressed not only with the infra-structure of Virginia Tech, but also with the kindness and attentiveness of everyone who received us.

The new lab — which we named RECOD — aims to embrace the research subjects of machine learning, multimedia retrieval and classification, multimodality and digital forensics.

The foundation of this new lab both celebrates a history of fruitful colaboration between its participating members and inaugurates a new phase of tighter cooperation, in which the synergy of our complementary competencies will be fostered in an optimized environment.

I cannot avoid to be proud that my colleagues have accepted both my name and logo suggestions for the  new lab.

Long live RECOD !

Here is the abstract:

“In Content-based Image Retrieval (CBIR), accurately ranking the returned images is of paramount importance, since users consider mostly the topmost results. The typical ranking strategy used by many CBIR systems is to employ image content descriptors, so that returned images that are most similar to the query image are placed higher in the rank. While this strategy is well accepted and widely used, improved results may be obtained by combining multiple image descriptors. In this paper we explore this idea, and introduce algorithms that learn to combine information coming from different descriptors. The proposed learning to rank algorithms are based on three diverse learning techniques: Support Vector Machines (CBIR-SVM), Genetic Programming (CBIR-GP), and Association Rules (CBIR-AR). Eighteen image content descriptors (color, texture, and shape information) are used as input and provided as training to the learning algorithms. We performed a systematic evaluation involving two complex and heterogeneous image databases (Corel e Caltech) and two evaluation measures (Precision and MAP). The empirical results show that all learning algorithms provide significant gains when compared to the typical ranking strategy  in which descriptors are used in isolation. We concluded that, in general, CBIR-AR and CBIR-GP outperforms CBIR-SVM. A fine-grained analysis revealed the lack of correlation between the results provided by CBIR-AR and the results provided by the other two algorithms, which indicates the opportunity of an advantageous hybrid approach.”

I will be travelling to Philadelphia on late March to present the poster. I am very excited about this upcoming trip to the United States, where I am to meet several friends and colleagues, but at the same time, worried about the radicalization of air security rules and the exaggeration of perception of threats. Have we got so scared to die that we decided instead not to live ?