First of all I'd like to say hello to everyone who participates in this Forum. I'm new in this Forum but I hope to get "old" soon!

Well, my question now. For the time being I'm involved in a AI Project and I experiment in backpropagation with Matlab 6.1. What I do is facial emotion recognition from a dataset of 15 individuals, I have personally obtained (each one has been photographed twice for each of the emotions "angry" "happy" "neutral" and "sad").

My main concern,for now, is the preprocessing of the data. Matlab documentation says that in order to do batch processing I have to create a matrix (for train) in which each column corresponds to one pattern (image in my case of 30x32 pixels). So the matrix for a training set of 80 images would be a 960(rows)x80(columns). Is this ok so far?

Next step is preprocessing. The problem is that if I use the function premnmx, which scales from -1 to 1, then matlab scales the graylevels at each row of the train-matrix!! What I mean if my matrix is mat=[ 1 2 3; 3 2 5; 1 1 0;9 2 3] then premnmx(mat)=[-1 0 1; -0.33 -1 1; 1 1 -1; 1 -1 -0.7143]. But if I scale from -1 to 1 each image-column I will get [-1 1 0.2; -0.5 1 1; -1 -1 -1;1 1 0.2]. Which preprocessing scalling do I need to use and why? In my case where I have column=images I saw that when I scale with the second way and plot with function "imagesc()", the image remains(correctly?) the same. However, when I scale with matlab's row-wise(first method I descibed) way I take a plot of the image that is not the same as the original(seems to be a distorted version of the original)!!! Common sense says that The second way is not going to be better..because I say that some information was lost from the image before feeding it in my NN!

I have excactly the same wonder , when I use the prestd function for normalizing the data to mean=0 and std=1. Matlab's function normalizes so as each row of my train-matrix has mean=0 and std=1. The other way is to normalize so that each column (hence each image's total pixels) has mean=0 and std=1. Once more, what method to use and why!...And to be honest..does it matter?(I gave an answer to this before..for premnmx...though not sure yet)

And finally I'm coming to the Subject of my post: Principal Component Analysis! Suppose we have a train-matrix of 80 images, hence 80x960 matrix. When I use prepca( I previously used prestd) matlab complains that rows are more than columns. Searching deeper in PCA I found that Matlab uses Singular Value Decomposition in order to find the Principal Components. But SVD( trainmat=U*S*V') will give a matrix S->960x80 with singular values(if I use svd(trainmat) and not svd(trainmat,0)!). But this matrix which is not rectangular will have only 80 non-zero values in the leading "diagonal". U would be 960x960 and V would be 80x80! Then the Transformation matrix (which keeps the eigenvector row-wise) is obtained TransMat=U' , which is a 960x960 matrix. Then, if ,for example, I decide to keep only the first 300 more important pixels(the ones that contribute more one the total variance) I cut the last 960-300=660 colums=eigenvectors in the 960x960 transormation matrx...hence my final Transformation matrix is 300x960. Finally, I project all my 80 initial images into the new 300-Dimenional space by simply multiplying PC=TransMat*trainmat, where PC are the projections-called Principal Components!....Yeah..alright.. but this only in papers! The problem is that when Matlab's prepca comes to decide how many important-pixels is going to keep it uses the singular values from matrix S which are only 80 non-zero singular values. From these 80 non-zero (in leading diagonal of S) singular values then obtains the eigenvalues=variances and compares with a min_frac to see how many eigenvalues are important and correspondigly how many eigenvectors to keep. But by doing this the maximum we can get is 80 eigenvectors(the same number as the number of patterns=images), since singular values are only 80(non-zero). What I mean is that after trainmat=U*S*V' (SVD) I get a 80x960 U matrix (as I said 80= maximum).And finally a prepca(trainmat) will give a final matrix of 80x80 (80 images=patterns=columns ,80 more important pixels=rows).

I've really stuck on this! And I can't find an other way to get a final matrix (after PCA) with more pixels than the number of patterns=images!!!!!!

When I use the following technique I get other results than the technique with SVD..why??:

trainmat=960x80.
Find the covariance matrix C (dimensionality=#pixels=960 which is going to be 960x960. Call [eigvec,eigval] = eig(C) and do eigendecomposition of C. So get the 960 eigenvalues and the 960 eigenvectors. Reorder eigenvalues from larger to smaller and do accordingly with eigenvectors, hence obtain eigvec_ordered. Take the first 300, for example, more important eigenvectors! Compose TransMat=eigvec_ordered(:,1:300)', which is a 300x960 matrix. Multiply PC=TransMat*trainmat I get a 300x80 matrix which says that I have 80 patterns each with 300 more important pixels!..yeah but this obtains different TransMat than when using SVD, and I'm talking when in both cases we choose to keep the first 80 more important pixels!!!....

To be honest either I scale/standarise column-wisely or row-wisely I get quite the same accuracy (in my NN 60-70% for emotion recognition). But I can't understand what is the difference. Additionally I want some help in PCA. I suspect that Matlab's prepca was designed for preprocessing when the patterns well outnumber the dimensions (in my case patterns are only 80 and dimension=960=pixels).

Thanks before hand for any help,
onegin

I don't know anything about Matlab, but I'm also working on the problem of classifying facial expressions. After tinkering around with some simple methods I've decided that I probably will need to use a neural net of some kind to do the classififaction.

I've used openCV to detect human faces from a webcam mounted on my robot, and have then used some simple (i.e. non-neural) methods to locate the main facial features. The main thing I'm now trying to do is firstly classify the gaze direction of the person by looking at the pupil positions and secondly classify the overall facial expression. My initial thoughts are to use a WTA or other simple type of neural net to do the classification.

I'd be interested to hear how you get on and how well you get your system to work.

Thanks for your reply, nice to hear from someone doing something so interesting.

Well to be honest, I had started my research trying to do head pose recognition for 4 separate poses: straight,up,left,right. My dataset was composed of people with somehow "excessive" head posing, if you know what I mean. So I did not had cases where a person looks slightly on his left or right..up..etc...hence the NN classifier had easy job to do with 90-95% generalization accuracy (I had also used the "winner-take-all approach").

In comparison with facial expression analysis, pose detection is less demanding, since ANN classifiers can identify more easily exaggerated intensity in some areas of the image (this happens when we have out-of-plane rotation of faces) and thus classify successfully. (actually if you use a 1-hidden layer NN and initialize hidden-weights to 0, finally you can plot the weights and get images of human-morphs looking right,straight..left..up or mixed images looking both up and left..etc). I suspect that in pose recognition the "important" pixels are much more than in emotion recognition. When refering to important pixels I mean pixels which contribute more on the total variance(for example if your pictures are taken in the same place then the background pixels are all approximately of the same intensity level..so their variance is small...hence don't contribute too much on the recognition process).Of course all these are my thoughts...and I'd like to discuss them with anyone who is interested! So the WTA approach with an MLP with 1-hidden layer would be ok for pose recognition.

Now, in emotion recognition the areas of image that contribute to the recognition process are... lets say...areas around the eyes..eyebrows..cheeks..mouth..etc. That is why the most Facial Emotion Recognition systems only extract those pixels areas and feed them on a MLP. But in my case I use the picture from the subject's shoulders and up (like an id photo.). Unfortunately, though prectictably , I get generalization accuracy of 35-45%. So I decided to search for a face detection stuff and I feed my network with only faces from now on (area between ears, hair and jaw). Now..and after input normalization(see my previous post) I achieve 60-67% emotion recognition accuracy, remarkably larger than before...and quite acceptable for a pixel-raw input system (no feature exctraction and feeding..only pure pixels, apart from PCA of course). I want to stress that subjects were looking only straight when photographed(variation in both pose and emotion looks to be a "too many irons in the fire" situation).

Don't hesitate to contact, if you want to ask anything else.
However, I warn you that all the above are merely my view!

onegin

You can see what I'm doing here

http://www.fuzzgun.btinternet.co.uk/rodney/components.htm#FaceDetection

The current algorithm extracts the relevant facial features - eyes nose and mouth - and I now need some reasonably reliable way of classifying these. I want the learning to be realtime, so I can just look in one direction and then click a "look left" button to train the robot that my eyes are currently looking to the left. A simple associative neural net such as a WTA should suffice for this pupose. The data which I would actually feed into the neural net would be an eigenimage (the difference from the average - eg the average eye image), which makes the learning process easier and more illumination invariant.

Avoid relying upon constant background or illumination. This is always a bugbear with vision systems, and developers are often tricked into believing their systems work, only to find that if you move the robot into a different room with different lighting it suddenly stops working. For the system to be practical it must work under normal variation of lighting and with any arbitrary background.

It may be that I need to use a backprop neural net, but these aren't well suited to situations requiring rapid learning and there would need to be a lot of slow offline training involved (which I want to avoid).

Another consideration is the way in which emotional expressions are learned. The traditional method is to learn this as a gestalt, so when presented with the entire face classify it as a certain expression. This may work, but it isn't very efficient and evidence from psychology suggests that this isn't the way that natural vision operates (see Aaron Sloman's "happy" and "sad" eyes as an example). Instead I'll be looking to classify different features as being in different positions - such as "eyes wise", "eyes narrow", "eyebrow up", "eyebrow down", etc. The expression would then be calculated by comparing it to known geometric configurations which correspond to different emotions.

hi I noticed that u have overcome the problem of facial feature extraction when recognising the emotion. In fact I'm an undergraduate doing a Facial Expression Recognition Sysytem as the final Year Project. I'm finding it hard to select a proper method to extract features from an image. I've started researching on GABOR wavelets but have no clue if it's really helpful. Pls mail me if you have any easy method of facial feature extraction technique in mind or in any form. Thank you!

not that i am an expert in PCA, but i think there is another function in MatLab called princomp. Perhaps it will help you. Good Luck

Thank you for your replies guys,

But is there anybody who could give me a hint as to why Matlab scales row-wise the data in the training matrix(see my 1st post) when I use premnmx or prestd. Does it matter anyway? (see 4th paragraph of my 1st post)

Hi,seniors of AI!!I am a beginner in Face Recognition project.I have done until collecting the input data into a file.The problem is i m not able to write MATLAB program for normalization.Can u all pls send me a simple or sample program to normalize input data from a file?Ur help will be appreciated.Thanks

Normalization is very important in face recognition.The first step for normalization is downsampling.Downsampling is done in both row and columnwise.The second step is the calcultion of mean value for the downsampled image.The final step is concatination of column vector.

we need help for our final year [under graduate]project.we have decided to extract the features from a database which contains about six subjects.These features are then used to classify the emotions using neural networks.
we were sucessful with pca[eigen face]algorithm to find out weights for classification. we need to know about other algorithms that call be used for the above purpose...
curiously waiting for the replies which can go a long way in our project......................