Writing a recommendation letter for a potential graduate student.

I served on the graduate admissions committee for two years in Carnegie Mellon University's Machine Learning Department in the School of Computer Science*.  I read a lot of good recommendation letters in that time, and some bad ones.  I jotted down a few notes on what makes a good recommendation letter and thought I'd share them here.  Recommendation letters are probably the most important part of a application package.  Great recommendation letters can save an otherwise ho-hum applicant, and bad recommendation letters can absolutely sink a otherwise good applicant.

First off, take a few sentences to describe yourself (the letter writer), your background, and your exposure to high-caliber students.  We want to know the group of students to which you are comparing when you say "great student" or "one of the best".

State clearly in what capacity you worked with the applicant, and for how long.

If you know the applicant in a research context, you NEED to talk about research.

Tell us, what did the applicant do? Give concrete examples.  Why was it good?  What was the effect of this contribution?  A paper, a successful project?  How would the group/project have faired without the student's contribution?

Compared to others on the team, how does the student compare?  Is he performing at the level of a PhD student?  PhD graduate?  It can be difficult to place a student in a percentile (i.e. to say top 5% or 10%), but know that many other people do put this in their letters, so committees look for it.

If you interacted with the student in a teaching capacity only, this makes your letter much more difficult.  Focus on research-related classroom contributions (paper discussions, projects).  This is more important than the student's final grade.

If you are in industry (and so the candidate is not a student, but an employee), the letter is difficult too.  Leadership, drive and determination are important traits to portray.  Solving very technical problems is also a plus.

Do not devote more than a few sentences to personality.  But do mention it if you feel it is important.  Be aware of language biases when writing letters for women (http://phys.org/print208513780.html).   I have personally seen good female candidates get very fluffy letters (friendly and cheerful is not a research strength), and have seen it negatively impact final ratings.

Don't be afraid to point out flaws.  No applicant is perfect, and pointing out the good and the bad makes the letter more honest.  What are their weaknesses?  Have they done anything to compensate for them?  Are they aware of the weakness and are working on it?

On that note, are there any notable flaws in your candidate's application?  A semester of bad marks?  A non-traditional background?  Discussing these in the letter can be very helpful.

Keep it brief.  The committee I was on read 300 applications in 2 weeks.  That's 900-1200 reference letters, not to mention all of the other material in application packets.  Make your points clear and succinct.

In the end, remember that I want to like your candidate!  Your letter should help me do that.

*I'm not on the committee anymore, students typically serve for 2 years.

Confidence intervals for rank accuracy

A quick write up on testing significance of a system when accuracy is measured by ranking a list of candidate answers

http://www.cs.cmu.edu/~afyshe/small_papers/rank_accuracy.pdf

Getting around Matlab pool's StorageLocation error

I just spent several hours trying to figure out why I couldn't open a Matlab pool on one of my research group's computers.  I kept getting an error like this:

> matlabpool open
Starting matlabpool using the 'local' configuration ... stopped.
Error using matlabpool (line 136)
Failed to open matlabpool. (For information in addition to the causing error, validate the configuration 'local' in the Configurations Manager.)
Caused by:
    Error using distcomp.interactiveclient/start (line 88)
    Failed to start matlabpool.
    This is caused by:
    You do not have permission to write to the current StorageLocation: /afs/dir/blah
        You should change directory to somewhere where you do have write permission

I do have write privileges to /afs/dir/blah, so I wasn't sure what Matlab was complaining about.  Perhaps it was something to do with afs, I thought.  So, I worked around it by doing this:

sched = findResource('scheduler', 'type', 'local')
set(sched,'DataLocation','/another/dir')

Then, everything proceeds as normal:

>> matlabpool open
Starting matlabpool using the 'local' configuration ... connected to 12 labs.

I hope this post saves someone else some time!

VC Theory & Shattering Coefficients

I've put together some notes on VC theory and shattering coefficients. Be forewarned, I'm told by my statistics friends that my proofs aren't rigorous. I do think they can help build intuition about shattering, so that one might write a more formal proof.

I tried to convert this latex file into html, but ran into many barriers regarding math fonts on OSX. I hope it's still useful in it's pdf form:

http://www.cs.cmu.edu/~afyshe/shattering-coeff.pdf