Writing a recommendation letter for a potential graduate student.

I served on the graduate admissions committee for two years in Carnegie Mellon University's Machine Learning Department in the School of Computer Science*.  I read a lot of good recommendation letters in that time, and some bad ones.  I jotted down a few notes on what makes a good recommendation letter and thought I'd share them here.  Recommendation letters are probably the most important part of a application package.  Great recommendation letters can save an otherwise ho-hum applicant, and bad recommendation letters can absolutely sink a otherwise good applicant.

First off, take a few sentences to describe yourself (the letter writer), your background, and your exposure to high-caliber students.  We want to know the group of students to which you are comparing when you say "great student" or "one of the best".

State clearly in what capacity you worked with the applicant, and for how long.

If you know the applicant in a research context, you NEED to talk about research.

Tell us, what did the applicant do? Give concrete examples.  Why was it good?  What was the effect of this contribution?  A paper, a successful project?  How would the group/project have faired without the student's contribution?

Compared to others on the team, how does the student compare?  Is he performing at the level of a PhD student?  PhD graduate?  It can be difficult to place a student in a percentile (i.e. to say top 5% or 10%), but know that many other people do put this in their letters, so committees look for it.

If you interacted with the student in a teaching capacity only, this makes your letter much more difficult.  Focus on research-related classroom contributions (paper discussions, projects).  This is more important than the student's final grade.

If you are in industry (and so the candidate is not a student, but an employee), the letter is difficult too.  Leadership, drive and determination are important traits to portray.  Solving very technical problems is also a plus.

Do not devote more than a few sentences to personality.  But do mention it if you feel it is important.  Be aware of language biases when writing letters for women (http://phys.org/print208513780.html).   I have personally seen good female candidates get very fluffy letters (friendly and cheerful is not a research strength), and have seen it negatively impact final ratings.

Don't be afraid to point out flaws.  No applicant is perfect, and pointing out the good and the bad makes the letter more honest.  What are their weaknesses?  Have they done anything to compensate for them?  Are they aware of the weakness and are working on it?

On that note, are there any notable flaws in your candidate's application?  A semester of bad marks?  A non-traditional background?  Discussing these in the letter can be very helpful.

Keep it brief.  The committee I was on read 300 applications in 2 weeks.  That's 900-1200 reference letters, not to mention all of the other material in application packets.  Make your points clear and succinct.

In the end, remember that I want to like your candidate!  Your letter should help me do that.

*I'm not on the committee anymore, students typically serve for 2 years.

Confidence intervals for rank accuracy

A quick write up on testing significance of a system when accuracy is measured by ranking a list of candidate answers

http://www.cs.cmu.edu/~afyshe/small_papers/rank_accuracy.pdf