{"slug": "advice-on-interviewing-candidates-for-ai-safety-fellowships", "title": "Advice on interviewing candidates for AI safety fellowships", "summary": "A former MATS fellow, who was rejected from several AI safety fellowships before being accepted into MATS on Team Shard, recommends that fellowships provide letters of recommendation for rejected but promising candidates and make application timelines clear upfront. The author, who co-authored a spotlight talk at the MATS Symposium and submitted to NeurIPS, argues that such letters would reduce duplicated review work across fellowships and increase the field's capacity, though it requires collective action from major programs like MATS, Astra, and the Anthropic Fellows Program.", "body_md": "| Read on [LessWrong](https://www.lesswrong.com/posts/jmGvMSnkemSPLs6qv/thoughts-on-interviewing-candidates-for-ai-safety) |\n\nAround July last year I decided I was going to go all in on technical AI safety research. To do that I’d need to get into an AI safety fellowship, quit my job, and sell everything that was in my flat in South Africa (hopefully in that order).\n\nI applied to every fellowship that was open 1, and got rejected from several\nof them before being accepted into MATS on Team Shard around mid-November. I\nhanded in my notice the following Monday, and told my landlady that I had to\nmove out in 6 weeks because I was leaving the country.\n\nMATS went great! My co-author and I got the first spotlight talk at the MATS Symposium, we’ve submitted to NeurIPS, and I’m more hopeful than ever that I’ll be able to help reduce x-risk.\n\n## Things are good, but they could be so much better\n\nI’m hoping that any goodwill I earned from [this\npost](https://www.lesswrong.com/posts/eFD3rozNCZKMe4rTs) ~~can be sacrificed as\na peace offering~~ allows me to make suggestions without people assuming I\nhave ill intent. If AGI goes well, I believe AI safety fellowships will have\nplayed a significant role. I hope these fellowships can become even more\neffective than they currently are at putting good people in good places. And I\nhope this post helps these fellowships be marginally more effective.\n\n## Don’t say “we were impressed by your profile” unless you mean it\n\nI got a lot of rejections, and all of them required someone to spend time reviewing my application. Sometimes, quite a lot of time! These rejection letters generally sounded like:\n\nwe are genuinely disappointed that we couldn’t find a match for you this time, as we were impressed by your profile\n\nThis is nice to hear! But also it comes across as a bit hollow: I would *love*\nto see fellowships put some weight behind the words. One option is to actively\nprovide references for rejected (but promising) candidates. I’m imagining\nsomething like:\n\nLetter of Recommendation\n\n`[name]`\n\napplied to`[fellowship]`\n\nin`[year]`\n\nand we were impressed by their application. We thought they were very strong because of`[specific points]`\n\n. We thought their main weak points were`[specific points]`\n\n. We were unable to offer them a position due to`[specific reason]`\n\n, but we identified them as having strong potential for`[field]`\n\n. We can recommend fast-tracking their application, and have attached their responses to our questions for your perusal. Please contact`[name]`\n\nat`[email address]`\n\nif you have questions.Kind regards,\n\n`[name]`\n\n,`[fellowship]`\n\nThis is a strong letter: it makes falsifiable claims, it makes clear the decision process of the fellowship, and it makes recommendations to other fellowships. It also takes time to write, significantly more time than sending a rejection email. If a candidate is not strong enough that you’d feel comfortable writing this letter, then don’t! But the current status quo around rejection emails is a mixture of appeasement of the candidate and obscurity of the rejection reason, which does not encourage rejected candidates to improve.\n\nThis letter does something else: it allows other fellowships to spend less time\nreviewing promising candidates, if that candidate had already been reviewed by\na similar fellowship. And in a complementary fashion, *you* can spend less time\nreviewing promising candidates if you receive a letter of recommendation from a\nfellowship you trust.\n\nLetters of recommendation would increase the capacity of the AI safety field, by reducing the duplicated work done across different fellowships. However, this requires solving a collective action problem and making letters of recommendation the norm, which I suspect is a process only the big players (MATS, Astra, Anthropic Fellows Program) can kick-start.\n\n## Make the whole application timeline clear upfront\n\nDuring the application process, candidates want to know how many rounds of “You ask, I respond” there will be. They want to know the deadlines for all those responses, and how much work is expected for each of their responses.\n\nNot being clear about timelines makes it impossible for candidates to ensure they’ve got free time to complete those applications properly. This is not a case of “good enough candidates will figure it out”: poor communication selects for candidates who have lots of time, rather than for the best candidates. This adds noise to your application process, increasing false positives and false negatives.\n\nIf you do not know those deadlines before you open your applications, please communicate your best estimates of these deadlines.\n\n## Word limits, character limits, and timed forms\n\nWord limits are good, but please ensure your application form calculates the\nword count as the candidate is typing, and displays this clearly.\nUnfortunately, using a word counter is not enough: some applications think that\nhyphenated words (“fine-tuning”) count as one, others think they count as\ntwo. Please tell the candidate what the count is, and tell them *before* they\nclick submit.\n\nIf you can, please quietly accept ~5% over the word limit without penalty. Having a word limit is good: it ensures the candidate can think concisely, and reduces the reviewer burden. I don’t think being <5% over the limit detracts from either of those goals.\n\nIn light of the above, I’d recommend having *alphanumeric-only character\nlimits* instead of word limits or plain character limits. Candidates can cheat\nword limits by hyphenating words, by playing tricks with the formatting, or\notherwise making their response harder to read. That’s terrible! Don’t create\nselection pressure for hard-to-read responses! Similarly, imposing a\n*character* limit encourages responses that are harder to read by removing\nparagraphs, headings, formatting, _emphasis_, citations, etc.\n\nIf you only count alphanumeric characters in your limit, then you’re not penalising candidates for adding paragraphs, headings, _emphasis_, or other punctuation that makes responses easier to review, while still enforcing something similar to a semantic limit on complexity.\n\n## Proctored tests are a terrible, terrible time\n\nMostly because proctoring software is really bad. I’m aware that proctoring\nsolves a real problem, and I’m in favour of the platonically ideal version of\nproctoring. I imagine that really good software wouldn’t be more stressful than\nan in-person test, but I’ve *never* used good proctoring software. Bad\nproctoring software artificially adds noise to your estimate of a candidate’s\nquality, meaning you’ll accept people you should have rejected, and reject\npeople you should have accepted. Some specific recommendations:\n\n-\nGive candidates a “dummy” assessment that they can take at their leisure, so they’re not figuring out the weird proctoring software while also trying to complete a timed assessment. This assessment should be as close to the real thing as possible, and candidates should be able to take it as many times as they’d like. Proctoring software is really bad at indicating the status of the session (is it working? has my timer started? has my timer finished? is my screen being recorded properly? can I open other tabs? can I go to the bathroom? if the software freezes, can I refresh the page safely? can I change WiFi networks without the software ending my session? etc etc) and giving a “dummy” assessment helps mitigate the issues caused by this.\n\n-\nEncourage candidates to manually record their screens\n\n*in addition*to using the proctoring software. I had a proctoring session be cancelled without explanation during my timed assessment, and was only able to submit my application because I was recording my screen and had proof to show the fellowship administrator. -\nMost proctoring software offers “live troubleshooting”, but it’s usually slow, and your timer continues to tick down while you’re troubleshooting. This is less than helpful when you’re attempting to do your best.\n\n-\nMost proctoring software doesn’t let you export your answers, so if the software decides to reset your progress without warning, you will have lost everything. This problem is solved by giving candidates a “dummy” assessment.\n\n-\nMost proctoring software requires you to take a video of the room you’re in before you start, and this is fine.\n\n*Some*proctoring software requires you to take a video of your room*after the timer has started*and this is not fine. Please ensure your software does the former, and not the latter. -\nFor one proctored session, I had to use my cell phone data for the 3 hour proctored video call because the proctoring software just wouldn’t let me connect via my WiFi, with no explanation given about what was wrong with my WiFi. This problem is solved by giving candidates a “dummy” assessment.\n\n## Make it clear who you’ve accepted in the past\n\nThis point is hard to make with certainty, but if you are happy with the fellows you’ve accepted in the past, you should make some information about them public in your applications. For example: alma mater, previous fellowships, previous AI safety experience, education level, gender, age, technical background. The goal here is to encourage promising candidates to apply. Many fellowships have built up an aura of being impossible to get into, and this is only valuable so far as it is true. If a candidate believes the odds are 10,000 to 1 and does not bother applying, this is a loss to the fellowship if the odds were actually 1000 to 1. Applications should encourage candidates who are more likely to get in, and similarly discourage candidates who are unlikely to get in.\n\n## Be aware of other fellowships’ deadlines\n\nWhile one week is enough to complete *one* fellowship’s round-2 application,\nit’s certainly not enough to complete *multiple* fellowships’ round-2\napplications.\n\nI’m sure there are reasons for fellowships clustering their deadlines together\nso that they’re all on the same weekend. But if you choose to do this, please\nbe aware of the cost: the increased workload will cause even very good fellows\nto *appear worse* if your deadline is the same as another large fellowship’s\ndeadline. The best candidates will likely ignore a less-prestigious application\nif its deadline coincides with a more-prestigious one.\n\nIf tracking the various fellowship’s deadlines is too much to ask, please consider giving two weekends between announcing a round of interviews and the deadline for those interviews. This allows fellows to give one weekend to one fellowship, and the other weekend to your fellowship. Allowing for two weekends is especially important if you want to receive applications from people with a full-time job and not just bright-eyed bushy-tailed college students.\n\n## Be aware of your *own* deadlines\n\nMost fellowship applications have two tiers: first you apply to the fellowship,\nand second you apply to the mentor(s) you’d like to work with. If you’re asking\nfor candidates to review many (10+) mentors and then to apply to these mentors,\nplease be aware of how long this could take to do properly. If a candidate\nwants to spend ~20m reviewing each mentor, that still takes 3 hours(!) to look\nthrough them all. This is *before* the candidate has started to answer the\nmentor’s questions. When deciding on deadlines, be aware that *even deciding\nwhich mentors to apply for* takes a long time.\n\nIf possible, make the list of mentors (and their biographies, research interests, etc) available publicly and early so that candidates can start reviewing the mentors as soon as possible. Gating this information behind the first round of interviews just reduces the quality of applications that the mentors end up reviewing, since candidates have less time to consider which mentors to apply to.\n\n## Fellowships compete for fellows\n\nIf your fellowship is clear and upfront about what’s expected of the candidates during applications, gives reasonable deadlines and makes reasonable accommodations, then your fellowship will have better fellows applying to it! Just as candidates compete for limited places in fellowships, the best candidates will have multiple choices for which fellowship to attend. It’s in your best interests to ensure the best candidates get through to your final round.\n\n## Anywhere-on-earth is the only timezone that matters\n\nMost rationalist fellowships are very good about this (thanks!), but it’s worth saying: The only timezone for deadlines should be anywhere-on-earth.\n\n## Your application is probably northern-hemisphere centric\n\nIt’s very weird seeing DEI disclaimers about equality and then having the applications be in “winter” while it’s a bright summer’s day outside.\n\n## Footnotes\n\n-\nAstra, CAIRF, ERA, MARS (multiple streams), MATS (multiple streams), Pivotal, and SPAR.\n\n[↩](#user-content-fnref-1)", "url": "https://wpnews.pro/news/advice-on-interviewing-candidates-for-ai-safety-fellowships", "canonical_source": "https://boydkane.com/essays/how-not-to-interview-candidates", "published_at": "2026-08-24 11:55:00+00:00", "updated_at": "2026-08-24 19:46:00.333670+00:00", "lang": "en", "topics": ["ai-safety", "ai-policy"], "entities": ["MATS", "Astra", "Anthropic Fellows Program", "NeurIPS", "LessWrong"], "alternates": {"html": "https://wpnews.pro/news/advice-on-interviewing-candidates-for-ai-safety-fellowships", "markdown": "https://wpnews.pro/news/advice-on-interviewing-candidates-for-ai-safety-fellowships.md", "text": "https://wpnews.pro/news/advice-on-interviewing-candidates-for-ai-safety-fellowships.txt", "jsonld": "https://wpnews.pro/news/advice-on-interviewing-candidates-for-ai-safety-fellowships.jsonld"}}