# Presentation: Getting Rid of LeetCode Interviews in the World of AI

> Source: <https://www.infoq.com/presentations/ai-lead-interview/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=global>
> Published: 2026-07-29 10:25:00+00:00

## Transcript

**Daniel Doubrovkine:** We're going to talk about getting rid of LeetCode interviews in the world of AI. How many people here are developers? I did a survey amongst developers about coding interviews and writing code on whiteboards, and things like that. About every single person I've asked really hates these LeetCode interviews, and "Feels that they're really dumb." I know some of you don't believe my data, and if you ask me for the source of this survey, I included it here. It's right there on the right. It's in Python. It's the source of the data in the survey. I promise you there will be many jokes here and some personal stories. It's really time that we get rid of these LeetCode interviews. I'm not going to bore you with pros and cons of LeetCode style interviews, why they exist and things like that.

I'm going to tell you a story instead. That's actually a very personal story, and one that I'm sad to admit is that once upon a time, actually exactly a year ago, I failed a very basic coding interview at one of the five very large FAANG companies. Very embarrassed to say that it happened. I didn't just fail a coding interview. I failed to implement the diameter of a binary tree. How many of you can implement the diameter of a binary tree from the top of your head now? It is a totally basic DFS, Depth-First Search. It's exactly 10 lines of code. This is one of the most absolute trivial, as the interviewer said, warm-up exercises that you can ever get in a coding interview at all. It's like you should be able to dump it from the top of your head. It's 10 lines of code with the function definition, the comments and everything like that.

It's five lines of actual code. In fact, just to prove to you that I can actually do a piece of code, the diameter of a tree, here's a recording of me doing it right there on the board. Look, I'm implementing the diameter of a binary tree, and it's going to give me the code. I made it into a GIF. I didn't want to type it. Yes, I'm doing it. You mean I didn't do it? Claude did it? Yes, you're right. Claude did it, but I asked it to do it. It's amazing. I was born in the USSR, and we had our share of difficulties of getting things done, of buying bread or finding technology like a VCR, things like that. Today, these tools like Claude, they're available to everyone, and they're available to everyone equally. We don't need to seize means of production. This is readily available for you right here at your fingertips. These tools are promising us a really bright future, a whole brave new world. Maybe in this future world, we're not going to have to solve puzzles on whiteboards.

Back to me. When I am on the receiving end of these coding interviews, when I'm debriefing candidates, and I see a candidate who has been doing code for quite some time, and they fail miserably at coding some of these puzzles, because, of course, I sometimes work for companies that do them. I just look at the room, and we try to talk ourselves out of the situation. We're like, ok, maybe this person that has been administered yet another diameter of a binary tree, maybe they're just too senior, maybe they're just rusty. Maybe they haven't coded for a long time. It's ok. Maybe we can convince ourselves that we should still hire them, even if they fail at these coding interviews. I worked at AWS. I think more than half of the principal engineer interviews completely bombed their technical coding interviews and still were hired. I've been in these debriefs endlessly.

The problem with me is that, actually, I'm not rusty at all. At the time of when I bombed my diameter of a binary tree, I was coding every day. This is my 2025 GitHub. You could say, ok, Daniel, 2025, you've been doing AI slop. You've been telling Claude to write the code for you, and so your GitHub looks amazing because you've been committing all this stuff with AI. 2025, I buy it. 2025 may have been all AI slop that I produced sitting behind the keyboard and telling it what to do. 2024, possibly AI slop, a little bit. 2023, pretty cutting-edge AI slop, if I was doing that at the time. 2022, definitely not AI slop. 2021 and 2015 and going all the way back, like in Hilary's keynote, we were really writing code. My GitHub green squares look pretty full, haven't really changed since 2014 or something like that.

I've written more tree structures. How many of you have written a tree structure of any kind, at any time? I've actually written, I think, more tree structures than every one of you combined here. I started in the '90s, and my first successful server-side commercial application was a search engine called Alkaline. It powered a small website, amongst others, called whereas.com. I don't know if you remember whereas.com, but whereas.com in '97 was the eighth most trafficked website in the world. I think we were hitting something like eight requests per second. Eight requests per second in '97 was completely insane. It was very hard to stand up a C++ server-side search engine that could respond to eight requests per second within millisecond response time. Whereas.com was used massively, because everybody needed a serial number for their favorite piece of software, and that's where you go to find it.

That search engine also was ported on multiple operating systems, Linux, Solaris, SunOS, Windows, IRIX, you name it. I didn't do Vax, but people were asking for it. Everybody was running this different hardware in these different operating systems, and so we needed a portable baseclasses library that could be compiled on various operating systems. STL was not really a thing at the time, or at least not performing well. This source code is actually on GitHub, but it's tree structures upon tree structures to make a search engine actually work, very different technology. I've written all this cross-platform tree structures to make sure they work everywhere, time and again.

In 2025, ironically, I found myself to be a top 1% Ruby engineer globally, according to this website called Algora. I actually co-maintained 65 or some Ruby gems. I have some credentials that was coding, and yet, even though I have all these amazing credentials, I completely bombed the diameter of a binary tree in a live interview. If you don't believe my code on GitHub, and you don't believe Algora, and the fact that I was top 1% of engineers out there, because you can say maybe you faked commits back to 2014. You can totally lie on your resume. At that time, I was on my sixth year of being a PE at AWS, working on OpenSearch and Search Engine. I was paid absurd amounts of money. If you don't believe me, this is a photo of me on my yacht. I see somebody is like, this is AI.

This is not actually me on my yacht, no? If you don't believe me, here's me in front of my Lambo in Hudson Yards in New York. I promise you, this is not an AI photo, and just believe me that this is my Lambo, and people pay me a lot of money to write code all day long. Definitely not AI. Plus, this is in Hudson Yards. It could be somebody else's Lambo, but I do live in Hudson Yards, and Hudson Yards is the most expensive neighborhood right now in Manhattan. Apparently, according to TimeOut, average home sale price is $6 million. It's pretty intense. I live there. While I'm on this topic, I want to show you my gym. I work remotely from here, often. This is a real picture. This is not AI. This is a drone shot of the Hudson Yards Equinox, where I do have a membership, and this is my office. In summer I glue a thing on my screen so that nobody sees me. I'm not a PM. I'm a developer. I write code just sitting in the sun and in front of the pool writing the code. That is, of course, when I have a job, when I don't bomb interviews for diameters of trees in those interviews.

## Studying Computer Science

We're jumping a little bit ahead. Let's start from the beginning. I was born in Moscow in '76, a long time ago, in the USSR. This is actually my building where I grew up. I grew up exactly right there in the middle on the second floor in an apartment that was like a normal New York-sized apartment, like 300 square feet with my parents, typical thing. Like all Soviet kids, I was really good at math. That comes with the territory of being East European. Being good at math led me to studying computer science. In the late '90s, when I was studying computer science, I absolutely loved algorithms. I was writing Hanoi Tower problems, solving them in three different programming languages, C, Lisp. I wrote x86 assembly. There's some code on the screen here, also you can find that on GitHub. I was trying to squeeze algorithmic solutions in x86 assembly just to get some performance out of these implementations.

Then I'd implement this time and again and spend a lot of time coding these tree structures and other solutions. I was implementing all these algorithms time and again, spending a ton of time in the computer lab in college doing this. One of my classmates even found an iterative solution to the queens problem, the queens that don't attack each other, to find the next position or count the number of solutions available. This was an actual computer science breakthrough that was published in a math magazine at the time. It's a pretty amazing thing. I was so obsessed with algorithms and structures and data that my very first commercial application was a calculator. There's an actual picture from it. I still have an unopened copy, but this was published in Germany, 3,000 copies on the CD-ROM and sold on shelves. This was called Expression Calculator for its shareware version. The commercial version was renamed to Global Calculator in German. My German is terrible. This, I got a paycheck, $3,000 for writing algorithms and calculators, like a basic interview question of how to make one.

## Working at Microsoft

I started two companies, and then eventually I started interviewing for large American companies, because I needed a job because my second startup had failed. My first interview of a U.S. company was for Microsoft. I'm actually working back there now, 25 years later. What I remember from that interview is that Microsoft brought me from Geneva, where I lived, to Paris, put me in an amazing five-star hotel, the Le Louis XV. I don't remember what the interview was, but I remember that it was mostly coding puzzles. I just stood in front of the whiteboard and coded puzzle after puzzle after puzzle of like how many of this and all tree structures and things like that. I remember it being extremely hard. After I got hired, I was told that on that day, the team that was interviewing only hired two people, me and the other guy, who's this French guy who I'm still friends with, who finished top of his class in the hardest telecom school in Paris you can imagine.

The thing where only genius level people can go. Now me, of course, I don't know why I was selected, but clearly, I was able to code these puzzles quite well. I didn't graduate with honors from any school. I was very mediocre at many things, like the AI is today. I absolutely love these coding problems. Then when I started working at Microsoft at the time, I worked on this project called Netdocs, which was like Google Docs, but never actually shipped. I worked on the server side of it. One of the biggest problems that we had on the server side of the software is memory fragmentation. We would allocate memory, release memory, allocate memory, release memory, and would run out of memory eventually because memory was fragmenting at the time on Windows. To solve that problem, we wrote a completely new from scratch C++ library that mostly used the stack as much as possible and then bled into virtual memory when it was running out of stack space.

That solved much of the memory fragmentation other than rebooting servers all the time. That was difficult and that was interesting. This is a library that had tons of tree structures to optimize performance. It was like STL for servers. STL was fragmenting very much, and we invented our own. I implemented everything from tree, graph, vector, strings, tons of practical applications of algorithms. When the project got canned, I wanted to continue to work on algorithms and solving this type of problems. There was a team that was hiring, the Rotor team that was writing the .NET framework for Linux. Before I took my job at Microsoft, I was like, Windows, never. I saw a job offer and I said, maybe I should take this job. Finally, I had an opportunity to go work on Linux again. I really wanted to succeed that interview, the internal interview for the team that was very competitive.

I was put in front of a whiteboard to code a bunch of things, again, tons of algorithms, tons of C++ implementations of all these puzzles. Then I got in front of a whiteboard to implement malloc, a memory allocator from scratch. I think this is the first time I truly bombed the whiteboard interview. I wrote something, but I don't think it was any good. I'm sure there were many other people who were a lot better than me. My self-doubt, the imposter syndrome, all of that really came back up. Like you pass an exam at school. I just remembered while I was standing there trying to code malloc how in like second grade of college, I had to pass an analysis tree exam and I failed it twice. Then I had to spend four months basically preparing for that one exam or risking being kicked out of the math faculty. Here I am standing, failing at the coding interview again. It's, again, very tree structure-like and very algorithmic. I, of course, didn't get the job. That really created a lot of doubt in myself for these coding interviews. I always dreaded them.

## Tenure at a New York Startup

In 2004, I wanted to move to New York. I went and interviewed at a very large bank. I was flown and posted in a typical not very fancy hotel, unlike my Le Louis XV experience in Paris, and was put in a windowless room and left with a computer, which was nice. It wasn't a lot of whiteboard interviews. Was asked to code all kinds of things, mostly unsupervised. I did actually quite well at those because most of the software that I wrote, it was like C, a little bit of Fortran. Most of that software worked because I was sitting in front of the computer myself. I would be given a problem and the person would walk out and come back and we discuss what I wrote. I did very well. The interviewer, the manager came at the end of the day. Like I've been in that windowless room for like six hours.

He puts a paper on the table. He's like, this is a job offer. I looked at the job offer. It was double what I was making at Microsoft. I was like, that's cool. I'm not going to take it because something really concerned me. They were talking about how they don't sleep and have PagerDuty 24/7. I was like, this is not why I'm moving to New York. I don't want to be on call like 24 hours a day, but this offer is amazing. Unusual to get an offer on the spot, but still. Another unusual thing about that interview, I was forbidden to contact any other people at the company. I actually was going to meet another manager who also had a job, but it was a little bit more interesting. That wasn't in fixed income derivatives, which tanked the market later, but I wouldn't know it in 2004.

I was told, you can't call that person anymore. That was weird. I was like, this is a bit weird. I haven't seen the sun all day. Maybe let me go and I'll think about it. Eventually they let me out of this room. The moral of that story is that I was able to do these LeetCode questions. I just needed to concentrate and not have somebody breathing over my neck.

Instead, I actually took a job at a New York startup. They flew me back to New York, I think three days after the bank interview and put me, not into a crappy hotel, but at the Royalton. It's a five-star, super fancy hotel in Midtown Manhattan. At the time they didn't have many developers at the company. It was just a startup. I pretty much got hired on my Microsoft credentials. They're like, of course, this guy knows how to code. He comes from a very important, large company. I wrote a data mining client in C++ that traversed your Outlook and Lotus Notes email to mine it to build a social graph out of it. Notice, tree structures, social graphs, and all this kind of stuff. Think of that application as LinkedIn reads your email, quite exciting. I know a graph is not exactly like a tree, but they have a lot of similarities.

I think I could at the time definitely implement the diameter of the tree, had it been necessary for that application. I had my revenge. Part of my revenge was working for this company where we flew a private jet to open our second office in San Francisco. That was really cool. The second revenge that I had at my failing .NET team interview is that I was giving a lot of interviews myself. The company had raised the series B. We had tons of money. We had a lot of salespeople, but not a lot of engineers. We didn't have a product to sell. We were definitely growing. Our CTO was trying to hire a VP of engineering at the time. He, of course, wanted the VP of engineering to meet the people who were working in the engineering team of the company. He's going to be my future boss.

He asked us to prepare a technical interview for the VP of engineering to see if they can be respected by the engineering team on their technical credentials. The only interview I remember is this guy walks into the room. He's got credentials as long as maybe I have now. Long LinkedIn. He's wearing a suit, which we appreciate in New York. He's telling me as an introduction that he writes code all day, every day. I'm like, this is somebody I would love to work with. We're a small team of five engineers. We have a VP of engineering who writes code all day, every day, yes. Something told me that maybe that's not true. I said, if I ask you to go and write code on the whiteboard right now, like most engineers would do, will you do it? He's like, of course, no problem. I said, like a sorting function?

Can you write a sort on the whiteboard if I ask you to do it? He says, of course, you could ask me to do it. I could do it. I'm like, go do it. The guy, his response was amazing. He said, "If you're really asking me to go to the whiteboard right now and write the sort, I'm walking out of the room." I said, "Feel free to walk out of the room." I was so sure I dodged a bullet. I had my power trip of sitting on the other side and administering a coding interview. There were many reasons why that person didn't get hired. I thought we really dodged it, because he just told me one thing and didn't want to do it.

## Crafting Torturous Live Coding Interview Questions

The more I did these live coding interviews from the interviewer side, the more I liked it. I was getting the real power. I could make a candidate really suffer — it's a very Russian thing — at the whiteboard. I came up with a few methods and wrote about them, about how to make candidates suffer. Number one, you have an expert in something, ask them the dumbest question possible as a warm-up. It's truly insulting. Somebody who has credentials, who can speak about algorithms, wrote tons of them. Just ask them to reverse a string and see if they can do it. They reluctantly will do it because they have to. I love it. The second one is from my own experience at this big American bank. Just put them in the room and leave them there. See how long before they go out of there, be like, can I please have some water?

Another one is, you have an hour. Some problems some people solve them quickly. Some problems, people solve them in a long time. Just ask them to solve it again in a different way. I don't know if there is another way to solve the problem, but they are the candidate. They should be able to tell you. When they say, I don't know if there is another way. "Thank you. Our interview is over. Goodbye." That really destroys the ego. Finally, the one that I absolutely loved in these whiteboard interviews is that as they write the code on the whiteboard, I'll be like, "Keep working on this. I'll be back." Go out, smoke a cigarette, have a coffee. Come back. Look at the board. See if they're stuck. Try to help them again. Maybe just leave them there. Like, "That's ok. We can stop here. We're done. Thank you." Have them get out.

The quicksort there was from my own experience as well. You can guess what happened to that company where I was administering this type of torture to the candidates. The company, the software was fine. The business, of course, failed. We just ran out of money. It didn't matter what we were interviewing these candidates for. Since then, I felt bad. Every time that I was in the room administering a whiteboard coding interview, I just thought, why am I doing this to these people? I would open my interview with saying, "I'm sorry, we're just going to have to work through this together. Let's work through it. I just have to do it."

## Growth at Artsy

In 2011, I wanted to go work for a startup. Got introduced to the founder of Artsy, a New York-based startup. I was the seventh employee. He just needed some advice of how to rebuild the site after a failed demo. I went around startups, asked how people build technology today in 2011 at different startups. I was told, use MongoDB, use Ruby, Ruby on Rails. In about three weeks, we rebuilt the site. I joined the company three weeks down. There was no coding interview. My coding interview was the prototype. I got hired on writing the prototype, the thing that would eventually become this company. We'll go raising $100 million and building the largest fine arts marketplace, most relevant arts publication. It's artsy.net today. Still a global company with a ton of employees. Working at a startup, it really teaches you a few things. Number one, engineers, they aren't lining in front of your door to get hired at your little startup that nobody knows about.

The really good engineers, they're totally impossible for a startup to find. Your whole proposition is that you're going to pay somebody half of the money to work twice as hard. That's really what you're offering from a startup. We went and designed a deliberately very different interview process at Artsy. That process was always centered around things that someone has built. Not some hypothetical coding puzzle, but something that I've actually built. We'd ask them, "What have you made? Tell us about it. Tell us a story. You worked on a project reticulating splines. What's a spline? I don't know anything, explain it to me. How did you build one? How did you reticulate one? How many splines have you managed to reticulate at the end of the day?" This kind of stuff, from their own experiences. Then, we decided that the kinds of people we wanted to hire were these T-shaped people.

People with broad set of interests and narrow specialties. We just wanted to learn about what interested them and where they got really good. One of my best stories, I met an engineer who was at the Art and Tech Conference. We were given a little project to do some soldering and some art thing that would blink. One of us had to do software and the other one had to do hardware. She said, I want to do hardware. Took a soldering iron. Completely blew the circuit up the first time that she tried to solder it. She was like, maybe it will work if I flip the cables. Flipped the cables and soldered it, and it worked. I was very impressed by the just trial and error, which is how I code, the trial and error of doing that hardware. Then this person ended up telling me that she went to RISD to get an MFA in painting.

She was an amazing painter. Then went to Flatiron School to learn to code. We hired her, she was our first junior hire at the company, and did amazing. Today runs, I think, a large consultancy that builds software, 15 years later. Very broad set of interests, art, technology, very good at what she did. The interview process would be an interview with an engineer, there'd also be an interview with a human. We want to see how you work, how you collaborate with others. We want to learn about your experience collaborating with other engineers, and see how you'll fit in a startup where you know everyone, it's a 10-people company, from your past experiences.

Then finally, of course, can you do the actual coding job? We don't need to test you on specifically writing the code, we can just talk about code that you wrote before. Ideally, if you have open-source code from before, we'll just discuss that, and that's good enough to see that you can code. Maybe we'll give you an assignment to take home, and then discuss it during the interview, if you don't have these kinds of credentials. Code is one thing, but you have to design systems. The process of designing a system is interesting, and so we want to understand your systems thinking decision-making. We'll do some whiteboarding of just building, designing a system from a high level, and see what questions you ask in a typical systems interview. Finally, that interview would have a round with somebody very senior at the company, maybe a director or better.

It's really an opportunity to ask questions about the company, roles, expectations. We ask some competency questions at the time to make sure that the person has value alignment with the company and potential for growth. You can see that very little bit is about the actual coding puzzles and things like that. In fact, what I told you before is just half of the interview. The other half of the interview was references. We would call people that you would give as references and just discuss them, what did this person build? Were they good? Why were they the best in the team? Why should we be hiring them? The story is that someone tells about someone else. It's like NPS score. How many people would recommend working with you that have worked with you in the past? That turns out to be a much better signal about somebody's abilities than coding.

We ended up with some pretty awesome and diverse candidates. We sourced a ton of them through open-source collaborations, and some we sourced through non-open-source collaborations. Some people would apply to the company and really appreciate our approach to interviewing without these whiteboard interviews. They would write about their positive experiences online. Sometimes they would not write about the actual interview process, but about how it felt to be in an interview like that that didn't involve coding puzzles. To date, I think that process is some of the best.

## The Individual Contributor Role at AWS

In 2019, I stepped down from being CTO of that company, and I wanted to become an IC again. I just burnt out on managing people and company growth with 400 people. I also wanted to learn something new. I thought, maybe I should go and be an IC at the most interesting technology company to me at the time. I joined AWS. I was flown to Seattle for a principal engineer interview and was told to brush up my algos. On the plane, I thought, I have not brushed up enough algos so far. I wasn't doing LeetCode interviews. Maybe I'll do some on the plane while I'm there. Before I fell asleep to a movie, I stumbled upon an LRU cache. It's like a linked list and a hash table implementation to make reading and writing fast. It's like Least Recently Used. You bound it on some max size, and then it's fast to insert and fast to retrieve.

I was like, this is fun. I haven't seen one of those in a while. I read about it, and then I fell asleep, and then I land in the interview. One of my interviewers is Marc Brooker, he's a distinguished engineer at AWS. The man invented everything, from compute to Lambda to hypervisors. I'm like, I'm standing in front of an impressive engineer here. Then he's like, unfortunately, we're going to have to code a little bit together. I go, sure, I'm prepared for that. Of course, I'm absolutely not prepared. He's like, implement an LRU cache. I ended up working at AWS. He seemed to be quite impressed with my implementation of LRU cache. The only reason I remember this is because I got lucky, and I brushed it up on my way to my interview. I worked on OpenSearch, I've worked on Elasticsearch, tons of code, things like that.

In 2024, I decided to leave, and I started looking for a new role. That is when I failed my diameter of a binary tree. I was actually only prepared to implement an LRU cache. I was absolutely not prepared to implement the diameter of a binary tree. The interviewer didn't know that. They should have asked the LRU cache. I just froze. I couldn't do it. I was thinking, why am I doing this to myself? Here I am doing this interview, asked to do a diameter of a binary tree. I couldn't turn my brain around. I was like, "I'm sorry. Let's not waste our time here. Cut the interview short." Then decided maybe I should stop looking for IC roles. Maybe I am rusty. Maybe I should just go back to management or something like that.

## A Managerial Role at Shopify

I'm good with people. I started interviewing for some managerial roles. Then I found this managerial role at Shopify. At Shopify, managers code, which I think is amazing. They were like, we're going to do some pair programming. Now, of course, the expectations from a manager is not to be hardcore algorithmic solution builder. Still, you have to roll up your sleeves. It's an amazing feature of everyone at Shopify. I was prepared to write code, pair programming with another manager in the room. I get into the interview, and he asked me to do an LRU cache. I absolutely aced my second, third LRU cache of my entire career of interviewing on whiteboards. I get hired. I keep asking people at these companies, why do you do this? Why do you subject people to these coding whiteboard interviews? Why are you still doing it? The most common answer that I get is that we just want a consistent and a fair and objective process that we can truly evaluate.

We want some signal from this process. We want to know something about this candidate in a very uniform way. I'm like, yes, what signal? The candidate can code. I say, obviously, this is not true based on my experience, because none of my interviews were showing anyone that I knew how to code. I just got lucky. I think if you ask a deep code question from any of your candidates, the signal is that they're just good at solving deep code questions. Or maybe that they're lucky. I don't think it's valuable at all. Then, I sometimes turn to shaming the people who organize these processes at companies, because they themselves are developers, and they hate it. Like, why are you doing this to others? This whole whiteboard coding is from punch card days, where we had to write code on paper and then submit it to the computer.

Then, eventually, thankfully, the bank gave me a computer to code on. That was already some progress. Things have changed. The whiteboard was replaced by a computer. Now we have AI. AI is a much faster horse than the computer, or the punch card thing that we had before. The candidates, they are adapting very fast to this world of AI, of interviewing with AI. I relied on luck, but some people just cheat. I could totally have had another computer in front of me, behind my monitor in this remote interview, listening to everything that was said, and spitting out the diameter of a binary tree for me, and I would have probably aced it. Some people are making bold predictions about these LeetCode interviews and saying that by 2026, these interviews will go away. I think by 2026, the LeetCode interviews are not going to be gone. This tweet did not age well.

The whole industry will catch up, and it will catch up sooner. Some big companies are doing. Shopify has been doing coding with AI, or interviews where AI is encouraged for a very long time now. That's one of the best companies which is doing it really well. Maybe Meta will be next. For now, they're still asking for diameters of binary trees, but maybe they will change someday. This battle of getting rid of the coding interviews will not be easy, because a lifetime of these processes around them have been built on top of very flawed assumptions. I actually think this tweet is fake. I don't think that's real, but I like it. Don't believe everything that's on the internet. The choice for companies that are administering LeetCode style interviews is simple. Either you adapt the interview to the age of AI, or you just face extinction, and nobody will want to work there.

## Redefining the Interview Loop, in the Age of AI

I talked to an executive recruiter recently at a company called People Project that I highly recommend. She said that all the companies are asking about how to make developers more productive. Productivity has become this main measure of what companies want to do. They want to increase productivity. The real differentiators in productivity is not the AI tools. It's really the human skills, things like empathy, things like ethics, collaboration, judgment. Those matter a lot more than the actual code that you can write, because the tools can write a lot of the code. We have to redefine the interview loop to be up-to-date to the age of AI. I think, very simply put, if you want to be an AI company, and I'm sure every single company you work for right now is actively wondering, how can we be an AI company? Start by hiring people that are capable of using AI, meaning, interview them with AI so you can see how they use it.

What better place to do this than a coding interview? Ask them to use AI. If you do it well, the interview process can yield what the candidate is really capable of. They have this incredible machine as a helper, and you want to see what they can do. Can they multiply their skills, the ones that are important, to produce something with the help of that machine? Let's reevaluate our system design interview. When we do system design, I showed you one example where you'd draw boxes on a whiteboard and tell, how would you design the system? In that 45-minute interview slot, you really don't have enough time to go from the design to the actual implementation of a system, because it's too long to write. Now, with AI, you can actually ask the candidate not to just design the system, but also build it by prompting AI to actually go and execute on it.

You can actually watch what the person is doing by building something hands-on. You can see what actually matters, how they break down problems, how they handle curveballs. Are they writing tests? Can they iterate when things are breaking? They have this partner that's really fast. Now you can do a system design question using AI, where the code is actually being written by the AI.

What problems do you ask today with this kind of tool available to you? Good problems are the same as they were before. They're the ones that are big enough to require clarifying questions. They have multiple possible solutions. You need some iteration, and it's not just dumb copy-paste. The goal is to reveal the skills of the individual, so have something fairly open-ended. It's just a normal interview, where you start a little vague, find a milestone, throw a complication, extend it, and so on. Some examples of the ones I like: autoscalers, rate limiters, data pipelines. It's pretty open-ended. For autoscaling, you can have problems like leader elections, stuck pods, if you are doing some Kubernetes operator type thing. The rate limiters can have questions like, how do you stop abuse in this distributed system in 50 regions? Ingesting events for a data pipeline. I really like AI Chat, because it's something people experience every day, like, build an AI Chat system yourself.

Just because AI is used, it doesn't matter. Don't change your entire process. The same interview features apply. You just want to see how the candidate is using these tools. If you're interviewing, see companies that use AI in their interviews. There's already a massive database of companies that don't do whiteboard coding. I put a link up there. Look at this list. The companies that don't do whiteboard coding, this should be the norm. We shouldn't need lists. It should be the default. I couldn't find a list of companies that encourage the use of AI. I made a new GitHub repo using Claude. I encourage you to come and submit your company to that list. If you are using AI in interviews, please come and add yourself to that list. Hopefully, it will grow.

The times, they're changing. The industry will catch up with this AI trend sooner than later. Then it will continue evolving, and we don't know what it will become. The roles that people have will evolve. Now I see designers vibe coding entire features as prototypes. They're no longer using just Figma. If you're stuck under some rock, and you're still not using AI, it's time to climb out of that rock. You'll say, "It's impossible. I work for a large company. I can't change the way my company is interviewing, because it's a huge undertaking. We have so much process built about it." Quote your CEO, and go to the recruiting people and say, the CEO said that every company is now an AI company. Are you not going to be an AI company? Are you not going to hire people that know how to use AI? In the interviews, you're literally working hard on preventing us from becoming an AI company. See what they say when that happens.

## Questions and Answers

**Participant 1:** [inaudible 00:43:58].

**Daniel Doubrovkine:** Yes, they're design questions with code. Like the code itself, I don't care. I want to see code you write maybe from a past life. We can talk about code, reason about code. I think that's important. Especially for senior engineers, forget about code. It's, design the system, implement it. I think that's the big difference today. With AI, yes. Maybe you need to tweak here and there, but I'm mostly interested about how you think. Can you get unstuck when the AI produces something that doesn't work?

**Participant 2:** What's your thought on the take-home project for some people?

**Daniel Doubrovkine:** The take-home projects is only to have something we can all talk about that you prepared. Not everybody works on open-source software out there. Maybe somebody has absolutely no history. Maybe they're straight out of college. This grounds it in a conversation for a problem that you can solve at home with AI or without AI that you understand thoroughly. We can talk about, is this code performant? Is this code interesting? Are there other ways to solve it? Not from memory, you can think about it at home with the tools that you have available.

**Participant 3:** You mentioned that coding AI challenge, that means different types of questions that you ask. How do you look at the assessment for those tests? Like you may have many managers who are doing those types of interviews? How do you make sure that your evaluation is consistent?

**Daniel Doubrovkine:** I work with some people who are looking into the rubrics and the evals of that. I think it's actually the same thing as from before. There are some red flags. There are some green flags. For example, some new ones, if you prompted an AI to write code and then you go and read the code, that's probably an antipattern, because you want to keep an abstraction layer away and actually work with systems. For example, write a test and then make sure the test passes. Rather than write code and verify as a human that it works. These types of things are new. Everything else is like, do you ask questions before you start implementing anything? Do you ask questions from your interviewer? Do you ask questions from the AI? What kind of questions do you ask? I think these are changing a little bit. Then the features are not changing very much.

**See more presentations with transcripts**