Image description

I try to get out to our global offices as often as I can. There's something about being in the room that you just can't replicate over a screen. Last week in Hyderabad, an individual asked this really important question: How should we rethink engineering interviews in the age of AI?

I was happy with what I shared in the moment, but I kept turning it over on the flight back home. Nineteen hours is a lot of time to think...

...so what should we be testing?

There are four areas that still deeply reveal how someone thinks as an engineer, and they all share one thing in common: they're hard to shortcut with a prompt.

1. System Design: Think Big, Think Structurally

Most candidates are thinking in terms of prompts. That's fine for individual tasks, but it doesn't tell you whether someone can reason about a system as a whole.

Idea: Give candidates a large, open-ended design problem. It doesn't even have to be a classic software problem. One of my favorites is an elevator load-balancing problem (yes, I've been SimTower-pilled since I was a kid) because it forces you to think about state, concurrency, optimization trade-offs, and user behavior all at once. No single prompt gets you there. You have to think.

2. Debugging: The Skill AI Can't Fake

Most candidates are probably engineering prompts and then editing the output. That workflow can produce code, but it doesn't mean the person understands what happened when something goes wrong. And things always go wrong.

Idea: Give candidates a buggy piece of software. Watch them identify the bug, reason about the fix, and implement it. This is where you see the real engineering instinct: the ability to read code you didn't write, build a mental model of what it's supposed to do, and zero in on where reality diverges from intent. Debugging is diagnosis, and diagnosis is one of the highest-signal skills in engineering.

3. Testing: Where the Rigor Lives

AI isn't great a generated test code...yet. It can scaffold some basics, but meaningful test coverage requires understanding the system's boundaries, edge cases, and failure modes. Things that demand genuine comprehension of what the software is supposed to do and what happens when it doesn't.

Idea: Give candidates an existing system and ask them to write tests. What do they prioritize? Where do they see risk? How do they think about the relationship between tests and confidence? This tells you so much about how someone thinks about quality, and quality is ultimately what separates software that works from software that survives.

4. Code Assessment: Reading Is as Important as Writing

AI generates a lot of code. Some of it good. Some of it slop. The ability to look at a body of code and assess its quality, to figure out what it's doing, where the logical failures are, where the abstractions leak, is a critical skill that's only getting more important as AI-generated code proliferates in codebases everywhere.

Idea: Generate a bunch of code and put it in front of the candidate. Ask them to figure out what it does, identify the slop, and find the logical failures. You're testing comprehension, taste, and critical thinking. Three things that are extremely hard to outsource to current models.

For High-Impact Roles: Embed Them in the Team

For certain roles, the four areas above may not be enough. The biggest hiring misses I've seen aren't about engineering intelligence. They're about emotional intelligence: the soft skills of actually working on a team. You can ace every technical exercise and still be someone who derails stand-ups, talks over junior engineers, or can't incorporate feedback.

For high-impact hires, after the earlier rounds, consider a pair programming session or even a short embed with the team they'd be joining. Give them a real problem the team is working on and let them collaborate. You'll see things that no solo exercise can surface. How do they communicate their thinking? Do they listen, or just wait for their turn to talk? Can they build on someone else's idea, or do they need to be the smartest person in the room? How do they handle disagreement?

This takes more time and coordination, so it's not practical for every role or every round. But for the hires that really matter, the ones that set the tone for a team, it's some of the highest-signal time you can spend.

The Bigger Picture

The goal isn't to make interviews harder. It's to make them higher signal.

AI has collapsed the value of "can you write code that compiles" as an interview signal to near-zero. That's fine. It was always a proxy for what we actually cared about: Can this person think? Can they build? Can they diagnose? Can they tell good work from bad?

Those questions haven't changed.

We just need interviews that actually answer them.