Coding interviews don’t measure engineering ability

I’m fully aware that the title of this post is a hot take and also a clickbait.

Also, so far as I can tell from nearly two decades of experience in the industry and literally hundreds of interviews on both sides of the table, it’s true. Coding interviews don’t measure software engineering skills.

To explain this paradox, let me first clarify which parts of the typical recruitment process actually measure them.

The first and most important one is the CV screening.

The reason why it’s so important is that employers are not interested… Look, I can’t speak for 100% companies on this planet, but a typical employer is not particularly interested in some ding-an-sich spherical-in-vacuum engineering prowess. They’re interested in getting a steady pair of hands who can get shit done using the technologies they like and within the organisational structure they have.

And by far the best proof of a candidate’s ability to get shit done is their success track at having done similar shit in the past elsewhere.

Yes, it’s that simple. If I’m looking for a Python backend dev, and you have several years of experience building FastAPI webservices in various scaleups, then I’m reasonably confident you can do it in my scaleup too.

Otherwise… Well, it depends. If you’ve been doing Node.js at scaleups, that one thing; if you’ve been doing COBOL at a government agency, that’s different. If I’m cool with a well-rounded generalist, that’s one thing; if I specifically seek a tech stack expert, that’s different. If you’re the only decent candidate on my radar, that’s one thing; if I’ve a long queue of those, that’s different.

But either way, hiring a person with less than 100% match experience-wise is a gamble for me, and therefore, it’s an uphill battle for you. All else being equal, I’d always take a “done it before, can do it again” over an “I’m sure I can figure it out” one. And so would virtually every employer out there.

Yes, this sucks balls, I can’t agree more. Everybody is so fuckin’ risk-averse, and that’s why entering the industry is tough, and switching between different segments of the industry isn’t much better.

It is what it is. Nonetheless, here are a few things you can do to help your cause.

• Adjust your CV to align with job requirements. Say, you’re applying for a job as a 50/50 full-stack, and your experience was mainly as a 75/25 backend-leaning full-stack. In this case, it would make sense to embellish your frontend achievements and trim down the less important ones on the backend. Yes, it’s cringey. Yes, it’s annoying. Yes, it’s borderline dishonest. And yes, it can be a decisive factor in whether you get a new job or an “we decide to move forward with other candidates” email. Also, you don’t really have to make a version for every single application: a handful of versions for different job descriptions would suffice.
• If you’re trying to reboot your career after an extended period of time driving trucks for Doctors Without Borders, designing cowsheds, directing porn, or all of the above, make a portfolio of software hobby projects and put them on GitHub, where everyone can see them. It’s not as good as the commercial experience, but it’s definitely better than nothing.
• If you’re a fresh graduate, list your coursework projects on your CV. Again, it’s not as good as the commercial experience, but it’s better than nothing. If you have other hobby projects worth mentioning, mention them too.

In principle, in the ideal world, this would be the entire job application process. The hiring team gets your CV (and/or diploma and/or portfolio) and evaluates your hard skills based on those. Then they evaluate the soft skills and the personality fit based on your motivation letter. Then they either reject you or make you a job offer straight away, without the need for all that interview theatre.

Of course, this doesn’t work in the real world, and there are (at least) three reasons why.

Problem #1 is that the candidate might be overselling their achievements on the macro level.

Say, your CV says you built a “transcontinental general-purpose data synchronisation platform,” but what exactly does it mean? A cluster of Kafka brokers to send data between fault-tolerant data storages? A nightly cronjob to rsync files between an EC2 instance in us-east-1 and one in eu-west-2? A cluster of Kafka brokers in the context where a nightly cronjob would perfectly suffice? I don’t know, and to figure that out, I might arrange a project deep-dive interview where I’d ask you to walk me through the nitty-gritty details of something you’ve built, and the reasoning behind those nitty-gritty details, and all that jazz.

Note that not every company does it. Some believe that whatever you say on your CV is largely accurate and mostly true.

Problem #2 is that the CV doesn’t say anything about the micro level, i.e. the code quality and overall discipline.

Like, okay, we had a project deep-dive, and I’m convinced you can ship a mega-complex mission-critical project ahead of schedule.

But what should I expect to see under the hood of such work?

Decently clean and maintainable code? A cobweb of hacks, copy-pasta, and programming by coincidence that would be one huge liability for whoever takes it over? A cabbage of abstraction layers on top of abstraction layers, where one is expected to spend a whole week on writing DecoratedBuilderFactoryVisitor boilerplate before doing anything useful?

To figure this out, I’d like you to do a take-home assignment. It would be a coding problem that vaguely resembles what we do in my team, and it would be somewhere between 4 and 20 man-hours worth of work. You can use whatever tools you like and any documentation you need, and in the end, I’d like to see the code that you’d consider a merge request quality level. Afterwards, we’ll meet (virtually or in-person) to discuss your decisions, etc.

Note that not every company does this step either. Some believe that if you’ve been a paid developer for several years, then you can write decent code, or at least they can make you do it.

Problem #3 is that back in the days, motivation letters were unreliable. Sure, you could use them to filter out the clear outliers. Such as spammers who couldn’t be bothered to spend twenty minutes researching your company. Or “proud to be an arsehole” types who would say “hahaha, you’re such losers to require these stupid motivation letters.” But that was pretty much it. If you’re confident that you can tell the genuine passion from the competent bullshit, or calm professionalism from generic copy-pasta, either you’re a psychic or you’re delusional.

And, mind you, that was back in the days. With the emergence of generative AI, motivation letters lost any remaining semblance of meaning. Now you can just tell the computer to generate four paragraphs of slop for you, and that would fit the purpose better than anything you’d put together yourself.

And that’s why every remotely sane hiring process includes steps to validate the personality fit.

In fact, aside from the CV screening, the project deep-dive and the take-home assignment, everything else in the process of getting hired as a software engineer is a personality fit interview.

I repeat. Everything else is a personality fit interview.

Moreover, you can reverse-engineer their culture from their process.

Yes, this also applies to the coding interviews. I know it’s a tad counterintuitive, but bear with me. Coding interviews seem superficially technical because, well, you’re doing some technical stuff, namely writing some code. But, as a matter of fact, those interviews are pretty bad for evaluating the actual skills, simply because live coding is too different from regular programming work, and whiteboard coding is even worse.

At the same time, coding interviews are very convenient to put a candidate in a mildly challenging situation, while maintaining a somewhat level playing field, and also not looking too weird.

It’s a bit like, imagine you’re hiring a racecar driver, and you ask her to drive you around the city centre during the rush hours. It won’t give you much insight into her racecraft, but you’d get a better idea of who she is as a person.

The hiring team can also drag the candidate to a pub and get them drunk and chatty, or take them to a gym and make them do deadlifts, and learn about their personality that way. But then, different people have radically different attitudes towards alcohol and sports. So it all gets very messy and complicated, and to avoid those complications, the standard approach is to subject the candidates to the same sort-of-work-like routine.

For instance, one popular way to do coding interviews is they give you a log file (or, sometimes, an HTTP endpoint) and ask you to extract some interesting information from it. Technically, this isn’t a difficult task, so it doesn’t tell much about your engineering ability. Personality-wise, it says a whole lot. Some people will jump right into it, hack something together off the incomplete requirements, and iterate from there. Some will demand an exhaustive format specification and approval from the architecture steering committee, and otherwise it’s not “proper software engineering.”

In some environments (in particular, early-stage startups), the former is strongly preferred over the latter. So, if you have an interview with a startup, and they hit you with such a puzzle, you can make a note for yourself that “aha, these folks are moving fast and are conscious about it,” and then use this piece of knowledge to decide if this is the right place for you.

Another common way to do a coding interview is they hit you with an algorithmic puzzle (one that you don’t know from the top of your head, that’s important!) and watch how you improvise under pressure. I did a lot of that as an interviewer when I worked at various electronic trading firms, and that made perfect sense: in a volatile environment like electronic trading, you have to improvise under pressure quite regularly. If you don’t know how to transpose a non-binary tree, that’s perfectly fine, it’s not something we do here anyway. If, however, you can’t handle the stress of figuring that out… Well, that’s unfortunate.

Then again, it’s also a datapoint for you to consider if you want to work in a company like that or not.

Many hiring teams (especially the ones that have take-home assignments) would just skip the dedicated coding interview altogether and evaluate the personal fit by other means.

Oh, and some companies ask irrelevant bullshit. That’s because they want to make sure you’re desperate.

Let me go on a tangent here. The younger folks might not realise what a game-changer Stack Overflow was when it appeared in 2009.

Say, you’re writing some code, and you get stuck with a third-party library not doing what you want, and it’s 2008. What do you do? Well, first you go and read the bloody manual. Occasionally, you’re lucky and the manual covers your case. Oftentimes, the documentation is patchy or non-existent. So, you google for this issue, and then sift through the forums and mailing list archives. Occasionally, you manage to piece together a solution from the morsels of knowledge you’ve found. Oftentimes, you end up reading the source code of that third-party library, GOD BLESS FREE SOFTWARE.

Then Stack Overflow appeared, and we got an authoritative place for all kinds of “well fuck, what do I do next?” programming-related questions, and that was massive.

A couple of years prior to that, while I was still in the uni studying electronics engineering, Wikipedia became a thing. So that, if you wanted a rough idea of what gallium arsenide is all about, you could just open a webpage and read it. Compared to digging through the specialised websites, and digging through the physical books, and driving thirty minutes to the physical library to get those books, this was mind-blowing.

And over the last few years, we’ve got a flurry of LLM-based tools which are pulling towards the same direction of making information even more findable and accessible.

As technology progresses, the skill of knowing stuff is becoming increasingly irrelevant, especially when it comes to general-purpose publicly available information. It doesn’t matter how kind you are, German children are Kinder. It doesn’t matter how much you know, the Internet knows more.

The skills that become increasingly crucial in the world we live in are the abilities to find information, to contextualise it, to filter out the inaccuracies, and then to put that knowledge to practical use.

So, the point of this whole tangent is that making hiring decisions based on candidates’ knowledge of trivia questions is dumb and disrespectful, and it’s also very much a personal fit filter.

Do you need this job badly enough to drill the leetcode puzzles?

Do you need this job badly enough to rehearse that stand-up comedy routine where you design OnlyFans on a scrap of paper?

Do you need this job badly enough to memorise the subtle details of Dijkstra’s algorithm just in case?

Do you need this job badly enough to sheepishly agree when an interviewer says, “Hahaha, you didn’t use walrus operator in line 19, you’re so stupid,” instead of the well-deserved, “Child. I have about twenty years of experience with Python. You have about twenty years of experience taking off pants when shitting. Shut the fuck up.”

If yes, it’s safe to assume that you’ll also accept being disrespected after they hire, that you’ll toe the line and won’t ask inconvenient questions. Otherwise, we decided to proceed with other candidates.

Look, I know this is a controversial take, and I already see the flames of the burning buttholes on the horizon.

However, the fact of the objective reality is that when a company gears their recruitment process towards hiring people who are good at solving leetcode puzzles, fantasising about distributed architectures and dropping buzzwords, they end up with a lot of people who are good at exactly that.

Well, I mean, like… If their typical working day consists mainly of whiteboard coding and dropping buzzwords, I must admit that’s very unlike the jobs I had, but fair play to them. Otherwise, I think they’ve got questions to answer, at least to themselves.

Oh, and if you enjoy my style of shitposting, I’m sure you’ll also like my new book. It’s called “Spiritual Anarchist Handbook” and it’s available on Amazon.