Interview home coding tasks: my policy

In recent years there seems to have been a shift in interviews: less pairing exercises and more home coding tasks / projects.

After recent experience of home coding projects pre-interview, I’ve set a policy for my engagement in these tasks. I’m detailing it below, partly to avoid repeating myself to recruiters. Further down I give some reasons and discussion.

My home coding test policy

  • I expect transparency around the existence of a test, and its conditions, as early possible
  • Maximum one hour length for ‘fire and forget’ tests where I’m not guaranteed feedback or interview (and I expect that time limit to apply to everyone, otherwise, what’s the point?)
  • Longer tests are possible but they need a set length1 and to not be open-ended. I need guarantee of meaningful feedback and/or actual interview discussion after, depending on time I’m committing
  • Don’t ask me to design features or code relating to your actual product2

Discussion

There are alternatives to home code tests and we appear to be forgetting this. And I think they can be more informative in less time.

  • Viewing applicant’s open source projects
  • Have applicant talk about some of their open source code
  • Do a code review
  • Pairing session

The asymmetry of being asked to commit two or more hours before any interview (and without even getting any feedback or interview afterwards) is too much.

There’s also a question of fairness: longer home code tests significantly favour those with the disposable time to complete them.

Time creep

This is a dark pattern that is pretty insidious: you’re told how much time you should spend on the test (typically low-balled), but there is an unstated expectation that you spend more time. This is a red flag to me. Why would they punish applicants for following instructions? Maybe not a company you want to work with.

Removing time creep: a fairer test

  • candidate has N hours to do test in one sitting (strongly suggest N <= 2)
  • candidate elects a date and time to start the test
  • candidate is emailed a link to test instructions 5 minutes before the start
  • the candidate can push code changes to a provided repo as often as they want
  • the repo will refuse pushes when time is up (candidate is made aware of this)

War stories

I asked developer friends for their experiences around home coding tests:

“I once spent 4 hours on a ‘no more than three hours’ take home exercise. Left a list of all the things I hadn’t done but would have given the time, and a section discussing any potential controversial choices I’d made.

Guess what the only feedback I got was?”

“Last year I interviewed for a place that needed someone in a hurry. Like, it was Thursday, they wanted to interview on Monday. They said ‘We might give you some homework.’’ I said, ‘okay, I’ve got plans this weekend, but I’ll do what I can so long as there’s an understanding that I’m doing what I can in a short timeframe.’

Then, at 7pm on the Friday night, the recruiter drops a three-hour homework assignment in my inbox. Which I manage to mostly do on the Sunday in between other commitments, skipping only the last task, which was basically the same as the previous task. I figured ‘We can talk about my coding style and stuff in the interview, this seems redundant.’

The Monday interview went well, and then a couple of days later I get the rejection email. All of the feedback was about what I hadn’t done on the homework assignment.”

“I was given 4 days time in which to take a live-coding test on a website. The recruiter badgered me to take it ASAP rather than preparing for it. My test was horrible due to lack of prep (the website being awful didn’t help). I sent the recruiter my careful thoughts post-test, as they’d requested. Still waiting to hear back from that particular recruiter…”

“I’ve had a few coding tests where it’s perfectly clear that the job does not involve coding at all, but someone used this for another role once and so it’s showed up in some other role’s process.

Special mention for the company that put me through 11 rounds of interview including a take-home coding test and a live coding interview for a non-coding role, before deciding they didn’t have the budget to hire anyway.”

A notable lesson that has emerged is this: it doesn’t matter what assurances a company or recruiter gives you before the test, the company will judge all the tests in the same way: as if they had been done in ideal conditions. And it’s in recruiters’ best interests to get you to submit a test, even under non-ideal conditions.

A lot of recruiters are very happy for you to waste hours of your own time so they can have a (say) 2% chance of getting commission on a placement!

  1. and I really do mean a set length, not “this should take 0.5 - 1 day” and then you’re able to pour 3 days into it if you’re able and willing ↩︎

  2. I recently did a home test where I had to design and code a feature relating to the company’s product including comms, unit tests (they said to take 0.5 - 1 day, believe it or not, but of course they didn’t enforce that timescale because it’s in no way realistic). They actually paid people a little to do the test, regardless of outcome; a one-off in my experience. But they also committed most of the “sins” I’m detailing in this post ↩︎