Until recently I prided myself on having a reputation as a tough job interviewer.  I've espoused the importance of design problems and case interviews (a style thoroughly ingrained in me while at BCG) as these questions cannot be completely prepared for and so I believed they truly tested the candidate.  However, when I got tripped up on a very casual case-like question at a dinner party last week, I began to wonder if this was a good method and what in fact was the best interview technique.

In my first year at Stanford, I interviewed for three well known management consulting firms: BCG, Bain and McKinsey.  I got knocked out in the first round for the first two firms and survived to the second round with McKinsey only to flunk a bizarre group interview where I'm positive we were watched through a one-way mirror.

By second year I was determined land this dream job and practiced case interviews for a few hours.  This time I got dinged in the first round by McKinsey but excelled in the other two, landing offers from both.  Since consulting firms hire for intelligence & aptitude first (rather than experience or direct knowledge) my abilities here couldn't possibly have changed in one year and yet the outcome was diametrically opposite.  Clearly the case interview method is inconsistent in identifying talent.

There are several other lauded interview styles including math/programming tests, brain teasers (very popular with tech companies) and even unusual practices like Google's Sergey Brin's "talk to me about anything you know well for 5 min"; which, by the way, he took from someone else as I was asked that very question in a Microsoft interview in 1994.  (I flunked that interview... damn mountain bike topic I chose...but I got an offer from Microsoft the next year for the same Program Manager position; yet another inconsistency).

In sharp contrast to such greuelling interviews were my sessions with the founders of HubSpot - Brian Halligan and Dharmesh Shah.  Both emphasize "hiring athletes" and neither gave me a case interview.  Indeed I thought the discussions were very interesting and fun but hardly the nerve-racking session I was expecting.  While I did get the third degree from other a couple other interviewees I now realize that Brian and Dharmesh were able to see the athlete in me without needing to grill me with standard methods.  

This is where experience counts and why a tenured individual can often make a highly accurate judgement call on a candidate's abilities after just a few minutes of chatting with him/her.  There's a common finding that managers often decide in the first 10 minutes of an interview if they'll hire someone.  While I used to eschew such snap judgements in the past I'm realizing they may have merit with tenured managers.

So, what is the best interviewing method?  This depends on the interviewee's experience hiring/managing/firing people and if they are hiring more for aptitude or a specific skill set that can be consistently tested for.  Design problems and cases are useful but (a) give your interviewee some warning of what to expect so they can prepare mentally (b) combine this with other styles to give the candidate every chance to shine.

If you're the prospective interviewee, find out what interview style to expect and practice hard as every system can be beaten.  Interviewing isn't likely to get consistent anytime soon so learn to beat the game.  Then show off your best stuff in the more realistic setting.