Early-Stage Hiring for Engineering Teams: Part II

In Part One, we spoke about the overall, general flow of an engineering interview loop that might be appropriate for early-stage companies. As we discussed in that post, early engineering hires will set the tone for the company's future engineering culture and this will have a permanent impact on the company’s ability to deliver on product objectives. It can also be hard, costly, and time-consuming to course-correct after making hiring decisions which turn out to be incorrect. On top of all of this, early-stage startup employees may spend more time with each other than they do with their own family during the early years of a company. For all of these reasons, it’s critical to approach engineering hiring with a rigorous and repeatable process that ensures quality and consistency. With that being said however, it’s also critical to be fair and respectful of the fact that candidates don’t have infinite time to spend interviewing with a company. If you haven't read Part One, we definitely recommend that you do so now before continuing here.

In the first article, we outlined the full interview loop but in this article, we'll dive in to the three technical interviews in more detail:

• Technical Screen / Technical Get-to-Know
• Systems Design / Table-Top Architecture Interview
• Coding Assessment

We'll talk about how to construct each session and what the objectives of each discussion should be. We'll aim to provide details on the structure of each interview and commentary on different ways to prepare for and approach the meeting.

Technical Screen / Technical Get-to-Know

During the technical screen, it's often a good idea to start the interview by explaining the general structure of the entire discussion. This is useful because it allows the candidate to fully understand the flow of the discussion. This also has the added benefit of illustrating to the candidate that there is a lot to cover in this session so one shouldn't spend too much time on intros (for example).

Start the interview by briefly explaining the context of this discussion and your technical background. The intent here isn't to impress the candidate or to spend too much time talking about yourself. Instead, the intent should be on being brief and focused so that the candidate is inspired to be similarly brief and focused when it's their turn to give their introduction and background overview. It's also helpful to give the candidate an overview of your technical depth so that they know how much technical detail to go into when explaining their ideas.

One should then invite the candidate to briefly go through a similar articulation of the candidate’s technical background and experiences. You may wish to hop in with questions at this stage if the candidate touches on topics that you aim to cover later in the interview such as technical preferences, experience with specifically relevant technical subject matter, leadership experience, etc. This is often productive because it makes the conversation feel very natural and helps to make their intro feel less like a monologue.

Once intros and backgrounds are out of the way, it’s useful to get a brief understanding of the candidate’s specific technical preferences. Since this discussion will cover some technical topics, it is helpful to get an understanding of the candidate’s preferences in terms of languages, infrastructure, and architecture so that the following questions can be grounded in technologies that the candidate is proficient with if necessary.

The interview can then progress to talking about some of the technical subject-matter of your company. I will then ask the candidate about 3-5 specific technical challenges that the company is facing and how they might go about solving those challenges. If the role that we're interviewing for is engineering hire number one, consider asking how they would use their previously articulated technology stack to solve these problems. If the candidate would be coming on to an existing team, consider inquiring how they would orient their solution inside of the existing architecture.

Switching gears, ask the candidate about their experience and philosophies around building and leading teams. The objective of assessment will depend on the scope of the role but it's important to gain insight into this topic because both senior managers and individual contributors must be able to participate in (and sometimes fully own) various aspects of team growth and leadership.

Finally, it’s useful to leave at least 5 minutes for the candidate to ask any questions that they have and/or explain anything about themselves that they feel is important that we didn't get to cover.

Systems Design / Table-Top Architecture Interview

The systems interview is often one of the most fun interviews for both the interviewer and the candidate but this interview can be a challenging session for the interviewer to moderate given the level of technical depth required to be effective. At Moonfire, I often help portfolio companies perform these interviews if there is no sufficiently senior infrastructure/systems engineer on the team. If your team doesn't have someone who can perform this interview, consider enlisting the help of an advisor or angel investor.

There are a number of great resources for both interviewers and candidates for these kinds of interviews. For candidates, consider the following:

• "System Design Interview – An insider's guide" by Alex Xu (Amazon)
• "System Design Interview Book Review: Finally, a Book for Getting Better at Architecting Systems" by Gergely Orosz (Link)
• "The System Design Primer" by Donne Martin (GitHub)

The above resources are valuable for interviewers as well. The interviewer should also consider the following resources:

• "How to conduct a system design interview" by Roberto Vitillo (Link)

The intent of this interview is to assess the candidate’s ability to concretely reason about complex architecture and infrastructure problems. As Robert Vitillo mentions in his blog article (linked above):

"By the end of the interview, you should have a good idea of the candidate’s strengths and weaknesses and how they approach problems that don’t have a single best answer. A senior candidate should be able to quickly go through the first part of the interview and have no issues scaling up the system and addressing failure modes in the second part. They should have a good explanation of every decision they took and be able to list the tradeoffs of their design. When interviewing strong candidates, you should leave the interview feeling like you learned something from them.

The system design interview is a great way to get a feeling of a senior candidate’s problem-solving skills. Connecting boxes with arrows is just one part of it, though - and not the most challenging one. The tricky part is understanding the requirements, failure modes, and tradeoffs. This type of interview also encourages lively discussions, which gives you a glimpse into the candidate’s soft skills. Can they explain things clearly? Do they receive constructive feedback well?"

For everyone involved, there are a number of excellent blogs on distributed systems infrastructure that are definitely worth reading. A few of my favourites are:

• https://martinfowler.com/
• https://robertovitillo.com/
• https://jepsen.io/

Coding Assessment

When hiring engineers, they will invariably spend most of their time engineering things. It's absolutely critical that you assess their hands-on ability to write software during the interview process.

Historically, coding skills used to be assessed via "whiteboard coding interviews". In a whiteboard interview, the candidate is given a coding problem and asked to write the solution on a whiteboard. For a variety of reasons, whiteboard interviews have fallen out of favour across the engineering industry (except perhaps at large companies like Facebook, Google, Microsoft, etc). I have personally delivered hundreds of whiteboard interviews throughout my career and I will acknowledge that, while these interviews provide very few false positives, they commonly produce false negatives. More explicitly - if a candidate passes a whiteboard interview, they are likely to be a strong programmer. But if a candidate doesn't pass a whiteboard interview, it's not necessarily true that they are a bad programmer. If your company can afford such a high degree of false negatives (as can FAANG companies due to high application rates), this may not be a problem. If your company struggles to find strong engineering talent (like basically every other company in the world), then perhaps it's worth considering a more inclusive approach.

To deal with the growing unpopularity of whiteboard interviews, many companies have shifted to using "take-home tests" to assess a candidate's coding skills. A take-home test might be a GitHub repo that a candidate has to clone which will have a bit of application scaffolding already set up. There are usually included instructions that guide the candidate's efforts. The candidate will usually be asked to fix a few bugs and/or implement a few features. In many ways, take-home tests are a huge improvement because they provide a more realistic assessment environment which allows the candidate to exhibit their skills more naturally.

The major downside of this approach is that take-home tests are typically extremely time-consuming for candidates. Candidates that are especially keen might lie about how much time they spend on these projects because they want to impress the company. Additionally, these projects often result in a lot of time spent on boilerplate / setting up libraries which is relatively simple work and time spent on it doesn't provide much signal. For these reasons, there may be better alternatives.

A potential alternative to whiteboard interviews and take-home tests is a timed assessment using a tool like HackerRank. HackerRank has a product feature where you can construct multi-question assessments which impose a limit on how much time a candidate can spend on each question. This may seem like it will induce stress but it's important to reiterate that it's critical to be respectful of the candidate's time during the hiring process. By imposing time limits on the assessment, you can ensure that there is no way that the candidate will be forced to spend a superfluous amount of time working on the assessment.

Another reason why HackerRank tests are great is because the candidate doesn't have to spend time setting up environments, libraries, or toolchains. The HackerRank web-based development experience is very robust. By taking advantage of an existing toolchain and developer environment, the candidate can focus on small, focused problems which assess the skills that need to be assessed. For example, when constructing an assessment for a backend engineer, one may select two SQL questions, two algorithms/coding questions, and one question which requires the candidate to use an external API. All of these questions are small and focused individually but, when put together, they provide a relatively comprehensive insight into a candidate's skills.

It's worth noting that even timed assessments have some downsides. Most of all, they can be stressful and they are still relatively time consuming for candidates that have busy work schedules, children, etc. Some people advocate that the best way forward is to allow the candidate to share a project from their portfolio. The company can then read it and discuss details about the project with the candidate. Personally, if a candidate has a sufficiently relevant portfolio project to share (or enough open source software contributions), I am more than happy to consider this option and I always offer it to candidates.

Conclusion

As we discussed in Part One, early engineering hires will set the tone for the company's future engineering culture which will have a permanent impact on the company’s ability to deliver on product objectives. On top of this, it is just as important that early-stage startup employees are a good cultural fit given the amount of time spent with colleagues during those critical early years of a company. As such, it’s critical to approach engineering hiring with a rigorous and repeatable process that ensures quality and consistency. We hope that these articles will help you design a rigorous and repeatable process for your company.

🌓🔥