Designing Your Startup’s Interview Process

We built our founding team at Accompany by recruiting from within our personal networks. Hiring close friends and colleagues ensured everyone we added to the team was phenomenal, and it let us rely upon our past experience working together in place of a formal interview process. As we started to scale our team, though, we had to expand beyond our personal networks and source candidates we didn’t already know. This meant we had to design an interview process that worked just as effectively.

To do so, we approached interviewing very deliberately. We treated our interview process like it was another product we were building — one to be designed and iterated on. This consumed a lot of our team’s creative energy, but it paid to get this right. In the end, we designed an interview process that candidates enjoyed and that consistently led to hiring great engineers. ¹

In this post, I’ll present how we designed each step of our interview process and why. Every startup is unique, and their interview process should be too. Still, my hope is that a lot of what we found can serve as a useful starting point for founders designing their own interview process from scratch.

We began with a thirty minute introductory chat with the candidate. This was a chance for both sides to determine if we wanted to commit to interviewing.

We asked ourselves: “If this candidate were to pass all of our technical interviews, would we want them on the team?” If we wouldn’t, there was no point in proceeding, no matter how good at programming they might be. To determine this, we asked the candidate about their favorite past projects and what they were seeking in their new role. This quickly identified candidates who would be a great fit – friendly and optimistic hackers at heart, who prized a high degree of autonomy and responsibility, who wanted to make stuff that people actually use, and get equity for it. The kinds of people who would make great startup founders themselves. It also quickly identified mismatches, like candidates who either just weren’t very nice, prioritized more work-life balance than we could offer, or (whether they realized it or not) wanted to join a company much larger than ours. ²

We tried to help the candidate answer the same question for themself by telling them about our team, product, and ambitions. This introductory chat was always a blend of assessing if the candidate would be a good fit for us and pitching how our startup could be a good fit for them. Contextualizing the role to the specific candidate — by exploring where their prior experience could prove useful, discussing how our upcoming projects could meet their interests and goals, and imagining the impact they could have for our customers — enabled us to naturally do both at once.

We also used this call to check basic things like start date, employment status, location, and compensation ranges. We didn’t want to go through the entire interview process to only then discover the candidate was seeking a salary twice what we could offer.

We told candidates at the end of the call if we’d like to move forward. With practice, we were able to accurately determine this on the spot. Doing so pleasantly surprised candidates and helped build momentum. If we weren’t going to move forward, we found it was smoother to just let the candidate know the next day over email.

I’ve noticed founders can be tempted to either skip this introductory chat entirely or outsource it to a recruiter. This is a mistake. Nobody can tell if a candidate would make for a great addition to the team better than the founders. And the candidates you ultimately hire are only as good as the ones you decided to interview, so there’s a huge return on spending the time up front to get this right. ³

The other benefit is that candidates are more likely to decide to interview if they get to learn about the company directly from one of the founders. And in the early days, when your startup is still unknown, you have to do whatever you can to stand out. If done right, this introductory chat is a chance to convey to the candidate that you are keenly interested in them, their goals, and the contributions they could make at your company. You’re building a relationship with the candidate over the entire course of their interviewing with you; the sooner you start, the stronger it will be by the time they’re deciding if they want to join.

After the introductory chat, we conducted two thirty minute technical phone screens. ⁴ The goal of these calls was to determine if we should invite the candidate onsite for a full day of interviews.

We found that a single thirty minute call wasn’t long enough to accurately assess the candidate’s abilities and yielded too many false positives. But a single sixty minute call often resulted in us getting stuck spending an hour with a candidate that we could tell wasn’t a good fit a few minutes in. So two thirty minute calls ended up working best, and had the added bonus that two different interviewers could vet the candidate before we brought them onsite.

On each call, we’d spend five minutes getting to know each other, twenty minutes working on a programming problem, and then five minutes debriefing with the candidate on their solution. We had the candidate code up their solution in a collaborative editor in the browser or just by screen sharing their desktop. We had them actually run their code, and we provided test cases for them to work against. This helped remove some of the superficial dynamics that can creep into interviewing and let the candidate work more naturally.

We were surprised to find that easy questions told us more about the candidate’s abilities than hard ones. In only twenty minutes, good candidates could nail an easy question, and we’d get to see how they designed, implemented, and debugged their solution. When we tried hard questions, candidates had to spend too much time just figuring out how to even solve it, and we wouldn’t cover enough ground to determine if we should invite them onsite.

If the phone screens went well, we’d invite the candidate to come in for a full day of interviewing at our office. Initially, we fell victim to the mistaken belief that we should share very little information about what to expect. We thought it was important to preserve secrecy; why give away what’s going to be on the test? This was a huge mistake. It made candidates nervous — like they were entering into some mystical rite of passage — and left them unprepared.

Over time, we learned to invert this and became extremely transparent, arming candidates with as much information as we could to set them up for success.⁵

Telling candidates what qualities we were looking for was especially helpful. Don’t expect them to magically know what you value. Tell them in advance, and if you notice them falling short on it during the interview, coach them to see how they adapt. Once we started doing this, candidates had a better sense of what to prioritize and performed better.

We began our onsite interviews with lunch because none of us could reliably manage to get in much earlier than that. At first we felt a bit guilty about this, but it ended up being ideal, because it gave the candidate a chance to meet the entire team and work out any nerves before diving into interviewing.

Every startup is unique and their interviews should be too. Our product required strong software architecture and algorithms abilities, so we designed our interviews to assess that. Other startups should design theirs accordingly. A less algorithms heavy startup might benefit from replacing the algorithms interview with a debugging exercise or a second coding session.

Interviewing will always be an approximation of what it’s actually like to work in the role. Our aim was to make it as close of an approximation as possible. We needed to figure out how good the candidate would be at the work they would actually do if they joined the team, and not just how good they were at interviewing.

We actively interviewed. We weren’t on Slack, clearing out messages while we waited till the time was up to see if the candidate had coded up the right answer. Instead, we engaged fully with the candidate, observing how they think and approach their work.⁷ We realized this mattered more than if they came up with a completely correct answer. There were rarely circumstances on the actual job (production outages aside) where we had to perfectly solve something in under an hour. But we always needed to work productively. Actively interviewing let us assess if the candidate did.

It was helpful to ask the candidate to reflect on their work at the end. What were they happy with? What did they wish was better? If they had another hour, what would they do next? We treated this like an informal code review, and would share what we liked and wished were better too. This told us a ton about what the candidate valued and how receptive they were to feedback. We gained a lot of confidence in making offers to candidates who didn’t come up with a perfect solution if we were able to discern that they saw its shortcomings and had good ideas about how to address them.

Sometimes candidates really struggled at certain points in our interviews. If it became clear they were completely stuck, we’d make a mental note and shift from testing to teaching. This was just the nice thing to do, and it also enabled us to move onto the next question, which a candidate might do better at. It’s important to help candidates preserve their self-confidence along the way. Nobody does great work when they’re feeling demoralized.

After a long stretch of rejecting nearly every candidate we interviewed, we had to take a step back and remind ourselves that the goal of interviewing was to hire great people onto the team. Since we weren’t doing that, we needed to improve our interview process until we were. This may seem obvious, but it was surprisingly easy for us to lapse into assuming that the reason we weren’t making any offers was that the candidates just weren’t good enough, when in fact it was actually our interview process that wasn’t good enough. Over time, we improved how we sourced candidates, changed the questions we asked and how we asked them, and refined our criteria for determining if a candidate had passed our interviews.

Making good decisions required rigor. The mechanics mattered. We made each interviewer write up their feedback right away while their memory was still fresh. Everyone followed the same template, noting the candidate’s strengths, weaknesses, and if we should hire them. Each interviewer’s decision came in the form of: Strong No, No, Yes, Strong Yes. ⁸ The same evening as the onsite, all the interviewers would get together (including whomever conducted the introductory chat and phone screens) and share their feedback with each other. We’d discuss, debate, and decide.

We didn’t have a fixed rule treating Strong No’s as vetoes; we just discussed why the interviewer gave that vote. But usually, if a single interviewer had strong objections, that would be it. Or if all the Yes votes were only lukewarm, we still might not hire. Having all interviewers in a room made this a more consensus driven process rather than just tabulating the votes and applying some formula.

Ultimately, our decision came down to if we believed the candidate would be effective in the role we were hiring for and if the company would be stronger for having them on the team. However they had performed in the interviews only mattered insofar as it answered this question. For example, we’d hire candidates that didn’t quite perform at the level we were hoping for if we could tell that they were determined and were fast learners.

Speed was one of our competitive advantages, so we let the candidate know the next day if we would be making an offer.

We didn’t always give the actual offer right away though. Instead, we asked candidates when they’d be ready to decide by (which was typically whenever they finished interviewing with other companies) and then gave them our offer a week prior to that. This let us avoid giving an offer without a deadline — we didn’t want candidates to shop it around, or to have to rescind it if we ended up filling the role with someone else — without having to give an “exploding” offer that forced the candidate to decide before they were ready. Once a candidate had all the pieces on the board, a week tended to be plenty of time for them to decide.

When we gave candidates the good news, we also shared a quote from each interviewer capturing something that had really impressed them during their time with the candidate. This showed candidates that we were genuinely excited to have them on the team and what the camaraderie would be like if they joined.

If a candidate didn’t pass our interviews, we let them know via email and offered to do a phone call to debrief. Most companies don’t do this, and it meant a lot to candidates that we did. It just seemed like the right thing to do — the candidate spent all day interviewing with us; the least we could do was spend half an hour on the phone giving them feedback. But we discovered it was a great way for us to get feedback too. When candidates agreed with our remarks, we knew our interview process was working well, and when they disagreed, it often identified things that we could improve. Following through like this left such a positive impression with candidates who didn’t pass our interviews that they would often later refer friends who did.

Once we made an offer, we kept in close touch with the candidate over the week that they had to decide. Different members of our team emailed the candidate to share why they were excited to work with them and to offer to chat. The hiring manager (namely, me) would schedule a call to check in and see how the candidate was leaning. We found that encouraging open communication let the candidate share any concerns, and if needed, let us adjust our offer to ensure we were competitive. It also provided an opportunity to share our own perspective on how to think about the decision, which new grads found especially helpful.

Some candidates wanted to dive deeper into the company because it’s always too rushed to fully do so during the interviews. We’d invite them to swing by our office or hop on a call so that we could further demo our app, share more about our roadmap, tell them about our customers, and even have them meet some more people from our team, advisory board, or customer base.

We were careful to never pressure candidates to join and to always respect what was best for them. This was less stressful for everyone, and things ultimately wouldn’t work out well if we forced it anyways.

Interviewing was a tremendous amount of work, but it was only the start. To make all that work pay off, we then had to successfully onboard the new hire and retain them over the long run. Thus began the great craft of management. Since interviewing and managing are such a full time investment, it pays to put it off for as long as possible, at least until you reach product market fit. Once you do, you’ll have the fundamentals of your startup in place, and recruiting and retaining will become easier anyways.

But when the time is right for your startup, it’s worth intentionally designing your interview process like this. The greatest joy of working at our startup was our team, and none of what we accomplished would have been possible without it. So go forth and build the best one you possibly can.

I’m only going to be covering how we interviewed engineers, although many of the ideas will be applicable to other roles. Early stage software startups should mostly be hiring engineers, though, if they’re hiring at all. The founders should do the rest. ↩
Candidates who hadn’t worked at early stage startups before would often be seeking things that only make sense at larger companies, like specializing in a single deep technical problem for multiple years straight, building out a team of engineers underneath them upon joining, or frequenting various conferences. ↩
Eventually the Head of Engineering, and then after that the hiring managers for their respective teams, can conduct the introductory chat. Here I’m talking about before the team has scaled to the size that those roles exist. And arguably, if you’ve hired the right kind of people along the way, those engineering leaders will be like founders in their own right. ↩
In its episode on hiring, the excellent Metamuse podcast points out how “phone screen” is a bad name because it implies that the goal of the call is to filter out bad candidates instead of to attract great ones. This emphasis on filtering comes from the few big tech companies like Google who were flooded with so many candidates that their primary problem actually was screening out bad candidates. But most startups have the opposite problem: attracting great ones. I wish we’d realized this and called this stage something else, like “Technical Skills Fit”.

In general, a lot of Silicon Valley startups’ interview processes have been cargo culted from big companies like Google, and it really pays to challenge assumptions, think from first principles, and do things differently whenever it makes sense. Whenever we did, we got better signal and candidates appreciated the authentic reflection of what we were all about. ↩
This was an instance where treating our interview process like it was just another product we were building paid off. We just asked ourselves what we would want if we were the ones coming in to interview, and the answer was clear: information about what to expect. In general, asking what interview process we ourselves would enjoy going through helped us improve it for candidates. ↩
At one point, we had four interview segments, but we found this was too draining for candidates. ↩
To ensure everyone on our team interviewed with the same level of care, we had new interviewers shadow seasoned ones until they felt comfortable conducting their own. Once they did, we reversed it and had the seasoned interviewer shadow the new interviewer until they no longer had any feedback. ↩
“Maybe” wasn’t an option, and if an interviewer ever was unsure, we diagnosed what we needed to do differently next time to ensure they got enough signal to decide. ↩