How to really interview good PM candidates - A Primer

Big thank you to @luckiesharms for her helpful comments and support!

I have now interviewed as a candidate at over 50 companies — FAANG, leadership roles for CPO positions, PMs, Group PMs at startups at all levels. I have also hired really good PM candidates across the board to build my teams in my current and previous role. In my past articles, I have talked about hiring for PM or PM leadership roles depending on the size or the stage of the company. In this article, I want to give real examples of sample questions that can help you test for the right type of attributes.

There are three types of typical PM interviews:

1. Probing into past experiences aka behavioral interviews — Amazon, Netflix base their interviews on this type of interview style

2. Testing for abilities — example case study interviews, whiteboarding a problem, walking through improving an existing product etc.

3. And a mish-mash between the two. Some companies interview for everything. They probe into past experiences, have a hurdle or two of assignments, an interview loop with their leadership team, an interview loop with their PM team, and so on.

I encourage you to avoid falling into the trap of a long drawn process of a written prompt, followed by a probe of the prompt by a panel, followed by multiple rounds of interviews with product, engineering, design, several levels of leadership team and finally two rounds of presentations. More data points does not have increased marginal returns.

Identify what you need.

First step in hiring a good candidate is to identify very clearly what you are looking for. Write down 3–5 bullets on what makes a great candidate for the PM candidate that you are looking to hire. What are your 3 must haves?

examples could be:

1. Technical skills — must have data science background, must understand machine learning

2. Must have 0–1 thinking

3. Must be able to deal with cross functional ambiguity

4. Must be a customer facing PM

5. Must have hustle

6. Must be able to write great JIRA stories

Having great clarity in your 3–5 must haves is amazing, it will really help you distinguish between a good candidate and a good fit. Talk to your team about what they think is needed in terms of skill-set.

As a hiring manager, have the first conversation

Ignore the noise, have a quick 30–45 min conversation to assess how you feel about the person. Your recruiters can sift for resumes, but as the hiring manager, forming your own impression about the candidate can help you from wasting cycles and cycles of time. If you have never hired a PM before, find someone in your network whose judgement you trust and let them help you think through what your must-haves are.

An important and handy tip is while you have well-meaning recruiters, board members, friends who may all be competent human beings, it does not qualify them to be good hirers. Hiring is an art form that if done well, should leave you and the candidates feeling excited about each other.

Hire for attributes, do not be swayed by resumes

Create a very simple interview process to test for skills.

Some skills I can think of which most good PM candidates should have —

Visioning

Execution

Structured Thinking

Leadership

Hustle

One of my favorite interview questions is to design a billion dollar solution for senior care. Here’s why I like this question. I am testing for the candidate’s ability to take an unconstrained problem and put constraints around it. I want them to ask me questions on what kind of constraints can they use to curtail the problem and define some scope. I want them to demonstrate customer empathy, in lieu of a market research team, good PMs have to rely on a mix of good researching abilities and their own intuition based on their experiences. Everyone is familiar with seniors, they have interfaced with some, or have family in that age bracket. Lastly, I want them to demonstrate structured thinking in arriving at various solutions and decision making. Demonstrate their ability to think about platforms, forward thinking technologies, and user experiences.

Create a great candidate experience by providing clarity on your process and pick interviewers in your company who can adequately test for each skill. Remember not everyone is a great interviewer, coach interviewers by offering them a consistent framework. Every candidate experience is representative of your company and culture. Develop a question bank of common interview questions, have an interview format — 5 min of intro, 30 min of questions, 5 min for Q and A. Keep the experience consistent.

The Take Home Assignment

I loathe take-home assignments, especially elaborate ones. It’s inherently unfair to ask a candidate to do a take-home assignment which will take 40–60 hours of prep, research, and writing time. If you are hiring for PMs early in their careers (3–5 years of experience), a take home assignment may make sense.

Have the take home assignment written up. Give an example of a good take home assignment on your site so candidates have clear expectations on what level of detail you are looking for.  Avoid having the prompt to make the assignment look like “free work”. Pick a prompt that doesnt require the candidate to do something for your company that still has to be designed/done. Examples of take home assignment prompts: Design a new service for increasing our revenue goals. How would you improve our startup? Present your favorite product that you are proud of. A take home assignment can help you test for a couple of things — their resourcefulness, their detail oriented-ness and written and oral communication abilities. Be clear on what you are testing for in your take home assignment or presentation. A good take home assignment should not take more than 10 hours of a candidate’s time. Remember the take-home assignment is not there to satisfy your smugness, it’s unfairly skewed to your favor if you are asking candidates to research your company, your competition and come up with ideas. Try to reduce information asymmetry as much as possible.

If you are hiring for senior PMs, avoid a take-home assignment. Chances are good PM candidates find roles through their networks, probably already have time consuming jobs and on the rare occasion that you were able to filter for a few good candidates, try to make the interview process short and effective.

Interview Process

Start from a place of trust and respect. Some candidates are not great interviewees because they are nervous or anxious. Ease them into the process, so they can present their best selves to you. An interview process is to hire the best candidate, try not to make it a series of gotchas.

Here are some steps in an interview cycle:

1. Writing a good JD — identify aspects of the role you want to hire for and write out a well thought out JD. Use websites like textio that can scan your JDs for masculine language and help you make them more equitable.

2. Post it somewhere

3. Reach out to your network/Hire a recruiting agency/use an internal recruiter

4. Sift through your candidate pool

5. Phone screen Interview candidates

6. Interview candidates onsite

7. Collect interview feedback

8. Make a decision

9. Communicate decision

An average interview funnel for a role — let’s say you parse through 100 resumes, at 2 min per resume, thats 200 min of your recruiters time just perusing resumes. After picking let’s say 10 candidates out of that to go to the next stage, its about 10 hours of your or your hiring manager’s time for the initial call. Next after that is your onsites — let’s say that’s about 5 interviews, that’s 5 hours * 100/hr * 5 candidates = $2500 per role for onsite interviews, plus your follow up discussion and communication. Assuming you are doing this for every role, you are now wasting hundreds of thousands of dollars just interviewing candidates.

Who should I hire? Decision Making

Create an effective decision making process for candidates. Get into a call after the interview with all the interviewers when they still have their feedback top of mind and discuss the candidacy within 24–48 hours of your onsite. Arrive at a decision and communicate that decision to the candidate within 2 days of the final interview. I cannot emphasize this enough. ALWAYS communicate your decision to the candidate. Ghosting sets a bad precedent of what is acceptable.

Lastly, here are some practical ways that different large tech companies interview.

Amazon — Behavioral questions based on past experiences, and culture fit. Interviews based on their 14 leadership principles. Questions range from “Tell me about a time when you had a conflict with your engineering manager and how did you overcome it” to “Tell me about a time when you had to go against the grain”. They are testing largely for ability based on previous experience. The questions are detailed. And mostly test candidates to see how they think. Why did they do what they did? What obstacles did they overcome? What were the results?

Google — Tests for practical abilities and culture fit — usually about 1–2 phone screens and 5–6 interviews onsite assessing analytical, design, strategy, leadership, technical based on a list of pre-thought out questions. How would you design an alarm clock for the blind? How many electric cars will be sold in the United states in 2021 and so on. They are looking for big vision thinkers, problem solvers, technical abilities.

Facebook — Tests for practical abilities and culture fit — usually about 1–2 phone interviews with recruiter and hiring manager followed by an onsite with 3 interviews testing execution, design and leadership. Mostly based on solving problems for the facebook product, aligning with their mission and so on.

Based on my experiences, I dislike long, drawn-out interviews. I think a total of 5–6 interviews with maybe one presentation/assignment should give you a strong enough sense of the candidate to make an informed decision. Engineering managers, design managers are crucial to the interview process. Continuously derive feedback on your interview process and re-calibrate.

Lastly here are some golden rules -

1. Be transparent with your process and what you are testing for

2. Offer direct feedback if possible

3. Make the process fun and desirable

4. Be respectful of all your candidates — Today’s interview candidate is tomorrow’s advocate

5. Stop the gotchas

6. Offer examples of your interview questions — the more transparency the better, you want well prepped candidates

7. Coach your interviewers — have a written doc that’s stored somewhere for posterity to help your interviewers become good interviewers

8. Get help — if you are an early founder and struggling to figure out how to interview a PM candidate, ask help from your operator friends to step in, your VCs can help too but check for their operator background

9. Keep the interview process short — 3–7 interviews is plenty data points. If you can’t make a decision after all that, re-check your process and re-calibrate

10. Have analytics for your interview cycles — looked at 100 resumes, phone screened 20, onsite 8, made offers to 2. Measure the amount of time spent regularly. Have success metrics and learn from your failures.

11. Lastly, maintain a list of some nearly-there candidates. In a pinch, you should be able to reach back out to those candidates, rather than re-start the search process all over again.