[Guide] PM Growth assessment example

Topic

Product Management

Type

Guide

Intro
Assessment
Assessment breakdown
Data understanding
Product user flows
A/B Testing
Prioritization

Intro

In this post, I will share an assessment that we used when we were searching for a product manager for our consumer growth initiatives. Consumer growth at efood signifies that the product manager will be accountable for product initiatives and projects that center on user acquisition and retention. Most often, these projects correlate with the user experience on our website and our apps.

Even though this assessment is old, it remains relevant.

There are many online resources about all things Product Management. Still, I couldn't find many worthwhile public product assessments. I believe that by sharing this, I will help professionals understand what we expect from a Product Manager in charge of our growth projects and the implementation roadmap for our Android & iOS mobile applications.

Assessment

Assessment breakdown

This assessment has four main pillars. We evaluate these pillars separately to obtain the total score. A primary challenge here is not to force the candidate to spend excessive hours of their free time, while not wanting to exclude any of these pillars from the evaluation.

The purpose here is to assess the candidate's thought process, not the length of the deliverable (we even mention this in the small print in the deliverables section).

The case is about mobile web. The mobile website of every business is a tricky area. There are numerous examples of poor and best practices, but generally, a website that incorporates the advantages, such as Google's discoverability, and disadvantages, like limited capabilities compared to apps, of web browsers within the small real estate of a smartphone. Mobile web is not dead, and e-commerce businesses that invest in mobile web gain significant value over those that ignore it.

Data understanding

This report doesn't allow candidates to delve deeper. It only provides overview data. The reason for this is we want to understand if the candidate can comprehend how Google Analytics works and identify potential challenges and opportunities from this high-level overview.

Content group has a specific definition in Google Analytics. Through this report, you can discern which landing page group (which means which product flow) converts better, which group has significant traction and which group has the largest potential.

This report will help the candidate to prioritize their A/B tests efficiently. When candidates ask which specific restaurant’s page they should examine, it is a clear indicator that they are not familiar with how content group functions in Google Analytics.

There is no right or wrong approach, as long as you have sound arguments to support it. For example, some candidates might focus on the homepage because it generates the most traction in terms of sessions, so even the smallest increase in CVR will lead to more transactions.

Others might focus on the menu page landing group because the combination of a low bounce rate with a low conversion rate might lead to “easy wins” A/B tests. A small bounce rate indicates that users are attempting to convert, so there must be impediments within the flow preventing user conversion.

Product user flows

Product flow optimization varies across industries. Some services don't require account creation for a transaction, while others do. In our case, if you want to place an order online, we need you to create an account and provide us with your delivery address.

As you can see in the report, we're talking about landing pages. A landing page is the page where a user "lands." This means the user hasn't seen the flow steps that occur before this landing page. Hence, you should be able to create an account and/or provide your address on every landing page that users "land" on the most.

The best suggestions focus on how to encourage users to add everything necessary in a smooth and efficient manner. Other good suggestions identify and close the gaps between the restaurant list and restaurant page (e.g., if a restaurant offers loyalty points, it should be highlighted in both the restaurant list and restaurant page).

An experienced product manager will check not only the primary path or the "happy path" as we define it, but also some other less straightforward paths.

For example, there are suggestions to enhance the restaurant reviews section if a restaurant has excellent reviews. Checking reviews is not mandatory to place an order online, but it can dramatically influence user behavior.

A/B Testing

When it comes to A/B testing, we mostly consider the ideation process plus the mechanics of the test. Ideation may seem like an easy task. Ideas are everywhere, they're cheap, and there are literally tons of good, bad, unconventional, and innovative ideas available for free. What distinguishes a good candidate is how she selects the ideas that she believes are worth exploring.

We've seen a lot of A/B test suggestions that propose bigger copy (order now → ORDER NOW or O R D E R N O W) or changing the color of the Call To Action (CTA) button. We don't favor such suggestions.

This doesn't mean that we don't try these kinds of A/B tests, but we prefer to see something more relevant than an A/B test suggestion that's considered a best practice (some might call it basic).

The simplest way to come up with good ideas is to do a brief analysis of the biggest online delivery apps out there. There are fantastic services worldwide that allow you to order food online, such as PedidosYa, Talabat, Foodpanda, Doordash, Zomato, Swiggy, etc.

There's also cross-industry inspiration, where you can generate ideas based on services from other industries like Fintech, Social Media, etc.

Prioritization

Prioritization is a hot topic.

It is one of the most crucial skills of every Product Manager.

There are various prioritization methods available; some are popular, some are more ROI-oriented, and some are anti-patterns. Since each method has its advantages and disadvantages, it helps us gain better context regarding a Product Manager's prioritization style.

Some PMs are iteration-oriented (small and fast changes), while others are ROI-oriented (bigger but more impactful changes). Every prioritization method is welcome because, from time to time, we adjust our prioritization schema to meet specific business needs.

One argument about prioritization is that, because PMs aren't aware of our Tech stack, they'll mostly fail to predict correctly the effort required for task implementation. Firstly, this applies to all candidates, so it's fair.

Secondly, an experienced PM can understand the difference between the effort required to change the placement of a button and the effort to change the entire product flow. For the record, no excellent assessment lost points due to poor effort estimation.

Let me know if you like this assessment and feel free to try it and share your efforts with me (email me at giamalisk@gmail). Good luck!

Relevant posts:

[Notes] PM Habits Seminar: Get Shit Done by Shreyas Doshi