Revulytics Blog

Q&A with Steve Johnson, VP of Products, Pragmatic Institute

August 8, 2019

Subscribe

Revulytics sponsors a series of Product Management Today webinars featuring innovative ideas from top software product management thought leaders. In these blog posts, we ask the presenters to share their insights - we encourage you to watch the full on-demand webinars for even more details.

Steve Johnson is VP of Products at Pragmatic Institute, former CEO and founder of Under10 Playbook, and author of Turn Ideas into Products. In his webinar, Maybe We Should Be Problem Managers Instead, he explains how to redefine product management to focus tightly on the value only you can add. You’ll learn why product managers should focus on identifying problems for others to solve, how to set priorities more effectively, and how to spend less time doing other people’s jobs.

When you talk about best principles of product management, I’m afraid you’re going to give me lots more to do. I’m busy enough already. Where do I start?

Often, product management teams are trying to do too much. Specifically, they’re so busy helping other departments, they aren’t doing the job they were hired for. We need to focus on doing a few things really well instead of trying to participate in everyone else’s dysfunction.

Instead of being product managers, we should be problem managers.

When I talk with executive teams, it’s clear what keeps them up at night. They worry:

  • How do we sell more of what we’ve built?
  • Can we build what we’ve planned?
  • Have we planned the right products?

Often, they see these as siloed issues. Selling more is a sales issue, and building is a development issue, and planning is somebody else’s problem. But they’re all product management problems. Yet, when you ask a lot of product managers, what do you do here, it’s hard for them to answer. Often, it seems like the old video game Asteroids: you sit in the center and try to shoot bad things as they come by, and sometimes, they shoot back.

When I’ve talked with product managers, on average, 47% of their time is spent in unplanned activity. Walking back commitments the sales team made. Walking back announcements that marketing – or worse, the CEO – has made. Cleaning up others’ mistakes. So it’s helpful to step back and think: what exactly do we expect of product management?

Consultants like me need a quote from Peter Drucker. So here you go: “The aim of marketing is to know and understand customers so well that the product or service fits them and sells itself.”

He’s not talking about marketing communications. He’s talking about what we call product management. Our #1 role is to understand the customer so well, we create a product that sells itself.

You said we need to focus on problems, not solutions. What does that mean, and how do we do it?

Developers don’t say I code features. They say, I solve problems. They’re solutions people. What they need is an understanding of problems. That’s a primary role of product management: to go to the market and understand its problems, so we can inform developers and marketers about those problems, and they can provide solutions.

Development and marketing are better “solutions people” than product managers. Developers know more about the technology, and what’s possible. What they don’t know is the problems you’re trying to solve. I sit down with my developers and say this is the challenge I’m facing, this is the outcome I want, and I’m amazed at what they come up with.

As a product manager, instead of focusing on features intended to deliver solutions, focus on finding problems with research.

Can you share insights on how to approach customer research?

Research gives you data to make decisions. But when I say research, people imagine scientists, lab coats, one-way mirrors, artificial environments. In fact, every conversation is research – but only if you learn from it, and write it down.

All our customer interactions fall into one of four squares on a grid. On one axis: anecdotes vs. data. On the other: problem discovery vs. solution verification.

customer-interaction-grid-steve-johnson

We have lots of anecdotes, and perhaps not much data. Or, worse, we have data, but we don’t understand its context because we don’t have the anecdotes. As Jeff Lash of Sirius Decisions says, “Best-in-class product teams use quantitative data from online sources as a supplement to qualitative research, not as a substitute for it.”

Some research is about discovering problems and others about validating our solutions.

The diagram calls out techniques you’re familiar with, starting with interviews and observations. You must have first-hand experience with a problem to understand it viscerally. You can’t get that from a research report, and frankly you can’t get it just from feedback from salespeople or even customers. You need first-hand experience, through observation.

Imagine I’m pouring milk into a measuring cup and trying to get the exact right measure. It’s hard. If people who made measuring cups were in the kitchen with me, they’d see the problem. But if you’d just interviewed me on the phone, it’s unlikely I’d mention it.

Once you have an observation, it needs validation: let’s experiment to see if our solution will actually solve this problem. We sit down with our engineers to brainstorm solutions. They have some really creative ideas and put together a prototype. I think, wow, that’s pretty cool, let me share that with a few clients to see if they like it.

But it’s still anecdotes. Development tells me, you talked to three people, I need to see 3,000. In my experience, salespeople and executives love anecdotes, developers love facts. We need both, to determine: are we really solving the problem, and have we really discovered the right problem?

We can leverage two other techniques you may have available: customer forums and product usage metrics. With today’s in-app tools, you see somebody bought the measuring cup but they never use it, so we haven’t really solved their problem. Or, by participating in customer forums, we can see people saying, well, it doesn’t really work for me, in this way, or that.

Then you can say, maybe there are other solutions, let’s do some A/B testing. If you have an active website, you can A/B test there on messaging and price points. A/B testing isn’t very good at uncovering problems, but it’s very good at validating solutions.

How soon should you think about pricing?

Early. Back at the interview stage. But don’t ask specifically about prices, because we’re terrible at thinking about prices, and customers are even worse. Instead, ask: OK, you have this problem. What bad things happen as a result? Quantify the pain. Maybe because the measuring cup is hard to use, you put too much milk in the cookies and you ruined them. You wasted your ingredients and your time.

On the discovery side, you’re understanding the problem and its impact. On the validation side, you’re using techniques like A/B testing to ask what customers will pay. On your website, you show 50% of your customers a $20/month subscription offer, and the other 50% a $200/year offer, and see which attracts more “Tell Me More,” “Send Me Your Newsletter,” “Tell Me When It’s Available” responses. You can do that kind of observation throughout your product lifecycle.

How does a “problem” focus affect how you set priorities and collaborate with development?

We tend to tell developers: here’s the feature I need to solve the problem. Instead, we should talk to them about the problems, and then collaborate with them on the solution.

Many teams use some form of Kanban-style queueing. Here’s the method we’ve implemented in our Under10 productivity tool for product managers. All new ideas, problems to be solved, jobs to be done, go into the planning queue and sit there until the product manager sits down with the product or engineering team, and they say: I agree this problem is ready for us to work on.

steve-johnson-queue

Many teams use the user story format. That’s OK, but most user stories aren’t very good. They tend to be, as a user, I want this feature so I’ll have this feature I want. That’s not “ready for the developers.” But in our planning stage, we say: here are 30 problems we’ve identified, and we’ve prioritized a few to talk to you about, and we’ll discuss them until you say, this is ready for you to work on.

That means the 3Cs of user stories: card, conversation, and confirmation. A card or Jira record isn’t enough. It’s just a placeholder for a conversation that moves the idea from “planning” to “ready.” And when the developers say, hey we’re bored, got any work for us? The answer is, yeah, here’s what’s at the top of my ready queue.

Then, they work, work, work, and say we’re done, and you say, show me what you’ve done so I can formally accept it, based on our acceptance criteria. That’s how they know they’re done. And once we’ve accepted it, we might hold it until a formal launch, or we might release it right away.

Here’s a key point: product managers put their problems into “planning,” and those problems get converted into items “ready for development,” but we haven’t handed them to developers yet, so we can continuously reprioritize. Maybe something suddenly gets important, like augmented reality or GDPR, and we have to quickly move it to “ready.”

To prioritize, we need to convert market research into product insight, and understand how a solution will impact our customers (and the company). You can use factors like these:

  • Is this an innovative way of solving the customer’s problem?
  • Do most of our customers need this?
  • Will it generate new revenue, or more renewals?
  • Are we losing deals because we don’t have it?
  • Do most of our competitors have this, and we need it for parity?
  • Will it reduce our internal support costs?
  • Are customers dissatisfied with their current solution?
  • Do we have the expertise to build this?
  • Will it be easy for us to deliver?
  • Is it a prerequisite for other things we want to do?

You can weight these factors differently, depending on where your product is in its lifecycle, and where company leadership is focused. So if you’re playing catchup, maybe product parity solutions are more important. You can also weight other criteria – such as a request from your biggest enterprise customer.

Could you talk more about aligning with your market research department?

At a very large company, someone from research told me, we have limited resources, so we go to one product manager a month to see if they have any research needs. That’s the tail wagging the dog: you need research when you need it.

As you embark on a project, you can say: I need to set up some interviews or do some observation or A/B testing. A research department can facilitate that. They know the mechanics. What they don’t know is: what hypothesis are you trying to prove?

Here’s another Peter Drucker quote: “In a well-run organization, each role has a single orientation: they either support customers or support the market.” That is, they either support customers one at a time, or they support the market of many.

Is product management about supporting customers one at a time? Or about supporting the market of many?

Here’s another 4-square grid. Some activities are market based, and some are customer based. Furthermore, some are about future deliverables – say a roadmap. And some are about what we have now: for example, a demo to a single customer.

steve-johnson-grid

Development focuses primarily on future deliverables for a market full of customers. Marketing also focuses on the full market, but primarily for products we have now. Sales deals with one customer at a time, and it should also talk primarily about current products. And professional services teams create features for future delivery to one customer, even if those might eventually be added to the base product.

And it seems like product management supports all those things.

That’s the #1 issue: product managers are so busy providing support for all these groups, they’re not actually doing product management. Marketing needs someone to stand in a trade show booth and demo, or write their eBook. An RFP comes in, and sales says: this involves writing and technical stuff and we don’t know any of that so please write it for us. Professional services says, we’re trying to make everybody fully billable, so write our Statements of Work so we don’t have to.

Look at your emails and meetings. I typically see this priority: product managers tend to be focused on the development quadrant, supporting development for the overall market for the future. #2 is we the time we spend providing website and other product content for marketing and sales enablement. We spend a fair amount of time helping salespeople with individual customer issues like RFPs or demos, or doing things traditionally called sales engineering.

But where’s the actual product management?

Executives rarely measure product managers on product support. They say things like: we don’t seem to have a product strategy. A 2-week sprint is not a strategy. Or: I don’t see product managers doing anything to increase product growth. We’re so busy supporting others, we haven’t done strategy or planning or growth.

While I’m talking about this, there’s another metaphor for project management that causes me grief: “CEO of the product.” I like this better: steering the raft from the rear.

I took my nephews rafting, and they wanted to sit up front to steer, and I sat back next to the guide. And the nephews got exhausted because they were trying to steer from the front, whereas the guide was in back, comfortably controlling things. Often, I find sales and development try to steer product strategy from the front. I want the product manager to be the guide, sitting in the rear, saying I’ve been down this river before, I know where the rocks are, I can suggest how to make this a more enjoyable trip.

So if the product manager isn’t providing support for all those departments, who should be?

I believe in those departments taking care of themselves. Sales shouldn’t rely on product management on a daily basis: they should have their own product support. It’s called sales engineering. If you sell a complex product through a direct sales force, the sales team needs that, and it tends to be under-resourced. The VP of Sales says, well, I could hire 4 sales guys and a sales engineer. Or I can hire 5 sales guys, and just have the sales guys call product management because they’re just sitting around waiting for our calls.

Similarly, services wants its people 100% billable, but that’s unrealistic – and they shouldn’t have other people writing their SOWs anyway. If you’ve ever provided services, you know it’s crucial that the services team itself defines what it’s delivering. As for developers, they often have business analysts or project managers; if they don’t, they rely too much on product management.

This relates to another big piece of baggage: “product manager” is kind of a nonsense title. It’s like “developer.” There are so many specialties within development: QA, UX, front-end, back-end, database, analytics, and so forth. I recommend three similar specialties in product management.

Some of what we do is about what we’ll be building in the future. For that we need a product strategy manager, who’s probably more of a businessperson.

Some of it’s about planning our next deliverables, the next release. That person tends to be more technical. I call that role product planning manager; sometimes people call it product owner.

And some of product management is about growing sales of what we already have: getting more leads, awareness, acquisitions, and addressing other problems related to product growth. That person is a product marketing manager, and of course they’re more marketing and sales oriented.

How do you get other departments to take care of themselves, without being told you aren’t a team player?

The first step of any 12-step program is to admit you have a problem. Talk with your leadership team about how we should measure success in product management. Is it about supporting other teams? Or is it about getting product strategy right, successfully planning products, and achieving product growth?

But it’s even more a product manager’s problem: you want to be a good guy, but you’re letting other departments define your role. Here’s one technique to start changing that: Product Management Thursdays. Pick a day a week to work from home and focus entirely on product management.

Salespeople have a clear definition of success. It’s called quota. If in December you’re behind quota and you tell the VP of Sales, well the good news is I sat in on a lot of marketing meetings, your VP would say you’re doing other peoples’ work, not what I hired you to do. That’s my argument: What job were you hired to do?

I agree we can’t just say you’re on your own. But we need to help enable those teams to support themselves. For example, if you’re always asked to write RFP responses, sit with marketing to come up with one set of answers to questions RFPs keep asking, so they can be reused.

What KPIs can measure product management’s success?

If product strategy is about business development – creating new business ideas – measure based on how successful were we in articulating those new ideas. For product planning, did your priorities lead to a marketable release that satisfied multiple stakeholders? For product growth, how many new customers did we acquire? How many did we retain?

If you pay product managers based on the wrong thing, you encourage them to do the wrong thing. For example, if you measure based on sales revenue, you’ve created a sales engineering department: your compensation plan tells product managers to support salespeople individually rather than taking their product growth role seriously, which would help everyone.

Focus on the role’s core value add, and define a metric for that. In product management, it’s product strategy, planning, and growth.

Keith Fenech

Post written by Keith Fenech

Keith is Revulytics’ VP, Software Analytics and was the co-founder and CEO of Trackerbird Software Analytics before the company was acquired by Revulytics in 2016. Following the acquisition, Keith joined the Revulytics team and is now responsible for the strategic direction and growth of the Usage Analytics business within the company. Prior to founding Trackerbird, Keith held senior product roles at GFI Software where he was responsible for the product roadmap and revenue growth for various security products in the company's portfolio. Keith also brings with him 10 years of IT consultancy experience in the SMB space. Keith has a Masters in Computer Science from the University of Malta, specializing in high performance computing.