Build better products with continuous product discovery | Teresa Torres

[00:00] Everything in our backlog is a bet. [00:02] Everything, whether we do discovery or not, everything is a bet. [00:05] Discovery is helping us make a better bet. [00:09] Now sometimes in our organizations we need to do a lot of discovery and make as good of a bet as we can. [00:15] But there's other times we can make a risky bet. [00:18] Like there's times in business where it makes sense to make a risky bet. [00:23] If it works somewhere where all of your bets have been risky because you're doing zero discovery, [00:29] The best way to kill any appetite for discovery is to say, let's stop making bets until we discover. [00:36] No, don't do that. Keep making bets in parallel. [00:41] start doing some discovery so that eventually, [00:44] Those bets get better. [00:46] And I think the reason why people make this mistake is they think about it as phases. [00:51] First I discover and then I deliver. [00:53] No, you're always delivering and you're always discovering. And the more you build this discovery habit, the better those bets are going to get with time. [01:01] So it's not that you do one first and then the other, it's you're always doing both. And the benefit of always doing both is, [01:07] is with time you make better bets. Teresa Torres is a speaker, a teacher, a consultant, a product coach, and also the author of Continuous Discovery Habits, which is the number one most recommended book in my newsletter Slack community. I'm also pretty sure Teresa is in the top five people in the world when it comes to the number of product managers that she's worked with, taught, and impacted. In our chat, we get deep into two topics, the opportunity solution tree

[01:37] but incredibly powerful framework once you know it. And two, we go deep into how to create a system within your team where you're talking to customers regularly. We talk about ways to make a case for spending more time talking to customers and doing user research, the most common mistakes people make with interviewing and generally how to interview customers better, how to automate this process so that you don't spend a bunch of time, and so many other ways to bring you and your organization closer to your customers. Teresa's amazing, and I can't wait for you to learn from her. [02:07] This episode is brought to you by Persona. Persona helps founders, product managers, and engineers easily solve any identity-related problem, including handling KYC, AML, and basically all manner of identity fraud. You can integrate Persona in an afternoon and personalize your flows using their SDK to meet your users on any device. Persona's identity building blocks allow you to manage your entire end-to-end onboarding flow. [02:37] verifying that each new user and their data are legitimate. Persona is trusted by both startups and the world's largest companies, including Square, BlockFi, Gusto, and Udemy. And for a limited time, Persona is offering listeners of this podcast a free end-to-end KYC and AML solution, where you can collect a user's government ID and or their selfie and automatically verify that those two pieces of data are legitimate.

[03:07] see if the person exists on various watch lists. Just go to withpersona.com slash Lenny to get started. So many product managers are basically treated like project managers. They get a hire thinking they'll be deep in product strategy, vision, and getting to know their customers, only to wind up organizing other people's work and refining backlogs and organizing tiny, [03:37] true heart of product management is understanding what customers want, why they want it, and how to give it to them. That's why Dovetail built a suite of user research products that help you get to the core of what your customers really want and why they want it. Dovetail offers powerful analysis tools to help you identify themes, patterns, and insights in your customer interviews, allowing you to make better data-informed decisions about what solutions you should build next. Organizations the [04:07] Inva, Datadog, GitLab, Neal's at Norman Group, Sketch, Deloitte, all use Dovetail to get a better understanding of their customers and build better products. Try Dovetail's products for free for as long as you need. You can sign up and dive straight in at dovetailapp.com slash Lenny. [04:29] Teresa, thank you so much for being here.

[04:37] recommended book in my Slack community. And also, I've personally learned so much from you from your writing and just your tweets and all the ways that you share your lessons. And so I'm really excited to [04:48] delve into continuous discovery and all the things that you teach, [04:51] And so again, thank you for being here and welcome. [04:54] Thanks, Lily. I'm excited to be here. [04:56] How many PMs would you say that you've worked with through your consulting and through your courses, just to give us a ballpark? [05:03] That's a good question. I know through Product Tech Academy, we're just at about 11,000 students. I'm not very good at updating the number on our homepage. I think it still says something like 8,500. [05:14] But officially, I think we're just about to cross the 11,000 mark. [05:18] which is a little bit mind-blowing to me and a ton of fun. [05:20] And then on the coaching side, it's in the hundreds, not in the thousands. Obviously, this coaching is a little bit of a different beast. So, yeah, I would probably say maybe 12,000. [05:30] That's incredible. Would you say that maybe puts you in the top five PM teacher influencer types that have worked with maybe the most PMs just roughly just to give people a little context? I don't even know how to evaluate that. Like I still think about like some of the really early people that had an influence on me and they'll see that some of them now recommend my book or it's just mind boggling. It's pretty cool. Yeah, I feel the same way sometimes with my newsletter. I'm like, okay, this person that I always looked up to as a legend is now sharing my stuff. That's [06:00] Before we get into the meat of the chat, [06:02] just real quick to plug your site and kind of where to discover all the things that you do. What's the site people can check out while we chat?

[06:07] Yeah, so my site is producttalk.org. And then there's a few things like I blog at producttalk.org slash blog. We put up two articles a month. Then we have a whole bunch of courses related to discovery at learn.producttalk.org. [06:19] Awesome. And we'll chat a bit more about that at the end. So in our chat, I was hoping we'd cover kind of two main topics, things that I've maybe learned most from you over the years. One is, [06:31] the opportunity solution tree framework. And then two is just the general idea of continuous discovery and all the ways to approach that. [06:39] Does that sound good? Yeah, that sounds great. Okay, sweet. So... [06:43] Starting with the Opportunity Solution Tree Framework, it's such a simple but such a powerful way to visualize your strategy, your levers, how to prioritize, get buy-in from people, get everyone on the same page. [06:55] And it's probably the thing I share most of what you've put out with people. [06:58] So I'd love to maybe start with just like what is this framework? What problem does it solve for people? And how can people apply it to their product problems? Yeah, really good question. First of all, it's just a really simple visual. It's funny how simple it looks because using it in practice is really complex. And I definitely have a new appreciation for that, trying to teach it over and over again. [07:19] and seeing where people struggle [07:21] so it's a tree visual so it's just like a decision tree [07:24] It starts with an outcome at the root of the tree, and then it branches into the opportunity space, and then it branches into solutions. [07:31] and maybe even assumption tests from there. [07:33] And the purpose of it is [07:36] I recognized that while as an industry, some companies are moving from this output focus to outcome focus.

[07:43] Most product teams, [07:45] don't really know how to manage this really complex problem of how do I start from an outcome and figure out what to build. [07:52] Like, it's a really unstructured... [07:54] wide open hard problem. [07:57] And a lot of teams, they learn how to do their jobs building products by being told [08:01] Build these features. [08:02] That's a really structured... [08:04] "Okay, I just churn out some work," kind of problem. So we're asking teams to fundamentally do a really different type of job. [08:12] And I think teams needed some scaffolding for how do you make that shift? And so that was the purpose of the opportunity solution tree is how do I add some structure to this wide open, messy problem? [08:22] Now the reason why it looks simple but is really hard in practice [08:26] is what's an opportunity and [08:29] how do I structure the opportunity space and I can tell you that opportunity is an unmet need pain point or desire [08:36] And that's great, but I can tell you that 98% of people that write opportunities write them as solutions. [08:42] so we tend to just really struggle with this distinction [08:45] between the problem space and the solution space. [08:48] And I think that the heart of good product. [08:51] is really getting comfortable in the problem space or the opportunity space. [08:55] Really taking the time to frame a problem will help [08:58] and to really get into what's needed before we jump to solutions. [09:03] But it's like the opposite of how our brains are wired. [09:06] And so, [09:08] Teaching people to be comfortable with that. [09:11] discomfort of staying there is hard and I think

[09:15] A lot of people, I mean, I see blog posts written about how they're using the opportunity solution tree. [09:19] And I cry a little bit because their opportunity space is all solutions. And I don't want to knock down somebody's blog post. [09:25] But I also don't want this like bad example out there in the world when I'm trying to teach, like, how do we do this? Well, I haven't found the right line there yet, other than I'm going to blog about good examples. Is there an example of a tree that just to make it even more concrete, like for a company or product that you've worked with or that you think about? [09:40] as a good example. I like to use streaming entertainment as my examples, because like literally everybody in the world is familiar with [09:46] Netflix, [09:47] And if I think about like their opportunity space, I recommend teams structure the opportunity space using an experience map. If we start with Netflix. [09:55] If you think about the experience of like you're trying to get me to engage with Netflix more, I want to understand what's the experience of somebody using streaming entertainment to entertain themselves. [10:06] maybe even broader than Netflix. If you watch, [10:09] YouTube TV, that's probably still relevant for me to learn about and understand. [10:13] And so the way that I'm going to structure my opportunity space is I'm going to look at what's the overall experience of trying to entertain yourself with streaming entertainment. And that might look like first, there's this trigger of I need to decide to watch something. [10:26] And then there's this experience of like, how am I deciding to watch? [10:29] And usually wrapped up in that is what platform am I watching it on? [10:34] Those are sometimes intermixed. Because maybe you're deciding, I want to watch Game of Thrones, and that's right away sending you to HBO Max. Or maybe you're like, I want to watch a movie. [10:42] and I could be on any platform. And then there's the evaluation process of, is this movie good or not? Does it look like something I want to watch?

[10:50] And then I want to get into, okay, I'm ready to watch. Is it a good viewing experience? [10:54] And then for a lot of these platforms, there's also this sort of post viewing experience. How am I going to encourage you to keep watching? Things like that. [11:01] But like, that's how I would structure that opportunity space is just there's these distinct moments in time. [11:06] and then what i'm capturing is below each of those what are the needs and pain points and desires that arise so if we just focus on that one around like how do i decide what to watch [11:16] There's all sorts of needs that come up. Some are really tactical, like, [11:20] I have a movie title in mind and I just don't know how to find it. [11:23] right that's a pain point or hey i was watching a show how do i get back to it [11:29] It's also a pain point. But then there's also these big media opportunities like, [11:33] I can't tell if the show is good or not. [11:36] Right. And so like, [11:37] Everything that I'd just said, there was no solutions in there. [11:41] In fact, whether I work at Netflix or I work at Hulu, our opportunity spaces probably look pretty similar. [11:48] Now the ones we choose to go after and how we solve them might look really different. [11:51] But like the core human needs of what's your experience as we go about our lives, entertaining ourselves. [11:57] is pretty similar. [11:59] And I think the companies that build really good products, they either intuitively or explicitly take the time to really understand. [12:06] What does our journey look like and what are those needs and pain points and how do we create [12:10] a really seamless experience. [12:12] We're going to link to the blog post and a few examples in the show notes so people can look at this kind of visually because I know we're trying to describe it through words. A quick follow-up question. So you have this kind of tree with the outcome say, get people to watch more Netflix. I imagine you recommend max like three to five

[12:29] levers below that. Is that right? And then the rest just kind of sit somewhere else? Yeah, that's a really good question. So at the top level, [12:37] i tend to map those opportunities to like steps in that experience map so in that [12:41] Netflix example would be like the trigger of I want to watch something, deciding what to watch, the viewing experience. And I do find that like, oh, is it like Miller's magic number, the plus or minus seven? [12:54] rule is pretty good. [12:55] I would say nine is probably a lot. So I would say maybe in that three to seven range. And that's just because it's [13:02] You could cognitively process your tree. [13:05] So like the other thing I get into is like as you move vertically down the tree, your opportunities are getting smaller and smaller. [13:11] This is like a really key to helping us unlock a continuous cadence. [13:14] So if I start with that example, if I can't decide what to watch, it's a really big, hard, evergreen problem. Like as long as Netflix is in business, they're probably going to have people focus on that problem. [13:24] But we can deconstruct it like, maybe I can't find something to watch because I don't know if this show is any good. [13:29] And then we can learn about how do people evaluate shows. [13:32] And maybe there's a small opportunity in there of who's the cast. [13:35] It's one of the ways I evaluate a show. [13:37] Now we're getting into an opportunity that we can actually solve. So that's one of the other benefits as we work our way down the tree. We get to smaller and smaller opportunities. We get to things that we can actually address. [13:47] we're still contributing to that bigger, harder problem. [13:50] And so what it's allowing us to do is get this big picture view of where we could play, [13:56] And then we can make more strategic decisions about where do we actually want to play. And then it's very customer focused because it's really all about what's your need in this moment. How can I help satisfy that? For PMs and just maybe founders that are listening and they're like, "This sounds really useful. I want to create my own little opportunity solution tree." I know that's a big question, but how do you go about figuring out what kind of goes into each of these steps? However briefly you can give people some guidance there. Yeah. So the reason why this is so hard is that it sounds really simple, right?

[14:22] But here's why it's hard. Opportunities emerge from our customers' stories. [14:26] I don't think most people, when they're interviewing, they collect stories. [14:30] So if Lenny, I was going to interview you about your Netflix experience, the vast majority of product teams are saying things like, hey, Lenny, what do you like to watch on Netflix? [14:38] How do you decide what to watch? We're asking these direct questions out of context, [14:43] And the challenge with that is we know from like, [14:45] Human psychology and cognitive psychology that we're not very good at answering those questions at a context actually it sounds weird We're very good at answering them your brain will come up with a fast answer and [14:56] but that answer doesn't necessarily reflect your behavior. [14:59] And it misses context and nuance. Like you can tell me like, oh, I like just like action movies. I always look for action movies. [15:06] I can't visualize your experience. I don't know what your experience is. [15:10] watching action movies. I just collected a fact about you. [15:14] But if I ask you, like, tell me about the last time you watched a movie... [15:18] Now I can collect things like where were you and who were you with and set the scene for me and what happened first and how did you choose that? I'm going to get answers to all those direct questions, but it's grounded in a specific instance. It's going to be a lot more reliable and. [15:33] I'm going to start to hear unmet needs, pain points, and desires. And the really powerful thing is I might hear needs that you're not even aware of. [15:42] Right. We're so used to like. [15:44] Everything's being mediocre. [15:46] We're not even aware of a lot of these needs that we have. But when we tell our stories, especially if you start to train your ear for this,

[15:53] You start to hear those needs. [15:55] So the first thing that makes this hard is you have to interview well. [16:00] And I think interviewing is a grossly underestimated skill. [16:04] grossly underestimated skill. [16:06] That's the first thing is that if you're not collecting rich stories in your interviews, it's going to be really hard to identify opportunities. [16:14] Then the second thing is that you have to be able to hear those opportunities. And if you're still stuck with what's an opportunity versus what's a solution, [16:21] It's tough. And then the third thing is this opportunity framing. [16:25] like i believe opportunity should be really specific like a really great opportunity in the streaming spaces [16:31] It's hard to enter my password. Select specific letters on the screen with the Apple TV remote. If anybody has an Apple TV, especially the old remote, it's like not a very precision device. Selecting those letters on the stupid on-screen keyboard is a horrible pain point, right? It comes up when we're entering our passwords. It comes up when we're searching for movie titles. That's a really specific opportunity. [16:53] And the value of that is we can solve it. [16:56] Whereas teams tend to frame opportunities as, "I wish this was easier to use." [17:01] Okay, well, what? We can spend our lives making this product easier to use. [17:05] What are you solving for who? And so we're skill stacking. And then we've got to interview, we've got to be able to hear opportunities, we've got to be able to frame them well. [17:11] And then in order to structure the opportunity space, [17:14] We have to be able to come pull out this common experience map structure across [17:18] seeming to unique stories. [17:20] There's a lot of skill involved. Like I'm not, I wish I could just say, Hey Lenny, it's super easy. Everybody should do it. I think it's really powerful and I've seen it be a game changer for teams.

[17:29] It's hard. I tell my teams like when I teach it in clocks, I say, look, [17:33] We're going to focus on structuring the opportunity space, and I'm probably going to make you think harder. [17:39] than you've ever had to think in your job. Because we don't think that much at work. [17:43] We go from meeting to meeting, we stay surface level a lot. And here I am coming in with this like really hard critical thinking exercise. But I've just seen from teams that are willing to put in that work. [17:53] It really is a game changer. You have a deeper understanding of what your customers need. [17:56] and you build better products. [17:58] Man, thinking deeply. No fun, but so important. I want to chat about interviewing and all the advice you have about just how to interview. But before we get to that, one last question around the opportunity solution kind of work. So the whole idea is to think outcome oriented. And to your point, a lot of companies, [18:15] have product teams that are just like build these things for us. Don't worry about why we're doing these. If your company is of that latter sort, more of like a feature factory, can you use this framework to kind of push [18:27] the team and the company in a direction of thinking outcome oriented? Or is there like a more direct approach to address that problem? Yeah, okay, let's talk about this based on the rule. Like, if you're an individual contributor, [18:38] And you're not at a 10-person company? [18:40] I would say don't try to force organizational change. Organizational change is such a hard and messy problem. [18:46] I feel like what I would do in that situation... [18:49] is I would just change the way that I individually worked. [18:51] And this is what I always did in every job. I made a lot of mistakes trying to change the organization, but also just [18:57] carved out a way for me to work this way.

[18:59] And I think we underestimate how much ability we have to do that. [19:03] So even if you're being prescribed a fixed roadmap, [19:06] you still can find customers to talk to. [19:08] And I hear from people all the time say I'm not allowed to talk to customers. [19:12] I go, "Okay, your company doesn't own you when you're not at work. And I bet you know people like your customers. Why don't you just start there?" So we overthink it. We think we have to go through these proper channels and we have to get permission from sales. And a lot of us, especially if you work on a consumer product, just go find somebody that's like your user. But I've also seen instances in B2B environments where I worked with a team that worked on badges for healthcare, the badges that nurses and doctors use to [19:37] unlock a workstation that they chart in. [19:40] This team, for weeks, ran into problems finding a customer to talk to. [19:45] And I just said, hey, do any of you know doctors or nurses in your personal network? [19:49] And the product manager was like, "Yeah, I have two uncles that are doctors." Huh. [19:53] Maybe we could just start there. Go talk to somebody. Hey, I think that even if you [19:58] aren't being tasked with an outcome. [20:00] If you do the work to understand these are the outcomes that matter to your business, to your product, it's probably going to start with your business model. [20:07] and then work to understand how the work that you're doing [20:11] contributes to that. [20:12] all those little teeny tiny decisions we make every day [20:16] Even if you're being prescribed solutions, [20:18] He'll make better decisions. [20:20] because you have a fuller context of what your business needs, you have a fuller context of what your customer needs. So I think for most of us, if you're in an individual contributor role,

[20:30] just focus on developing the habits yourself and i'm always amazed like i was always amazed at how much i could do by just [20:36] kind of ignoring everybody around me and how they were working and [20:39] finding a way to do it. I love that because it lets you empower yourself and not wait for permission, for excuses. And that is always such a recipe for success for any role, especially PMs that are annoyed by how maybe their company works. Yeah. Do you want to reduce friction in your onboarding flow? Then let me tell you about Stitch. And that's Stitch with a Y. Stitch is on a mission to eliminate friction from the internet. They're starting by making user authentication and [21:09] Super flexible, out-of-the-box authentication solutions for companies of all sizes, from email magic links to SMS passcodes, one-tap social logins to even biometrics, Stitch is your all-in-one platform for authentication. Stitch customers have been able to increase conversion by over 60% after spending just one day integrating, and with their API and SDKs, you can improve user conversion and retention and security, all while saving valuable engineering time. [21:39] Your engineers will come and thank you for using Stitch, because Stitch keeps you from having to build authentication in-house, and the integration process is super fast and super smooth. To get $1,000 in free credits, just go to stitch.com, and that's Stitch with a Y, and sign up and just mention that I sent you. [21:58] This is a really good segue to our second topic around continuous discovery, and we've been touching on a lot of the

[22:03] elements of interviewing and understanding pain points and all that. And so maybe just to set a little bit of foundation, [22:10] What is continuous discovery? Your book is named after it. You have courses on this. What's kind of like the general idea of continuous discovery? [22:18] Yeah, let's just start at the beginning. So we often talk about discovery in contrast with delivery. Discovery is just used to describe the work we're doing to decide what to build. [22:26] Right, so everybody, every company is doing discovery. [22:29] Everybody is making decisions about what to build. [22:32] We have a few trends that have been evolving very slowly over the last 20 years. [22:36] One of which is we're recognizing that if we want to make good decisions about what to build, [22:41] We probably should include the customer somewhere in that process. [22:44] Right? [22:45] So I teach a customer-centric view of discovery. Let's build in some feedback loops. [22:50] of are we making the right decisions or are we making good decisions [22:54] Because there probably aren't right decisions here. [22:56] Then there's a second trend. [22:58] that we're seeing across the board, which is we're recognizing that digital products are never done. Not like the Netflix team is going to show up to work one day and be like, hey, our product's good enough. We're always iterating. We're always improving. Customer needs are always evolving. There's always more we could do. [23:13] And so we're seeing a shift from like this project mindset that worked in a world where we were just trying to get products on a store shelf. Right. Like we designed them. We built them. We manufactured them. We put them on the store shelf. We were done. We moved on to the next thing. [23:27] But with digital products, there's no done. [23:29] so we're seeing this shift to more of a continuous mindset we're continuously evolving our products

[23:35] which means we're continuously making decisions about what to build. And therefore, I think we need to continuously include the customer in that process. [23:44] So for me, I define continuous discovery as building in those continuous feedback loops. [23:50] Awesome. That's such a simple, clear way of thinking about this, because, yeah, broadly, it's like a new term people have to get used to. Yeah. And I think you saw I made a call on Twitter for people to ask you questions, to ask me, to ask you questions about continuous discovery. And so I'm going to try to get as many of those in there in this chat as I can. Cool. And one actually was around what do you do when your leaders tell you there's no time for discovery? Yeah, that's a tough one. I think this comes from old project-based research methods. [24:19] So we don't. [24:20] most of the time have time to stop what we're doing. [24:24] and go do some research. [24:26] And I'm not putting research. I've been I've worked as a user researcher like I, [24:30] Research is critical. [24:31] And if we had the luxury of doing like long longitudinal studies, we would probably build better products. That's not our business environment. [24:39] Our businesses are expecting us to deliver continuous value. [24:42] So we need to look at how do we match that cadence? [24:45] What I think is really nice about continuous discovery... [24:48] You can do it in as little as an interview a week. [24:52] and on the interviewing side, on the discovering opportunities side. [24:56] Assumption testing! [24:57] People always ask me, like, how much time should I be spending on assumption testing? [25:00] I don't know how to answer that question because for me, assumption testing and delivery are the same work.

[25:06] write like assumption testing, [25:08] is the start of your delivery. I don't know where one starts and one ends, which is a little bit hard to conceptually work through, but maybe we can talk through an example. So when somebody says, I don't have time for discovery, I think what they're really saying, [25:20] is I don't have time for project-based research. And I agree with that. We don't have time for project-based research. [25:25] So if I'm getting that pushback, I want to look for [25:28] okay i definitely don't like [25:31] People make this mistake of we shouldn't put something in our backlog that hasn't been properly discovered. [25:38] Is that true? [25:39] Right. Everything in our backlog is a bet. Everything, whether we do discovery or not, everything is a bet. [25:46] Discovery is helping us make a better bet. [25:49] Now, sometimes in our organizations, we need to do a lot of discovery and make as good of a bet as we can. [25:56] But there's other times we can make a risky bet. [25:59] Like there's times in business where it makes sense to make a risky bet. If it works somewhere where all of your bets have been risky because you're doing zero discovery, [26:09] The best way to kill any appetite for discovery is to say, let's stop making bets until we discover. [26:16] No, don't do that. Keep making bets in parallel. [26:21] start doing some discovery so that eventually, [26:24] Those bets get better. [26:27] And I think the reason why people make this mistake is they think about it as phases. [26:31] First I discover and then I deliver. [26:34] No, you're always delivering and you're always discovering. And the more you build this discovery habit, the better those bets are going to get with time.

[26:42] So it's not that you do one first and then the other, it's you're always doing both. And the benefit of always doing both is with time you make better bets. [26:49] You said that you could do this with one meeting like an hour a week. And I know you have a system that you recommend for people to make this kind of automated so you're not just constantly pinging your customers, "Hey, can I chat with you this week?" Can you just share that? [27:02] Yeah, so my book, Continuous Discovery Habits, I do share some of the most common ways to automate the recruiting process. [27:09] So this idea came from I had just read Nudge by Thaler and Sunstein when it came out a few years ago. [27:15] And they have this idea of like when you're designing a choice architecture, how do you make it easier to adopt the behavior you want to see than to not adopt the behavior? [27:25] So I started thinking about this in the context of interviewing. I want to see product teams [27:29] interview every week so how do I make it easier for them to do that than to not do that [27:33] Okay, well, a lot of us have recurring meetings that we go to every week because they're on our calendar. [27:38] So I just started to think about how do I make an interview a recurring meeting? Can I make it so that when you wake up on Monday morning, [27:44] There's an interview on your calendar and you literally did nothing to get it there. [27:48] And so there's a few ways to think about this. The most common strategy [27:51] is to allow your customers to opt in while they're using your product or service. [27:56] so just like almost everybody's seen an nps survey embedded in the product [28:00] That's pretty prevalent now. [28:02] Same idea, but instead of saying, would you recommend our product or service to a friend or colleague, it says, do you have 20 minutes to talk to us? They say yes. [28:09] You send them some scheduling software, they pick a time on your calendar, and voila, you've got an interview scheduled.

[28:16] You obviously can get more advanced. Where do you show it? Who do you show it to? How much do you tailor it? How do you position it? But the core idea, [28:23] is to let people opt in while you already have their attention. That works really well for consumers and B2B end users. [28:30] If you're trying to get in touch with like buyers and decision makers, [28:33] Same idea, but use your internal teams that are already on the phone with those folks. [28:39] That's sales people, account managers, maybe support folks. [28:42] right they're literally on the phone with those people all day every day so instead of using your product to recruit you can use those teams to recruit [28:50] And what I do is I just have teams define a trigger every week. Like, hey, this week we're looking to talk to somebody who's experiencing this need or pain point. [28:58] If you happen to be on the phone with someone who's experiencing that, again, just go ahead and use scheduling software. [29:04] Put it on our calendar. [29:05] The goal is for the product team to not be involved at all. [29:08] They literally just have to show up and conduct the interview. That's amazing. Are there tools that you recommend that are just like plug and play that make this easy? I know Calendly is probably a part of this. So many. Right. So even on the on the scheduling side, I think Calendly innovated in that space, but there's so many fast followers. Outlook does this now. I think Google has a tool that does this now. I think even Salesforce has a tool that does this now. So if you're doing your schedule for a sales team, the scheduling through Salesforce, [29:35] And then on the like intercept side, like how are you asking those? We have survey tools and like Qualaroo, I think, innovated in this space. And then I think Ethneo was a fast follower. But Intercom does this. Usabila does this. Chameleon does this. Hotjar might even do this. We have so many like user research tools.

[29:54] that they're all now enabling this type of thing. [29:57] Awesome. For when you're actually doing the interviewing, we had a couple of questions from some Teresa fans. One is when you kind of know what the solution should be, how do you stay disciplined and keep an open mind and keep searching for maybe something even better? First of all, you don't always have to do that. [30:15] Right? Not all solutions need a lot of discovery. [30:18] That's like a common misunderstanding, I think. [30:21] I think we need to do really robust, good discovery on the things that are part of our core product experience or going to be differentiators. [30:29] We don't really have to do really amazing core discovery on like the forgot password flow if it's working fine and you're not hearing about it as a pain point. [30:37] Right now, to be fair, Slack with their magic link to the cool thing with the forgot password flow. And there was a nice innovative thing that I think moved the industry forward. So if you want to do discovery on that, great. But you probably don't have to. [30:49] Right. So I think the first thing to assess is. [30:53] We're making a bet. [30:55] How much risk is involved in this bet? [30:57] And how much of that risk do we need to mitigate? [31:00] Now. [31:01] Most companies think there's no risk in any bets and they do zero discovery. [31:05] And if you're not instrumenting your product and actually measuring the impact of those solutions, [31:10] you may not be catching that there actually was a lot of risk. [31:12] So I think you do need to instrument your product. You do need to measure the impact of everything that you [31:17] release so you can start honing your judgment on where is there risk and ideas and when you're new to discovery i recommend you over index on doing a little too much discovery

[31:27] so that you start honing your judgment of that risk. But if you're working on an opportunity that's really core to your product functionality, it is a differentiator, it's where you want to make sure you have a really robust, [31:37] Good solution. I think the best way to guard against what you think is the obvious solution is, [31:42] is to work with multiple solutions for the same opportunity. Compare and contrast. [31:47] Right? [31:48] We already know this intuitively, like when you're looking for a place to live, [31:51] You don't look at one apartment or house, you look at multiple, you compare and contrast. [31:56] When you're looking for a job, you don't talk to one company. We know if we want to make good decisions, we need options. [32:01] And we need to evaluate the pros and cons of each. [32:04] and the same is true in the product world so if you're feeling like [32:07] This needs to be a really good solution, and we're over committing to one. [32:11] That's when you need to increase your options. I'd love your insight on as a PM, [32:16] In theory, you should be a little bit unbiased and giving people a chance to change your mind and come up with ideas that maybe you disagree with. On the other hand, as a PM, you always have opinions about what the right answer is. In the PM function, do you have a perspective on how much more say a PM should have maybe over what ends up being decided? [32:36] Yeah, this is a tough question. There's such strong opinions about this. I see it allergies of the product manager is the decider and they're the CEO of the product. I think this is coming from toxic business culture personally. [32:47] I... [32:48] Business has taught us we all play a role. We have our functional silos. I have territory, you have territory. [32:54] We're going to play the internal office politics game. [32:57] and I need to defend my territory and you need to defend your territory.

[33:01] And the outcome is that we don't really collaborate. [33:03] And when we don't collaborate, I don't think we build very good products. [33:06] So if we just go back to like real life and when you're hanging out with friends and you're trying to accomplish something, whether it's some, you know, like the example I gave is like when you're a little kid. [33:16] and you're playing you don't like first stop and say what's my role what's your role like it's just not how humans interact right [33:23] We all collaborate and we all do it intuitively. [33:26] and business has taught us otherwise. [33:28] And there's... [33:29] I'm going to forget the researcher, but there's a really cool marshmallow test experiment. Are you familiar with this? Where like teams are given spaghetti sticks and some tape. [33:39] and some string [33:41] and a marshmallow, and they're told to build a structure to get the marshmallow as high as possible. [33:45] And the study's been done so many times. It's been replicated a million times with lots of different [33:49] groups and it's a really cool story because kindergartners outperform almost every adult group including like mba students and it's really telling and why is this kindergartners just start doing [34:02] They don't worry about their rules. They don't worry about who's in charge. They just brute force trial and error. [34:08] What do MBA students do? There's posturing, like who has power and who is the decision maker and who's right? And we need a plan and we need to have a strategy. And they spend the whole time like. [34:18] negotiating this political social space [34:21] instead of just doing. [34:23] and i really think we gotta get we gotta learn how to get back to just doing [34:26] People think I'm like Pollyanna naive about this, but I've worked on teams that work this way and I've coached teams that work this way.

[34:33] where the trio really does decide. [34:36] So the trios, the product manager, the designer, and the software engineer. [34:39] And if you've never worked on a well-functioning trio that breaks people's brains, [34:43] Because they say, what are we going to do when we disagree? [34:45] you're going to find an option where you don't disagree. [34:48] Right? The thing is, if you only worked on a siloed dysfunctional team, that sounds like a nightmare. [34:54] But if you worked on a well-functioning team that's doing discovery well, together, [34:58] You're working from a shared understanding. [35:01] So your disagreements right away are going to go down because you're working from a shared understanding. And when you disagree, you recognize, OK, we don't agree. We don't have the best option yet. [35:12] and you keep looking for that better option and what's hard about talking about this is i [35:17] fully understand, [35:19] Probably 98% of the industry has never worked on a, [35:21] well-functioning product trio and this idea sounds crazy, but I've also seen it in practice over and over again on really good teams. [35:28] And there's something magical about it. So I'm going to keep promoting it. And I'm going to hope that eventually we get from 2% to 3%. And that'll be my little dip I put in the universe. [35:37] Yeah, and I was just going to say that you're helping make that change. And I'm excited for that to be the way that people operate. And so maybe one takeaway is if that's something that you're spending a lot of time on, and it's causing a lot of stress, probably means you're working at a company or on a team that maybe isn't optimal. It's not, I don't mean that to say there's something wrong with you or your teammates. [35:57] Right? Like this is a symptom of business culture. [35:59] It's how we've been taught to work. [36:02] So we have to unlearn that. We have to learn new ways of working.

[36:05] And we do this in our courses. We force people to work in teams in our courses, and some people really hate it. But I think learning to work well in a team, especially when there's different perspectives and you disagree and how do you reconcile that? [36:17] is a really important part of product work. [36:19] Awesome. Going back to discovery and interviewing, I definitely wanted to ask you, what are like, I don't know, two or three tips and best practices for interviewing, slash what are like two or three things people usually do wrong that they should try to avoid? Yeah, the first one is the questions they're asking. So many people write these who, what, why, how [36:40] 50 question long. [36:42] like interview protocols. [36:43] And it leads to a cadence of the interview that is not a natural conversation. [36:49] So I think like the first thing to remember is that you're just talking to a human. I actually tell people like if your interview feels like you're having a beer with a buddy, [36:57] That's a good sign. [36:58] Like it should be that casual and that conversational. But we're not going to get there if I pepper you with 50 questions. [37:04] right we're going to get there but i'm going to collect your story i'm going to be really curious [37:08] I might still have to pepper you with 50 questions to get your story. [37:12] Right. Because there's this conversational norm of. [37:15] I say something, you say something. So I got to... [37:17] teach you that I want your whole story. [37:20] and help you open up so that's one piece of it it's just the cadence of the conversation really should feel like a natural conversation and then the second piece of it is how do we do that what's the skill how do we elicit that story [37:31] And I teach in our interviewing class, you really don't have to think about what to ask. You could run an entire interview by asking them one question. In fact, let's role play this a little bit. Lenny, tell me about the last time you watched something on a streaming entertainment service.

[37:44] Just last night I was watching Obi-Wan Kenobi on Disney+. Okay, yeah, great. Okay, so it was last night. Set the scene for me. Where were you? I was at home on my couch, just lounging. Okay, and tell me about the moment where you decided you wanted to watch something. It was like eight o'clock and I'm like, "It's time to watch something." Okay. [38:02] Just is that part of your normal routine? [38:04] Yeah, in the evenings it's a good way to unwind and let my brain relax a little bit. Okay, so you're sitting on the couch. [38:11] You decided it's time to watch something. What did you do next? [38:14] Turn on the TV. [38:16] Went to Netflix, didn't find anything. Went to Prime, didn't find anything. And I'm like, oh yeah, Obi-Wan. Let's check that out. Okay, so I literally could continue this entire interview. [38:27] by just saying, [38:28] Oh, you opened Netflix. What happened next? Oh, you didn't find anything. How come? All I have to do is just be curious about your experience. And what I'm doing with my questions is just helping you. [38:39] Tell the timeline. [38:41] right like where set the scene i'm situating you back in that moment let's remember what you actually did is after dinner you're sitting on the couch what happened next i can do that over and over again and so one of the reasons why we get bad at interviewing [38:53] We're so worried about asking the next question. [38:55] we stop listening to the interviewee. [38:58] We just missed everything we were told. [39:00] So then had we missed those moments of, oh, there's some friction. You couldn't find something to watch. Tell me about that. [39:06] What did you consider on Netflix? Let's dig into that. And if I work on a team that's trying to help you find something to watch, that's a goldmine. [39:13] right like you just told me you went on netflix you went on prime what were you looking at and what didn't resonate and is it because you'd watched everything is it because it just didn't match your profile there's so much

[39:23] To explore there, [39:24] But what I see most teams do is they stay really shallow. [39:28] Oh, okay, so you watched Obi-Wan... [39:31] on, [39:32] Prime. Great. [39:34] Tell me another story. [39:35] And we just lost all the value. And so some of it is just slowing down and almost being like a five-year-old. [39:42] right like you really instead of saying why you can say what happened next [39:47] What happened next? Now, there's this technique of summarize what you heard, show that you're listening to them, [39:52] bring them back to the moment where you want a little more detail. But yeah, that's, it's a game changer. What happens when you collect stories is you hear about things you would have never thought to ask about. [40:02] And it's also really fun to share because I'm like, oh, this is fun. She's talking about what I did. I love that you just said that because people worry. Like how many times have you heard somebody say like, [40:11] You ask the sales rep, "Hey, can I talk to your customer?" [40:14] And they're like, I don't want to ask them a favor. [40:16] Okay, it turns out if you collect stories in your interviews, [40:19] Customers love it. [40:21] Most of the time, in fact, the sign that you ran a good interview is your customer is going to say, wow, when can we do this again? Wow, I love that. The other piece of this that you haven't mentioned is there's a lot of focus on what you've done, not on what you would do or you could do. I imagine that's an important part of this. We're product people are in the business of changing behaviors. [40:39] Understanding and changing behavior. [40:41] And I think that's a really big mistake that teams make. [40:45] is they both in their prototype tests and in their interviews, they focus on what people would do, on what people think, on why they think they do something.

[40:53] and it's all really unreliable it's a garbage in garbage out kind of situation [40:57] The real measure is. [40:59] Tell me about your behavior. What did you actually do? [41:03] then we have to help people do that. [41:04] Something else someone asked that I really wanted to cover is, how does this process change as your company grows from early stage to later stage? [41:12] In an ideal world, it doesn't change. [41:15] Because here's why. [41:17] Like if I have a tree and they have an outcome and they're empowered to reach that outcome. [41:22] And they're interviewing every week, and they're assumption testing to evaluate solutions. [41:26] And they're, [41:27] finding things to build and they're driving their outcome. [41:30] That's a really successful team, and they could do that in a three-person company. [41:34] or they could do that in a 100,000 person company. The primary difference [41:39] is in a three-person company, there's no adjacent teams. [41:42] Right? [41:43] In a 100,000 person company, there's a lot of adjacent teams. [41:46] And so you probably have some dependencies to manage. [41:48] but you still should start with an outcome, be empowered to go after it, be empowered to come up with your own solutions. [41:55] What's going to be different in that 100,000 person company is you probably have like design patterns and libraries you got to rely on for coherent user experience. And you probably have another team that's working adjacent to you that you need to share your discovery work and be aware of what they're working on because you do need to build a coherent product. [42:12] And so as our companies get bigger, we have a lot of that lateral collaboration we have to do to make sure we're still building a coherent product. But I think the fundamental base unit stays the same.

[42:25] Something that I've seen happen with larger companies, especially as companies grow, is a little bit of cynicism of user research. Specifically, how few people you talk to and how that leads to you making a decision. How do you respond to those kinds of concerns? I love this. I don't know why product teams suddenly are held to a standard that nobody else is held to. Like when somebody says something like, "Why is it reliable to make a decision based on one interview?" I just flip the question around. [42:51] Tell me about the decisions you made last week. How many customers did you talk to? What data did you use? [42:55] Like, [42:56] Every human in business is making decisions with zero data. So I'm going to go with one is better than zero. That's a little bit of a flippant answer, but it's true, right? Here's what's happening when that question comes up. [43:08] I have an opinion that's different than yours. [43:12] I don't like your conclusion. [43:14] So I'm going to nitpick it. And in the product world, [43:17] Unfortunately, everybody in business has an opinion about what we should be building. [43:22] And so that's why we face that and we get held to the standard. I have a real reason why we can make decisions based on small data. We're in the business of changing behavior, not seeking new knowledge. [43:31] and we have really good feedback loops and so we can make decisions based on small experiments because we're going to continue to get bigger feedback loops and more reliable data over time and especially as we deliver and we do live production prototyping we actually can get large scale data i don't want to start there because we're never going to ship anything so there is like legitimately a valid reason why we can work on small data but [43:53] It's an unfair question because we're not holding anybody else in business to that standard.

[43:57] Along those lines, when does it make sense to run an experiment versus rely on user research? Do you have kind of a mental model for how you think about that? Yeah. Okay, so our language around this is terrible. It's so ambiguous. What's an experiment? What's user research? I would say experiments are user research. I'm trying to just dramatically simplify this. I think from a discovery standpoint, we have two core activities. [44:19] qualitative interviewing and assumption testing. [44:22] And so with qualitative interviewing, [44:23] We're trying to learn about the opportunity space. [44:26] Where do we see unmet needs, pain points and desires? [44:29] Interviewing is not the only way to identify opportunities. Observations are actually a better way. [44:34] I focus on interviewing because it's something we can do sustainably week over week. [44:38] most teams don't have the ability to observe their customers every week on the assumption testing side this is what we're any for me anything that helps us evaluate a solution [44:48] We're starting with a very specific assumption. [44:51] is an assumption test. [44:53] So we have experiments that I actually don't even think we should be running. [44:56] because they're testing the whole idea. [44:59] before we have any idea if that idea has a strong foundation. [45:03] They're taking too long, they cost too much money, they're taking too much time. [45:07] so how do i break this down like [45:09] The first thing is we have to learn how to take an idea and break it into its underlying assumption. [45:14] We have to learn how to prioritize those assumptions. [45:17] And then we have to learn how to run tests [45:19] that are small enough that they're just testing that assumption. [45:24] And this is all critical because it's what makes continuous discovery sustainable.

[45:29] Right? [45:30] I tell people to work with three ideas at once, and teams are struggling to test even a single idea. [45:36] So how's that sustainable? That team that's struggling to test a single idea is still stuck in project-based research world. [45:41] They're running experiments that take weeks to get results. [45:45] Whereas when I talk about assumption testing, [45:47] I'm working with a team that's running... [45:49] half a dozen to a dozen assumption tests in one week. [45:53] And those assumptions span three ideas. [45:55] The end of the week. [45:57] they can start to compare and contrast those solutions. [46:00] So we got to shift our methods. [46:03] right like continuous discovery is sustainable if we change our behavior if we change our habits [46:08] This is a good segue to the end of our chat. We've been chatting for about an hour. I know you have to run. For folks that want to learn more about assumption testing, continuous discovery, all the things that you've been chatting about, where can they... [46:18] find you online and find these courses online. Yeah, so first I'll mention the book, Continuous Discovery Habits. It's available at bookstores around the world. It's in Kindle, Paperback, and Audible. Kindle, EPUB, Paperback, and Audible. And then I do blog about all of this at producttalk.org. And our courses are at learn.producttalk.org. [46:35] Awesome. I also love to ask guests, how can listeners be useful to you? [46:39] since the book has come out the reaction has been unbelievably amazing and a lot of fun [46:44] And I'm a little bit overwhelmed by people in the industry who have never [46:49] been exposed to this way of working [46:51] having a lot of skepticism. [46:53] that it's possible. So here's how listeners can be helpful. [46:56] I didn't make up this way of working. [46:58] Right. [46:59] This way of working evolved from teams figuring this out.

[47:03] Like I see it as [47:04] I'm looking at how do I collect sustainable practices, making it as easy as possible for other teams to work this way. [47:11] So I think the way listeners can help me is if you've never been exposed to this and you have healthy skepticism, that's awesome. [47:18] And... [47:19] Just ask yourself, imagine if this worked. Imagine if this was possible because I get really tired. [47:26] Like, having to explain to people, [47:28] There really are teams that work this way. I'm sorry that you've never been exposed to it, but there really are teams that have worked this way. And if you've never been exposed to that, go look for it. There's lots of evidence of it on the Internet. [47:41] Amazing. I'm hoping our chat helps fight the fight for that changing of minds. Teresa, thank you so much for being here. I had a blast. I learned a lot. Thank you. Lenny, thanks so much for having me. This has been fun. [47:53] Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. [48:11] podcast.com. See you in the next episode.