In this episode of McKinsey on Building Products, McKinsey partner Rikki Singh speaks to Tamar Yehoshua, president of product and technology at Glean, an AI platform that helps automate work by connecting and understanding a company’s entire knowledge base. Throughout her years of experience at large software companies, Yehoshua has learned how to build digital products that customers love and how to approach the change management needed for employees to feel comfortable using new tools in their roles. In their conversation, Singh and Yehoshua discuss lessons in product leadership and the future of work with AI. An edited version of their conversation follows.
Building products with curiosity and instinct
Rikki Singh: Tamar, tell us a bit about yourself.
Tamar Yehoshua: My background is mostly in product and technology, which has been an exciting area to be in now during the spike of generative AI. I joined Glean in March 2024, and prior to Glean, I was a chief product officer at Slack and spent many years working at Google Search and Amazon.
Rikki Singh: How have those experiences shaped your philosophy on product management?
Tamar Yehoshua: At each company, I gained something new. When you shift companies, you can take the things you like and leave behind the things you don’t. Besides the customer centricity, both Amazon and Google are very metrics-oriented—they understand what their customers are doing at scale. Companies with a billion users can’t talk to everyone, so they have to understand data and balance qualitative and quantitative data. Slack, on the other hand, was different in the level of craft and obsession to detail, which gave me a new appreciation and understanding for product craft.
Rikki Singh: What metrics or data should PMs [product managers] be looking for?
Tamar Yehoshua: I think it’s important to be data-informed but not data-driven. You don’t want every decision to be purely based on data because you have to have an instinct; you have to understand your customers and know what they need. Sometimes the data doesn’t show you the way. Sometimes changes you make could have a short-term negative impact but are better for the long term. You also have to be careful in picking your North Star metric because it could easily lead you in the wrong direction. At Glean, for example, we want to see activity metrics and make sure that people are using the product and finding value in it.
Rikki Singh: What skills can help a product manager be successful?
Tamar Yehoshua: Curiosity. Sometimes product managers think they need to have the answers to everything and come in with a fully baked strategy. That’s not what the job is about—it’s about listening, learning, and talking to customers, stakeholders, and your team because everyone has a different perspective and something to bring to the table. That ongoing curiosity to learn, perfect, and iterate will help you understand that there’s more you can do for your customers.
I think it’s important to be data-informed but not data-driven. You don’t want every decision to be purely based on data because you have to have an instinct; you have to understand your customers and know what they need.
Adapting to the pace of AI and embracing change
Rikki Singh: You’ve interviewed many founders and PMs who are innovating with AI. Could you share highlights from those conversations?
Tamar Yehoshua: Between working for Slack and Glean, I worked as a venture partner for almost a year. I was interviewing practitioners and start-up founders. All of them had the ability to recognize that the tide is shifting and they could act fast. For example, when ChatGPT came out, there was a panic because it was changing our road map and we were shifting in a big way. We knew it was going to change everything. What differentiates the people who are forward-thinking is their ability to see something that is changing and make bold moves to throw out the original plan.
Rikki Singh: What is your perspective on AI’s value and impact?
Tamar Yehoshua: I think AI has some similarities to the rise of the internet, cloud computing, and mobile phones, but the change is happening faster because it’s easier to adopt the technology. For example, with the move to cloud, companies had to build a whole new infrastructure to adopt it, and that was hard. Now, you can add AI to your existing products. Big companies are adopting it faster than I’ve seen them adopt other technologies.
Rikki Singh: What are the business use cases that would benefit most from this technology?
Tamar Yehoshua: There are several different classes of technology where AI is going to fit in best. The first is replacing simple job functions. For example, it can answer frequent questions, such as “How do you change your password?” The second is augmenting more-complex job functions, such as completing code for programmers. The third is automating something. These cases will be the majority over the next couple of years. For example, at Glean, we can use AI to assess calls across our customers to see which features come up the most in our sales calls, put them into a spreadsheet, and prioritize customers by revenue. There are so many tasks you can make faster or automate to make your job easier.
Rikki Singh: You are in a unique position because your customers are the ones adopting generative AI tools. What are some common challenges enterprise customers face when they’re trying to adopt AI tools?
Tamar Yehoshua: The challenge is that a lot of leaders are ready to bring in these tools but the organizations aren’t. So they’ll bring in a tool and not know how to get people to use it. Getting people to change their behavior is harder than you think. For jobs that have a top-down mandate to integrate AI, such as customer support, it’s much easier to see what exactly you have to do, so those employees tend to adopt it faster.
We’ve been building features into our product to make it easier for companies to adopt AI. For example, when people see an empty chat box, they don’t always know what to do with it. So we started enabling organizations to create prompts, save them, and then share them. If an employee goes into the company’s Glean, they will see the prompts their organization is using and can observe which prompts other people find helpful. This feature has had a great improvement in the utilization of the platform. The hardest thing is getting people to recognize that their jobs will be better and easier with this new technology.
Rikki Singh: Are there things that enterprise customers could do to overcome these challenges internally?
Tamar Yehoshua: Lead by example. I always advise executives to use an AI product and send out a note to the organization that ends with “written by Glean” to promote that you used AI to help you. Eventually, that visibility will help normalize AI within an organization and set up objectives around it.
Rikki Singh: For some generative AI tools, there is a skepticism around data privacy compliance. How can leaders ensure that they’re using gen AI responsibly?
Tamar Yehoshua: Sometimes these nerves are based on what people are hearing rather than how their product is architected. Every AI company has to understand what the risks are. Every company should also take the risks seriously and understand what the CISO [chief information security officer] is worried about. That requires the whole organization to understand how to avoid leaking any information to the models or through prompt injection.
Talent in the age of AI
Rikki Singh: Based on our research, we believe that AI will fundamentally transform the talent mix and capabilities among developers, product managers, and product designers. What is your perspective on this?
Tamar Yehoshua: There’s no reason to think that the way we’ve delineated the roles of engineer, product manager, and designer is the optimal and best way. We should be iterating. Over the years, we’ve had designers who are more so design engineers, and we’ve had product managers prototyping. With AI in the mix, it’ll be easier for a product manager to code, for a designer to generate code, or for an engineer to write a product requirements document. The lines are going to blur between roles, and I think that’s exciting.
You’re still going to need somebody defining the product, writing the code, and designing it, but AI might shift how these tasks are being done, who can do it, and how many people you need. Engineers might also work more cross-functionally or with different engineers. We’re already starting to see this blend in start-ups: They’re hiring fewer people in the different functions and having more overlap.
Rikki Singh: Is the increasing prevalence of chief product and technology officers [CPTOs] or chief product officers hinting at this blending?
Tamar Yehoshua: I had never heard the term CPTO until around a year ago, and now I hear it everywhere. More people are recognizing that having a single decision maker who is responsible for both areas makes things move faster. It is a relatively new trend, but I do think it is a direction we’re going to keep going toward in the future.
Building AI-centric products
Rikki Singh: Is there a difference in building products that are generative AI–centric versus other products?
Tamar Yehoshua: The pace of putting out products is faster than I’ve ever seen, but the framework for building them is not that different. The biggest difference is that AI is nondeterministic. I’ve been working with machine learning for years, and there wasn’t a generative answer that could change. When you check AI for quality assurance, there is a different set of evaluative tools and a different kind of product experience. Also, users don’t know how to fully use generative AI yet, so we’re working with a technology that is new to users.
Rikki Singh: A lot of organizations are stuck in the experimentation phase with AI. How do you see that evolving? Is there a path to scaling?
Tamar Yehoshua: A lot of companies use OpenAI for something and get a demo out, but they can’t productize it because it can’t scale. They don’t know how to measure the quality, and getting a product into production is hard. It’s easier to build AI applications on top of platforms that are more mature and can handle the data and the security issues. We work with our customers to make sure they can get to production and can see the value in their investment.
Rikki Singh: You touched upon quality issues and the fact that in an under-domestic model, companies have to be more thoughtful about the scenarios they test for. Are there tactics you use as an organization that others could learn from?
Tamar Yehoshua: We use the large language model [LLM] as a judge. We have a system that uses LLMs to validate the answers of the LLMs and measure them on things such as groundedness—for example, does it know what article was linked or if the answer is correct and complete? Once we have a repeatable way of evaluating the answers, it is much easier to test out new prompts and applications.
Rikki Singh: In terms of productizing AI, CXOs [chief experience officers] often cite pricing as a challenge. What are your thoughts on that?
Tamar Yehoshua: It depends on your product. You have to understand the cost of your product to understand what you need to charge, but the cost is changing rapidly. The cost of LLMs has gone down significantly, even in the past year. My prediction is that AI will be built into every product at one price, rather than SaaS [software-as-a-service] apps charging extra for AI capabilities.
We have a system that uses LLMs to validate the answers of the LLMs and measure them on things like groundedness. . . . Once we had a repeatable way of evaluating the answers, then it was much easier to test out new prompts and applications.
The features that make a real difference
Rikki Singh: What is the most counterintuitive thing you’ve learned about building products, from your breadth of experiences?
Tamar Yehoshua: There are few things that make a difference in products. There are so many features you can build, but there is a full graveyard of features that I built that nobody ever used, and there are only a couple of features that really matter. Few things change the trajectory of the business, and as a product leader, I try to focus on the things that will make the biggest difference.
Rikki Singh: How do you find the ones you think will make a difference?
Tamar Yehoshua: It’s not necessarily in the data; you have to have intuition and understand what your customers are trying to achieve. You have to understand customers’ day-to-day life and what is going to make the biggest difference for them. There’s no shortcut or formula. That’s why being a PM is more of an art than a science.
Rikki Singh: Do you spend a lot of time learning from customers?
Tamar Yehoshua: Yes. I think it’s important that PMs do. But you also have to have a vision for where the industry is going. It also helps if you use your product because then you can understand how the product works.
Rikki Singh: If you could leave our audience with one parting thought, what would it be?
Tamar Yehoshua: In the age of AI, be curious. You have to try the tools; you can’t just read about them. Try all the latest products because it will change how you view AI. That curiosity and willingness to try is important no matter what your job is or what stage of career you’re at.
“Our firm is designed to operate as one—a single global partnership united by a strong set of values. We are equally committed to both sides of our mission: attracting and developing a talented and diverse group of colleagues and helping our clients create meaningful and lasting change.
From the C-suite to the front line, we partner with clients to help them innovate more sustainably, achieve lasting gains in performance, and build workforces that will thrive for this generation and the next.”
Please visit the firm link to site