As product managers, what do we do about the demand for large language model features?
The problem: our discipline has coalesced on a concept of creatively solving real customer needs. Our textbooks call out Build Traps, Jobs to Be Done, and even Empowered and Inspired Transformation. We prefer doing lean development in lockstep with real customer demand, teased out though careful observation instead of pulled from our imaginations.
Unfortunately, there’s not a lot of user demand for spicy autocomplete. It’s a good technology for a fairly narrow set of use cases, but it’s not the pillar of a new era that is being promised.
- style help and content summarization for non-native English speakers, developers, security researchers.
- suggested things to try when you’re writing something (code or otherwise).
In real life, the tech offers copilot assistance, which a human must still use with their contextual knowledge and factual awareness. That is super quick to build in, especially if you can outsource the whole set of work to OpenAI. Of course, that’s only feasible if your problem domain is in the set of things they trained ChatGPT on. For instance, that was trained on StackOverflow for its coding. if you’re looking for help with a DSL (Domain Specific Language) that isn’t SQL (Structured Query Language), SPL (Search Processing Language), or some form of Python… you’re going to need to make your own SALAMI, which consumes more time and resources than the folks making demands might expect. Make no mistake, the demand is huge, even if it’s not directly user-driven.
The problem is that we need to do product things that aren’t customer driven, in other words. Let’s break that into two parts.
First, should we do this thing with low customer demand? The answer is undoubtably “yes”. There are positive reasons for this position: it is accurate to say that innovation doesn’t have a market yet, research needs to sprint ahead of demand, &c. “Genius aims at a target that others do not see.” We can all name technologies that seemed unimaginable or fantastic until they were built, and while potential customers might have expressed interest in Jobs to be Done terms, they certainly weren’t asking for that exact solution. A shining case in point: the capacitive touch on-screen keyboard. Smart money was on chiclet keyboards until Apple got it right.
However, even if you disagree with all of that, there are less positive reasons: markets are fashion driven, and the boss said so. Product management is a creative and analytic discipline, but it falls within the corporate segment of capitalist society. While our individual votes certainly matter at the ballot box, our opinions are just that at work. It’s true that the tenets of our discipline are calling for data-driven product decisions, but data isn’t the only driver of what will happen. We are employees who serve at executive pleasure, and all the organization “wants” is to grow; building well-designed solutions to real customer needs is only one of many means to that end. Ideally it is the preferred means in our organization and none of these depressing and demotivating truths usually need to be discussed! But if you find yourself arguing with an executive about whether an AI feature is necessary, you might be spending that breath unwisely.
On to the second part: it’s been decided, so how do we do the thing? Here, the basics of product management come back into play. We release early, we release often, and we gather feedback in search of higher value outcomes. It is important to note that we don’t give up though. Where a different project might get a small investment and a few months to prove itself, this project will persist until the conditions that led to the decision change. User feedback, usage statistics, and cost data will be very useful tools for informing any sort of change, so this is an excellent project to beef up your capabilities on those fronts. If there was ever a place to collect lots of data, this is it.
Monkeynoodle.Org 