Answers to questions can easily fit into two flavors: operationalized and free-form. Classify the use cases: there’s the questions you know how to ask, and the questions you don’t know to ask yet.

A question that you know how to ask is operationalized. You’re looking for yes, no, or broken, or perhaps a count. The operational nature of the question means you can improve operations of your system by using the questions. 

You may need to use a series of operationalized questions to drill down to success.

* A good operationalized pattern uses multiple questions to reduce the target set at each step: “are there systems with problems?” -> “there are systems with problems” -> “Show me the systems with the problems” -> “here are the types of problems on the systems with the problems” -> click “Show me the ones with the problem I know how to fix” -> “here are the systems with that problem” -> “deploy a fix for that problem”. This is good because it’s efficient: each step is small, and each step is hitting fewer targets. Whether your problem domain is management, computation, or surveying humans, you’ll use fewer resources if you ask fewer questions of fewer targets.

* A bad operationalized pattern == “give me all the data and i’ll search for answers in it”. This is misuse of an effective tool: search through raw data is powerful for discovering what you don’t know to ask, but it can be the wrong tool for daily repetition tasks. It works, but it costs more time and money than necessary.

Noted, it is possible to take the progressive questions pattern entirely too far, as is shown by Microsoft “click to see more” Teams. A forced wizard flow where it isn’t necessary is an anti-pattern. Progressive disclosure of necessary data can become an anti-pattern.

A question that you don’t know how to ask is free-form. You’re looking for weirdness, patterns, outliers, intuition. Is there an anomalous behavior pattern on a subset of systems? That’s hard to answer without a big data lake and a Stats101 textbook, so you stream data at the lake and see what kind of stuff can be found. Algorithms can help, but you’ll also certainly need human analysts. And the findings from that data lake, you will probably want to convert to operationalized questions.

This is a lifecycle of discovery, a process of learning. Operationalized questions grow stale over time, and need to be replaced. Part of the job as an analyst is to maintain the tools.