Establishing thresholds for idea validation
When performing experiments for idea validation, the only way of preventing post hoc ergo propter hoc fallacies is to set the threshold that defines success beforehand (not too different from OKRs: literature/202112050837 Definition of OKRs.)
However, it is hard to establish a clear value that is not overly arbitrary. For example, we can ask a pool of people whether they would use our product. Does it matter if 10% answer yes? What if it would be 73%? As long as it is more than 0% then it is an indication we would have customers.
Of course, if the product relies on some network effect, below 50% may be too low. But for standard products, what is a common adoption rate? How many of us use Google Drive as opposed to Nextcloud (Installing and running Nextcloud on Raspberry Pi)? However, a smaller adoption rate does not make it an unfeasible business case.
These are the other notes that link to this one.