User-design indirection is a tremendous problem. It's a form of the principle-agent problem, or of what I've been noodling at under the general category of "manifestation", which is the idea that there are problems which are obvious and evident because they are manifest: immediately apparent, visible, obvious, observed, experienced, felt. And those which are not: nonapparent, invisible, non-obvious, unobserved, not experienced (in the future, by others, elsewhere), unperceived.
Nonmanifest problems are a substantial class of hard or big problems. Things which are outside your own experience, or the experience of administrators, managers, judges, or executives (in the broadest senses of the words) are really, really hard to understand, grasp, or appreciate.
(This is a key argument for diversity, real diversity, of experience and background on your team(s).)
Two specific examples, both trivial, though illustrative, come to mind. One was a flyover-state hotel I'd stayed at recently during a road trip. There were numerous obvious design flaws of the building, ranging from the entrance (no way to get a wheeled luggage trolly up the steps) to the interior layout (lack of lifts, floor plan) which strongly suggested some architect and design committee who'd never visited the site having designed the facility.
Another was a pool party years ago at which a guest suddently exclaimed that something had touched them, under the water. Nothing was visible. Then a second and third made the same comment. The culprit? A clear plastic drinks tumbler, which would oscilate when touched, given the mass of the water it moved against, giving a lifelike contact response, but being all but invisible in the pool itself. Until directly experienced, and ultimately revealed, the effect was difficult to credit.
Though the tumbler was invisible, and hence, not perceptible. You could go through any number of cases of phenomena throughout history, such as the germ theory of disease, prions, electromagnetic force, subatomic particles, or mental health, all of which have, at least by current reckoning, some level of cause or mechanism, but which were seen as beyond description or explanation in the past.
The case especially of moving from some sort of divine or moral explanation, as has been the case for both infectious and (somewhat still the case) mental illness is a critical one. Similarly for natural phenomena (comets, asteroid impacts, earthquakes, volcanoes, cyclonic storms), often seen in the past as portents of the gods / a god, rather than events with specific causal mechanisms.
There are several Big Problems currently facing humanity which are straddling this manifest/not-manifest cusp, with media manipulation and global warming being two of the more prominent.
Nonmanifest problems are a substantial class of hard or big problems. Things which are outside your own experience, or the experience of administrators, managers, judges, or executives (in the broadest senses of the words) are really, really hard to understand, grasp, or appreciate.
(This is a key argument for diversity, real diversity, of experience and background on your team(s).)
Two specific examples, both trivial, though illustrative, come to mind. One was a flyover-state hotel I'd stayed at recently during a road trip. There were numerous obvious design flaws of the building, ranging from the entrance (no way to get a wheeled luggage trolly up the steps) to the interior layout (lack of lifts, floor plan) which strongly suggested some architect and design committee who'd never visited the site having designed the facility.
Another was a pool party years ago at which a guest suddently exclaimed that something had touched them, under the water. Nothing was visible. Then a second and third made the same comment. The culprit? A clear plastic drinks tumbler, which would oscilate when touched, given the mass of the water it moved against, giving a lifelike contact response, but being all but invisible in the pool itself. Until directly experienced, and ultimately revealed, the effect was difficult to credit.