The curse of the silent assumption

There seems to be a recent surge in the entitled never-do-wrong corner of the world. Or rather it seems that the "fuck you if you do not share my opinion" crowd has finally become mainstream.

This has brought me back to thinking about two little facts that I found to be true not only in the domains I learned them to be true, but also in many other areas:

• Silent assumptions are the death of a complex system.
• Interactions between seemingly simple things can unexpectedly be fatal.

I'd like to beg your pardon in advance, since my best examples for both of these things are from knowledge representation logics, probably because I spent quite a bit of my professional life so far in that area.

Silent assumptions are the death of all complex systems

If you have heard a class about formal (knowledge) representation systems, you probably heard about the "frame problem". 

Let's assume that you have a robot R and two boxes A and B.

If you introduce an action move(box, loc1, loc2) that can move a box from a location to another, you also need to establish certain facts about that action that are circumstantial to the primary outcome (box at loc2).

In particular:

• after move(A, loc1, loc2), B remains at its previous location.
• after move(B, loc1, loc2), A remains at its previous location.

If you introduce another box C, you also have to state the invariant for C, same for D, .... At this point, it should be easy to see, that this gets out of hand quickly. In fact, it gets out of hand so quickly, that probably the most common mistake when modelling a domain is to forget at least one frame axiom.

You do not even need a mutable universe, to realize the problem, though. For example, if we model the hierarchy

Animal
\- Cat
\- Dog

most people almost immediately assume that Cat and Dog are disjoint classes. And then they never encode that (silent) assumption, because it is just obvious.

This is coincidentally one of the most common mistakes from Computer Science students starting with conceptual modelling. They, nowadays, usually have at least some background in object oriented programming. And in any OO language I know of, classes are disjoint unless they have a common ancestor.

It is just too easy to make this into a silent assumption and transfer that assumption into other formalisms. And hence there are probably thousands of CatDog ontologies out there, where things that really cannot be the same thing are allowed to be.

Us humans are so very good at making these silent assumptions, that we tend to unconsciously skip over them. Obviously, when moving box A, B doesn't get moved. Unless A is the heavier of the two boxes, was initially on a seesaw, with B on the other side.

You get the drill.

Let's face it: You cannot avoid silent assumptions, because then you end up with what I call "world transformer" functions that have the whole universe in their parameter set and return a different, modified universe from that.

But here's a bit of well meant advice: Avoid making assumptions about the obviousness of silent assumptions. That stuff usually comes back to haunt you with a vengeance.

Interactions between seemingly simple things can unexpectedly be fatal.

This is also something I picked up the hard way from knowledge representation systems. If you have a System S and add feature A, the combination SA is totally fine. So is SB for some feature B. But if you make SAB, the whole thing falls apart and becomes undecidable.

Don't believe me? Let me quote two people quite a bit smarter than me [Branchman83]:

We illustrate how great care needs to be taken in the design of a representational facility, even when our intuitions about the language tell us that it is a simple one. As it turns out, even an apparently modest representation language can prove intractable.

Yeah. If you combine two innocent looking things, the result may no longer be that innocent.

Note how that combines with silent assumptions, where the interaction of two programs looks totally innocent to the casual reader, but if you add in the silent assumptions, you get a total, unmaintainable mess.

Or in good old Dijkstra's words [EWD340]:

The competent programmer is fully aware of the strictly limited size of his own skull; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague.

Another piece of well-meant advice: When you build a system out of parts, very carefully control the interactions of those parts.

Also from personal experience: The less people actually know and the less self-esteem they have, the harder they stick to their silent assumptions.

[Branchman83]
The Tractability of Subsumption in Frame-Based Description Languages (1983)
Ronald J. Brachman, Hector J. Levesque
Proceedings of AAAI-84
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.464.7663

[EWD340]

The Humble Programmer

Edsger W. Dijkstra

ACL Turing Lecture 1972