Hierarchies Are Overrated

by Curt

I’ve always had a disorganized office.  For me, it starts with the file cabinet, and that moment when I am holding a home escrow statement that is talking about my home insurance.

Should I put it in my home insurance folder?  What if I later have an escrow question?  Will I remember where to look for it?  I have an escrow folder from when the escrow account was created, but the document seems pretty insurance-related.  I could combine it all into a home mortgage folder, but is that too general?

At that point, I tend to give up and put the document on an unsorted pile, in some corner of my office.

The basic problem is that I have a physical item – a paper statement – and even though I can think of multiple ways to categorize it, I can only put it in one folder.  I can group folders together into one category folder, but they can still only go into one category folder.

That’s basically what a hierarchy is.

In this sense of the word, hierarchy1 means that an item can only have one container (or parent).  Containers can be nested, and many items can be in one container, but each item can only be in that one container.  One file cabinet; multiple folders.  One folder; multiple files.  Very structured, very orderly.

This is not how people think.  We do tend to think in terms of bundles of knowledge, but each item of knowledge we have might be contained in several bundles.  Our bits of knowledge and information link together in all kinds of directions.

Hierarchies, however, are common in the physical world.  A piece of paper can only be contained in one folder.  An article of clothing can only be put into one suitcase.  My heart is in only my body.  And this has conditioned us to try and apply hierarchical models onto other pieces of information.

There’s an old saying: “A place for everything, and everything in its place.”  It’s a comforting sentiment, isn’t it?  It makes me think of a matronly woman in a large house where everything is proper.  It’s kind of an archetypal fantasy, the Mary Poppins that knows just where everything goes and what everything should be.

Unfortunately, there’s a trick to the Mary Poppins illusion.  It is a result of a lot of very tedious work, and – more importantly – it assumes that every new thing that appears will fit comfortably with what is already there.

What really happens with knowledge?  It is learned or discovered.  It is analyzed, and synthesized.  It has to be reconciled with what came before, and sometimes it can completely change your existing understandings, busting apart your existing paradigm.

However, as a hierarchical structure grows, there is a growing cost to fitting new information into it.  As it gets larger, when a new element appears, it might require a new container or parent.  Sometimes, an entire branch might need to be moved or reorganized, and sometimes, the entire system has to be reworked.  It can get very unwieldy.

If you discover a new fact that doesn’t fit easily into the hierarchy, what do you do?  It is expensive and inefficient to reorder an entire hierarchy for every new piece of information.  And as more and more knowledge is discovered, it gets harder and harder (and slower and slower) to integrate.

 

What is the result?  People cope poorly.  They let the new knowledge pile up without synthesizing it, or they force it into the hierarchy ineffectively, or they simply stop gathering the new information.

And along the way, wisdom is lost.  People fall behind, sticking their heads in the sand, or they fail to realize implications of new knowledge.  Wisdom is lost, and evolution slows down.

What is to be learned from this?

Hierarchies have their place.  Hierarchies can be useful for bodies of information that are balanced, largely known and largely static, and when there are clear compositional units – parents and containers.  But hierarchies are a bad fit for any expanding or changing body of knowledge.  They’re not only a poor fit, but they can inhibit the growth of that knowledge.

  1. The word “hierarchy” unfortunately has many partial synonyms, all of which can be abused – taxonomy, classification, etc.  Here we specifically mean a “compositional” hierarchy, where each item can have only one parent.