The Problem With Document References and How Knowledge Ma...

The Problem With Document References and How Knowledge Management Fails Us

By Daniel W. Rasmus, Serious Insights

The other day a colleague called and asked about the best way to find a concept within an enterprise document repository.

After discussion, an enterprise repository was a bit of a stretch. What she meant was find the concept across the entire enterprise, regardless of where it was stored.

The target concept consisted of a set of management principles. In the current incarnation, the enterprise touted seven principles. After working with a number of consultants, it had honed its principles to five. The goal was to find all references to the seven principles and replace those references with the five.

I informed my colleague that there was no way to accomplish that task completely, even with the most sophisticated knowledge management system. I shared the following reasoning:

Structured documents do not structure around concepts; they structure around document organization, such as headings, tables, and other elements. This type of structure adds no value in a discovery task like this.
Search would likely find many, but not all documents, for many reasons, including inaccurate references (such as a reference to leadership principles rather than management principles, colloquialization, partial references (such as “in management principle number 7,” or “our principles”), and various technical issues like the precision and recall associated with the search algorithm. Also, because modern search engines typically seek relevance first, Intranet references that demonstrate use and inbound links may well appear, but those documents that live on their own don’t benefit from any social structure that the search engine would account for.
While most documents in formal repositories use tags, it is likely that the number of documents that reference the concept will be relatively small compared to the number of times those formal documents are referenced. Further, tags require their own management. The idea of “management principles” may not be an assigned tag, and if not, the tags will not contribute to the discoverability of the target documents.
Not all documents that reference the concept will be discoverable, such as downloaded copies, copies stored on removable media, copies e-mailed outside of the organization, or copies stored on servers not part of the enterprise index.

Document references: The reference problem

A reference problem quickly follows the discovery problem. Just finding the concept is not enough. The documents that reference the concept may reference it in several places and in different ways. A concept may be referenced in the text, and it may be referenced in a link (as a URL embedded in text that may not directly reference the content of the link). The concept may also be referenced in a footnote or endnote or perhaps in an illustration or table. While the “full official name” of it may be referenced for highly curated documents, the concept is just as likely to be referenced in some other way in less formal content.

The ambiguity of references makes it impossible to discover all the instances through search. The search set will likely prove incomplete in that all of the ways the concept was referenced were not maintained, and therefore, some documents will escape discovery because they reference the concept in a unique way.