Persistence and Flexibility: What Analytics Can Learn from Physics
I’m pretty certain that any philosophical text carries a hidden instruction set which directs the human brain to <loop infinitely> until a self-preserving interrupt command is issued on our behalf by some other part of the brain. Whether for infinite loop torture or mental floss, I do look for and sometimes come across interesting philosophical parallels with work applications. I recently found one worth sharing.
The Lowest Granularity
To set the stage, it’s worth considering a common question that arises in every business intelligence project that often proves not only difficult to answer but also carries a significant long-term impact on the system which proves difficult to change: the question of granularity.
For any entity in a data model, the granularity is the level of detail at which we will ascribe and record attributes and observations for that entity. For example, the concept of a product might be comprised of sub-assemblies which in turn are comprised of still smaller sub-assemblies, some of which might be developed by other manufacturers. What can we know (or efficiently track) about a product when we think of it in this hierarchical way? Other examples include all kinds of external entities like customers or suppliers. The decisions about granularity, or even what kinds of things we want to track and analyze in our business intelligence systems can present huge challenges.
Relations, not Entities
A similar problem dogs physicists who seek to model the nature of the real world at a miniscule scale. In “What Is Real?”, an article in the August 2013 issue of Scientific American, a radical new framework is presented suggesting that the physical world, at its finest granularity, is comprised not of tiny particles but bundles of properties like color and size. In a nutshell, the argument goes something like this: theories of both particles and particle fields are full of paradoxes and don’t stand up as a hierarchical model (or ontology). One of the more interesting paradoxes cited is that of a vacuum, where the average number of particles is zero yet the vacuum shows that particles are being created and destroyed seemingly from out of nothing.
The answer, as the framework suggests, is that the most basic constituents of the material world are intangibles (relations) alone. To get things started, the concept of structural realism is introduced, which says that we may never know the true nature of things, only how they are related to one another, as is the case with mass which is a relationship between two bodies, but that mass itself cannot be seen. The framework leads to a more radical conclusion that ultimately removes the bodies themselves. I leave the rest up to your curiosity.
Persistence and Flexibility
In sharing this whopper of a brain loop, I make no claims as to the implications or my understanding of the content. I will suggest that any data modeler reading this is already thinking about the modeling implications, but I want to focus on some interesting lessons that can be learned and applied to the field of business intelligence: persistence and flexibility.
The first lesson of persistence is derived easily enough. In considering the backdrop to this story, whatever we have learned of the seemingly unknowable carries an enormous history in terms of persistence. This can be seen not only in massive infrastructure like particle colliders in the U.S. and Europe, but also the invention of instruments springing from the efforts of individuals, like the Geiger counter, an inexpensive device that detects particles. The common thread is a dogged pursuit of knowing the seemingly unknowable. In business contexts, there are obvious limits, but a good dose of persistence should lead farther down the path when we come across difficult challenges in the detail and scope of our data.
Still, roadblocks are ever-present, which leads to the second lesson: flexibility. In the context of business intelligence, flexibility, or the lack of flexibility, shows up not only in our way of thinking, but sometimes in the tools we use. Like a relational database, most data tools require segregating concepts into things and relations in a model. New tools like Hadoop and semantic technologies are expanding the range of possibilities and adoption will be worthwhile in terms of what kinds of questions we can answer with analytics and the efficiency with which we can answer them. My point here is that we should expand our data toolkits to accommodate different kinds of problems.
Then there is the way we think about whatever it is that we are analyzing. Can a model that seems to be broken be turned on its head, or can we take a completely different perspective? In the article, removing the entities that are clouding the understanding and focusing on the relations instead is not only a great example of flexibility but also a bold position to take. The author makes a strong point about why we should bother. In the case of understanding the physical world, it is so that we can get to the next theory, ultimately leading to higher knowledge. The point is further made that just about every framework across history has eventually been scrapped or revised to make way for a new one. What possibilities will this kind of persistence and flexibility bring to those things we are now content to call unknowable?
Jared Decker has over twelve years of experience in the IT industry and eight years of consulting experience focused exclusively on data warehousing and business intelligence. He has been instrumental in the successful delivery of many projects, enabling organizations to achieve improved operational effectiveness through the timely availability of critical decision-making information. His breadth of experience entails everything from business data analysis through design and system implementation. He received a B.S. in Management Information Systems from the University of Tampa and an M.B.A. from the University of Houston. Mr. Decker holds several technical certifications related to database and corporate performance management systems.
You can subscribe to our RSS feed.