The Zen of Systems and Networks

zencircle01

My own work with Enterprise Frameworks and Networks has led me to come up with the following table.  It describes the Nodes and Links in a Complete System Network.  I am saying that the Nodes representing Goals, People, Time, Locations, Code, Data, Qualities and Quantities can all be represented as Scale-free Networks and that each of these Node Networks require only one datatype.  I am also saying that there are only three types of links in networks: recursive links within a set, multiple links between sets, single links between sets.  I know of no case where this has been attempted in the manner I am attempting to represent it.

systemnetworkslinks1

If you have been following my blog you are aware that I have been struggling for a long time to come up with a framework and a clean terminological set to describe systems.  I think I have come one step closer to that goal today.  The table above describes a Fact composed of eight Nodes (first white row illustrating entities) and the Links (last three white rows illustrating recursive, multiple and singular relationships) for each of the System Networks (Interrogative columns).  One of the interesting aspects of this System Network Model is every Fact is composed of a Unique Set of all eight Nodes.  However, all the Nodes in one Fact do not have to have Links to all the Nodes in another Fact.  Each Node within a Fact is independent regarding its Links.  Therefore you have a single set of System Facts with each Fact containing a single set of Interrogative Nodes each connected by their respective Link Networks.

I have recently been writing with the intent to challenge centrism on any one of these networks and advocate a more integrated view. I still remember dealing with data centrism, event centrism, user centrism, goal centrism, program centrism and schedule centrism over the course of my career. All of them have a role to play. My insight into all of these Nouns being Linked by Verbs in only three ways required me to look at all of the Enterprise Architectures and disengtangle the Nouns, Links and Verbs from the reasoning and representations that extend back beyond computing itself.

The Data Model below is a hybrid of Relational Models and Dimensional Models.  I call this an Associational Model.  It is using Relational Architecture to represent it.  However, I think that an alternate Entity-Attribute-Value (EAV) architecture called the Associative Model of Data would be better suited to the task.  I am using relational representation as I am still trying to communicate with a community only familiar with Relational technology.

The first thing to note about this model is Links are represented by Associations.  Associations link two Nouns using a Verb.  What is interesting about this model is every Verb, Association, Noun and Fact is unique.  The vertical connections are Many to Many relationships which allow two vertically adjacent Verbs, Associations or Nouns to have multiple unique relationships between each other.  What this means is there are no integrity problems (duplicate values) as the system network would enforce uniqueness.

amdschema031

The premise of this model is that the Nodes are not dimensions at all.  I am rejecting the traditional concept of dimensionality instead I am saying that there are three dimensions of Links: recursive, multiple and singular.  All we perceive are Facts, Nodes and the Links between them.

So you could come away with the following Zen koan:

entity without entity,

source without source,

path without path,

target without target,

size without size,

dimension without dimension.

Netular Technology versus Psuedo-Netular Technology

fishingnet

I have been having a very interesting discussion on Linkedin.com having expressed my opinion about current information technology and the netular  information technology I would like to see.

The people who have been exchanging their views with me cannot see the forest for the trees.  One is offended that I do not rave about all the social transitions the technologies are offering.  Another spews buzzwords like a chainsaw.  Another assumes my opinion is a product of my impatience for the convergence of the existing technologies.

Einstein once said he would spend a majority of his time defining a problem and a fraction of his time solving it.  A majority of the time on information technology is spent solving and a fraction actually taken to understand.  The consequence is most of the solutions out there are not designed, they are hastily assembled patchworks that because of the inertia of being first on the field are only replaced by further patches.

Our entire system of networks is built upon a foundation of linear and tabular architecture that is present in our CPUs, memory, storage, data structures, programming languages, organization, locations, events and goals.  In reality we are only dabbling in networks and doing an abysmal job of using them to their full effect.  We don’t understand them.

Marshall McLuhan said that when a new media is created the first thing we do is pump old media through it.  That is what we are doing now.  We are taking every form of old media we have and pushing it through the internet.  There is not a single case where we have successfully departed from linear and tabular old media.  I have looked at all the current technology, I have used it, I understand its internals and I stand by what I say.

We need a fundamental change in the way information technology works otherwise we are going to continue with an undesigned brute force attempt to solve our problems without ever understanding them.  The outcome will not be progress, but the perpetuation of flat earth thinking.

Linear and tabular thought are responsible for many of the problems we have in the world.  The biggest is the inability to fully appreciate the uniqueness of everything and everyone in this world.  The supreme example of this has been the long history of Religion, Genocide, Slavery, Nationalism, Imperialism, Racism, Eugenics, Fascism, Nazism, Communism, Marxism, Capitalism and Socialism.  All of them fail us because they depended on linear and tabular models of thought that denied the respect of the individuality of all experience.  True netular thought has the potential to challenge all of these misconceptions.  I think it is appropriate that this transition is on the horizon with the rise of globalism.  I doubt it will be a peaceful transition.

Actually, the insights into the underlying order in networks has made quite a bit of progress. One of the leaders in this area is Albert-Beszlos Barabasi who authored the book “Linked” http://www.nd.edu/~networks/Linked/index.html . Another researcher Kwabena Boahen made a fascinating presentation at TED http://tinyurl.com/6nnkb7 . There is also the work of Simon Williams that has come up with a new associative database architecture http://www.lazysoft.com as well as a commercial product, Sentences.

It is time for everyone to fundamentally change the way they think.

Databases: Structured Associative Model

oraclesentences

For years now I have been struggling with Relational DBMS technology and Associative DBMS technology attempting to get them to do what I want.  In my first efforts, Relational models were structurally restrictive, Dimensional models were unable to grow organically, EAV models are incompatible with relational architecture.  I came upon Simon Williams Associative Model of Data and although enthralled with its potential I found it too had limitations.  It was semi-structured and allowed for too much flexibility.  25 years in Information Technology had taught me that there was a single standard classification system for setting up databases not a plethora of ontologies.  I was determined to find the theoretical structure and was not concerned with hardware limitations, database architecture, abilties of current query languages or any other constraints.

The Associative Model of Data had made the difference in liberating me from Relational and Dimensional thinking.  A traditional ERD of the Associative Model of Data I at first thought would look like the following:

amdschema

Basically what you have is a Schema composed of Nodes with Node Associations through Verbs and Associations with Nodes Attributions through Verbs. The range of Node Entities, Verb Entities, Association Entities and Attribution Entities are endless.  As well the population of the Schema has an unlimited dataset of natural key values.  I have been challenged by Relational database specialists and SQL experts regarding the viability of this model within current limitations, however their arguments are irrelevant.  What is important is the logical validity of the model, not the physical validity.

After receiving the criticism I decided to revisit the model in order to simplify it.  I went over Simon William’s explanations of his model and its application and found I could reduce it to the following:

amdschema02

This was profoundly simpler and better reflected the Associative Model of Data’s Architecture.  But even with this simpler architecture I was not satisfied.  I felt that the Associatve Model although giving the benefit of explicitly defining the associations was a tabula rasa.  Research has shown that tabula rasa’s are contrary to the behavior of the finite physical universe.  There is an intermediate level of nature and nuture.  And this is what I sought to model.

zachman

When I first encountered the Zachman Framework, something about it struck me in a very profound way.  I could see there was something fundamental in its description of systems, however I felt that the metaphors that John Zachman used were wrong because they themselves lacked a fundamental simplicity.  The consequences of this were that those who studied under Zachman ultimately could not agree on what he was talking about.  Also the “disciplines” that Zachman’s Framework generated were continually reinventing the wheel.  Zachman had created a world of vertical and horizontal stovepipes.  To further the confusion Zachman refused to conceive of a methodology based upon his framework.  Consequently, there was no way to determine what the priorities were in creating a system.  I call this the Zachman Clusterfuck.

Zachman’s work spawned years of work for me.  I could see that systems had a fundamental structure, but I could not agree with Zachman.  Focuses and Perspectives were useless terms.  The construction metaphor was useless.  I read anything I could get my hands on dealing with systems, methodologies, modeling, networks and a broad range of other literature across the disciplines.  Out of this came a set of conclusions:

  1. There were a fundamental set of Noun Entities
  2. There were a fundamental set of Verb Entities
  3. There were a fundamental set of Association Entities
  4. There was a clear order in which the Nouns were addressed
  5. There was a clear order in which the Verbs were executed
  6. The structure was fractal
  7. The content was a scale-free network

I made some attempts at creating the vocabulary and experimented with this new Structured Thinking Language.  However, the real break came when I worked with John Boyd’s OODA Loop:

theboydpyramid

The OODA Loop revealed a governing structure for the methodology and guided my way into the following hybrid relational/dimensional/associational model I call the Structured Associative Model of Data:

samd

One of the key things this model demonstrates is the sequence followed by the OODA Loop.  Starting from the top, each dimension set spawns the next.  Choices are created from the dimensions.  There is no centrism to this model which is an inherent flaw in Service Oriented Architecture (SOA), Event based architecture, Data centric architecture, Goal-Directed Design, Rule based systems among others.  The stove pipes of Focuses and Pespectives disappear by reasserting a clear order of priorities and dependencies for achieving success.  The model also supports bottom up inductive as well as top down deductive sequencing.  This will make the system able to reconfigure to handle exceptions.

Some of the things I have learned in designing this model include the realization that unit defines datatype and that all measures are variable character string text.  This is because any displayed value is only a symbolic representation of the actual quantity.  If operations are to be performed on measures they are converted to the correct type as part of the operation.  I also recognized that Unit was necessary to define the scale and scalability of the system.  Further, it became apparent that analog calculations should not be practiced.  Every value should be treated as discrete and aggregated.

Another aspect of this system is the inclusion of currency and amount.  I have been critical of Zachman and academics for their hypocrisy regarding the economics of systems.  All systems have a cost and a benefit and they are measurable in currency.  Contrary to the reasoning of the majority, every decision is ultimately economic.

Tim Brown of IDEO has coined the term “Design Thinking” and has been toying with the concept for some time.  Many designers dwell on the two dimensional concept of divergence and convergence as modes of thought.  If we look at my model, divergence is the creation of choice while convergence is selection of choice.  There is no alteration or deletion of choice in my model as history is preserved.

Now what you have is a unencumbered framework with a clear methodological sequence.

czerepakcognitary

Welcome to the Cognitary Universe.

SQL: Old Soldiers Never Die

Structured Query Language (SQL) has been a phenomenally useful language for the relational database era. But I see that era coming to a close.

One of the primary flaws is SQL allows for database Alters, Drops, Updates and Deletes. When diskspace was expensive this made perfect sense, but with the unlimited disk resources we have today a greater principle holds true: NO SCHEMA OR DATA SHOULD BE ALTERED, DROPPED, UPDATED OR DELETED.

A second flaw is the lack of interactive modification of the schema in real time. Changes still blow most applications all to hell.

A third flaw is supertype/subtype hierarchies. Such things should not be hard coded into a design.

That being the case SQL has four unnecessary statements just waiting to be abused. We need a better language. In fact, we need a better database architecture.

A new language would provide no means for updates or deletes. I created the first Releases of this language I called “Structured Thinking Language” (STL).

STL has the following commands:

  1. CREATE – affordance concept (creates entities)
  2. DIRECT – affordance context (relates entities)
  3. POSIT – affordance method (entity output)
  4. OBJECT – affordance pragma (entity input)
  5. NEGATE – affordance cosmos (entity security)
  6. INTUIT – affordance chronos (entity manipulation)

As you can see there are no means to delete data.

Each entity (noun) has only one “attribute” in the relational ERD sense and each entity value is unique.

Each relationship between entities is called an direction with a subject, verb and object.

What we are actually dealing with is a database that has data states. Data being no longer affected by Alters and Deletes are instead affected by change of state without physical alteration or deletion.

After looking at STL recently I realized I had created a command language for an existing database architecture: The Associative Model of Data by Simon Williams.

The Book on the Model and a free copy of the Enterprise Edition software is available here.

An old release of STL can be found here.

Say, “Hello World”, with the Associative Model

In the beginning was the priest. Writing materials were produced in limited quantities. Education was monopolized. Scription was a laborious task and difficult to correct with the materials used. Text made filing easy.  Thus you had the Flat database model.

Then came the scribe. Writing materials increased in availability. Education became institutionalized. Scription and transcription were performed by trained personnel who recorded the dictate of untrained personnel in an academic tongue.  Libraries made indexing easy.  Entire scriptoriums were dedicated to the process of document production. Thus you had the Hierarchical database model.

Then came the writer. Writing materials were mass produced. Education became publicized. Individuals wrote their own documents in their own tongue. The printing press guaranteed mass distribution.  Formatted printing made tabulation easy.  Thus you had the Relational database model.

Then came the layperson. Publishing became universally available via the internet.  Education became personalized.  Networks made linking easy.   Thus you had the Associative Model of Data conceived by Simon Williams.

sentences3.jpg

Simon has developed an Associational Database Management System (ADBMS) called Sentences. It foregoes the use of tables and inferred relationships for the use of single attribute entities and explicit relationships. The schema is intrinsic to the database making the business rules immediately available to anyone who accesses it. Finally, it is internet ready with the capability to be distributed across servers. It is a simple, elegant concept well executed, however there are still some hurdles.

The main hurdle is acceptance. Simon has met strong resistance from relational model advocates. He currently has a website offering the Sentences Enterprise Edition for free to anyone who wants it without technical support, but I do not think that is the answer. I believe that the potential of the Associative Model of Data is not fully realized in the Sentences proprietary implementation. If Sentences is to become the industry standard database for the internet, Simon Williams will have to open up Sentences to global collaboration as an open source project. Only then will the Associative Model reach the tipping point that puts it ahead of the relational model as the database architecture of choice for the lay internet user.

Simon needs Java Programmers for development and support and Mathematicians to develop Associational Calculus.

Links:

Lazysoft – Home site of Sentences

I highly recommend going to lazysoft.com and downloading the Sentences Personal and Enterprise Edition to get a feel for this new architecture.  Downloading and intalling the latest Java runtime environment and Sentences can be done in roughly ten minutes.  A populated sample database is ready for exploration.

Sentences Open Source Project

sentences3.jpg

Simon Williams, the creator of the Associative Model of Data, its architecture and the Sentences Associative Database Mangement System, had a discussion with me the other day. He told me that to make Sentences an Open Source project, he needs two things. First, he needs a Lead Java Developer to guide the Sentences open source project as Sentences is completely Java based. Second, he needs to get the user group for Sentences to reach the tipping point, which means developing Sentences databases and applications. I’ve used the relational data model for twenty years and the associative model is the next logical step in database architecture. If you want to advance internet database technology by helping this architecture and this product move into the mainstream, here’s your chance. Simon can be reached by email: simon.williams at lazysoft.com

Again, Simon’s book, The Associative Model of Data, and the Sentences Enterprise Edition is available for download free at the LazySoft.com website.