What’s Wrong With the Neural Network? Lack of Data Structure that Enables Governance


A recent article at CACM brought up a topic that I’ve been thinking about off and on for quite a few years surrounding the relationship between network architecture, security, functionality, and sustainable economics in the neural network economy.

The article in question is titled “BufferBloat: What’s Wrong With the Internet?“, in discussion format with Vint Cerf—an old acquaintence, Van Jacobson, Nick Weaver, and Jim Gettys. Thanks to Franz Dill who alerted me to the article on his blog.

For this discussion let’s define the current neural network as: encompassing any individual entity, device, application, or sensor that is either part of, or connected to, the Internet, World Wide Web, or extension thereof—such as telecom, cable, or wireless networks. By looking at this problem systemically in the context of what is required of a neural network to perform as a sustainable ecosystem—which requires certain elements and features, we can then perhaps be far enough removed for a perspective that may help address this and other specific problems in the neural network economy.

Like most ecosystems, the first rule of neural networks can be taken directly from biology, which is that in order to maintain the network it must contain essential elements that sustain its existence. Otherwise it will simply perish. For the sake of this discussion let’s place a label of economics on the discipline and define the activity as one that serves the purpose of achieving sustainable resource management. In this context sustainability of the ecosystem would include any entity providing a subsidy.

In order to achieve sustainability, especially when dealing with economic entities, a critical mass of essential resources must have motivation to behave in such a way that favors sustainability. To manage the impacts from that behavior, usually through incentives and penalties called regulation, a structure of governance is generally required. And in order for the ecosystem to adapt to constant change, the governance structure found to be most effective is one that embraces markets, aka market economics, or in long form: resource management through governance of markets by a regulatory scheme that incentivizes and enforces adaptable behavior towards a sustainable ecosystem. Of course any sustainable governance system for neural networks must consider such issues as preventative maintenance, security, growth and contraction, among others.

When we observe the type of behavior demonstrated in the BufferBloat problem, the underlying cause is quite often a combination of factors that include network engineering and governance engineering, both of which appear to be critical causal factors in this case. In the neural network economy, regardless of the many types of motivation that would impact individual behavior, the network must first contain sufficient intelligence to inform the individual (whether human or machine) about the impact of their behavior, and in order for machines to collect sufficient intelligence to do so the global computer network must provide an adaptable model that balances security and privacy according to regional regulation.

Finally, as our patented system provides, that adaptability should in my view include the quality and quantity of data as defined and managed by each entity’s profile within the governance parameters, whether national regulations or business rules in the enterprise. For example, the operating system I am using to write and upload this to the Web has a basic profile with built-in security and tracking features, but of course unlike Kyield the profile functionality in this OS does not extend to my specific data needs or that of my organizational entity.

Security requires data governance, which requires data provenance, which requires data structure

One of the intriguing aspects of the BufferBloat challenge is that it highlights the problems that can arise when very little provenance exists, without which accountability and security remain elusive, especially given regional regulatory schemes. This is why we long ago embraced the concept of universal standards for data structure, aka the ‘semantic web’. The conceptual framework contained the potential to allow a more intelligent, functional, and one hopes—rationally sustainable, medium.

But that gets to this awkward problem of not knowing what the source of the congestion might be unless you’re monitoring source/destination pair traffic or something along those lines. – Vint Cerf

…it can turn into an impossible problem because someone who doesn’t want to be policed will just spread traffic over a zillion addresses, and there’s no way to detect whether those are separate flows or all coming from a single user. – Nick Weaver

As is the case in the physical world, when followed down the maze to inevitable conclusion, governance can be reduced to an individual human rights issue that overlaps civil liberties, privacy, security, individual property rights, and in the U.S. those covered by the Bill of Rights. Many people believe that provenance and privacy are incompatible, but actually it depends again on a combination of technology architecture and choices made in governance policy. The problem outlined above by Nick Weaver is only possible due to the specific architecture of the Internet that allows it. That is to say that anonymity and lack of provenance is intentional policy of the Internet, which in some parts of the world, and perhaps at certain times, are more necessary than others, particularly relating to threat by heavy handed governments, which calls for adaptability.

In a general sense the BufferBloat problem is partially about alignment of interests between the supply chain of vendors, or as is often the case –misalignment of interests. For example, search engines have a business model that is dependent upon advertising, which is in turn dependent upon a ‘free Web’, and somewhat reliant on unlimited bandwidth. Of course as the BufferBloat case demonstrates, the Web is certainly not free, rather it’s a question of who is paying for what, when, and how. The interests of hardware and desktop software vendors have been fairly well aligned with search engines to date as they all tend to profit from demand for their products subsidized by free content. Telecoms and cable companies also profit from free content on the Web—although not necessarily unfettered bandwidth; and generally speaking consumer products companies have also shared alignment of interests with the ‘free’ advertising model of the Web, provided of course the ROI in advertising remained beneficial. These dynamics are beginning to change, however, as consumer companies, IT vendors and governments begin to realize that their customers need jobs for income to buy their products and pay taxes.

The current neural network economic model conflicts to some degree with the interests of countries like the U.S. due to our service-oriented economy, as well as most organizations within the so-called knowledge economy, many of which are in the process of being disrupted and/or replaced by the business models of the Internet and Web. Essentially the economic model of the Web is made possible by massive subsidies by content providers worldwide of every type with an advertising revenue sharing structure that requires tracking and favors volume. This was well understood before current search leaders were conceived, but the model was viewed as a necessary step, albeit a reluctant one by Silicon Valley which did not want to see its future limited to advertising.

Apple is one exception to the IT industry that has profited greatly from poor structure and governance in the neural network economy by creating platforms and models that are more economically sustainable and better aligned with the interests of content providers, but is the Apple case not yet another symptom of the lack of network governance?

The BufferBloat problem is a good example of what can occur in a neural network environment where the limitations of a single discipline of limited scope and interest can impact the global economy, where misplaced ideology and good intentions have the potential to wreak havoc, and where the business models of those influencing governance can be directly conflicting with the broader needs of both the physical and neural network economies.

The scale challenge with structured data

While scale has occasionally served as an excuse not to embrace standards or deploy structured data, scale has been a legitimate technical obstacle to wider adoption of structured data, as has been ease of use, representing significant if temporary technical risks. However, the trajectory of innovation in underlying hardware, software and trade craft has been pointing towards resolution on a parallel path with Moore’s law all along. What only a supercomputer could accomplish a few years ago with large data sets that required hours to run can now be performed in minutes or even seconds in most data centers. Moreover, the short-term trajectory reflects a very high probability of near-real time execution at consumer search scale soon.

Conclusion

While structured data can only provide a partial solution in the near-term to the specific BufferBloat challenge, the combination of a Semantic Web and Internet with deeper embedded intelligence is foundational in providing the potential to overcome a host of complex technical and governance challenges within the neural network economy, including the potential to assist greatly with security, privacy, data quality, data volume, business model alignment, and more stable global economic equilibrium.

Advertisements

New paper: Optimizing Knowledge Yield in the Digital Workplace


I am pleased to share a new paper that may be of interest:

Optimizing Knowledge Yield in the Digital Workplace
A new system design for thriving in the data-intensive universe

From the abstract:

The purpose of this paper is threefold. First, it briefly describes how the digital
workplace evolved in an incremental manner. Second, it discusses related structural
technical and economic challenges for individuals and organizations in the digital
workplace. Lastly, it summarizes how Kyield’s novel approach can serve to provide
exponential performance improvement.

Background of patent #8005778


I am working on multiple articles relating to the patent I was issued last week, at least one of which will be posted here in the next few days, but in the interim I thought some might be interested in the common English portion of the patent. I hadn’t visited this section in some time–from early 2006.

Patent #8005778

Title: Modular System for Optimizing Knowledge Yield In the Digital Workplace (USPTO link to patent)

FIELD OF THE INVENTION

The invention relates to the management of human intellectual capital within computer networked organizations, and more particularly to managing the quantity and quality of digital work flow of individual knowledge workers and work groups for the purpose of increasing knowledge yield, or output.

BACKGROUND OF THE INVENTION

The volume of data transfer and related human consumption of information is growing exponentially in the network era, resulting in a condition commonly referred to as information overload. The result for the modern organization is an ever increasing challenge to manage the quantity and quality of information being transferred, consumed, and stored within computer networks.

Enormous amounts of structured and unstructured information is being consumed by knowledge workers that is redundant or irrelevant to the knowledge worker’s job, or the mission of the organization, creating serious challenges for organizations while reducing the return on investment for information technologies and knowledge workers.

Systems deployed previously attempting to reduce information overload and increase knowledge worker productivity have been designed primarily to address either the symptoms of the problem, or a specific portion thereof; including desktop productivity suites, higher performance search engines, and reducing unsolicited e-mail.

In recent years, computer standards bodies have been approaching the challenge by improving machine to machine automation and structure to documents with XML, RDF, SOAP, and OWL, commonly referred to as the Semantic Web.

Emerging positions within networked organizations attempting to optimize the digital workplace include the Chief Knowledge Officer (hereinafter “CKO”) who is responsible for improving the value of human and intellectual capital to better achieve the organization’s mission.

Despite these individual and collective efforts, the problems associated with information overload continue to grow exponentially. According to research firms IDC and Delphi Group, the average knowledge worker spends about a quarter of his or her day looking for information.

A related serious problem for knowledge workers affecting productivity and innovation is that intellectual property converted to digital form is simple to copy and distribute, providing disincentives for creative problem solving, the sharing of knowledge and intellectual property, and therefore improving work quality.

Given the complexities of the digital workplace environment, it would be beneficial for organizations to employ a holistic metadata system including modules to manage the knowledge yield for the entire organization, for each work group within the organization, and each individual member of the organization so they can continually optimize his/her knowledge yield for the continuously changing work environment.

Regulatory Failure on the Web; Consequences and Solutions


I have argued consistently since the mid 1990s that the global medium (combined Internet and Web) increasingly reflects the global economy and that rational, functional regulation is essential. I started this journey then with a very similar ideology to Alan Greenspan’s before the financial crisis; that self-regulation should be sufficient to prevent systemic crises, but unfortunately in practice it has failed to do so.

Most of the actual regulation in computer networking today is accomplished via manipulation of architecture in one form or another, but technical standards on the web are voluntary, as the Tech Review article The Web is Reborn highlights, which was apparently in response to the article The Web is Dead at Wired earlier in the year. In the U.S. we are really reliant primarily on one form of regulation on the web, other than proprietary architecture and voluntary standards, which is social. Social regulation has evolved with the consumer web, occasionally demonstrating some power—as was recently demonstrated with Facebook security issues, but social regulation has also proven to be self-destructive at times, particularly regarding sustainable economics and jobs. Few if any consumers can see how their actions on the web are impacting their own regional economy or industry, meaning that the blind is often leading the blind towards dangerous hazards in a similar fashion to the housing crisis. Ignorance is being exploited.

Two eras, opposing needs, yet same reaction

In order to provide some context with continuity let’s begin with the PC revolution about 30 years ago when the lack of interoperability allowed Microsoft to extend its growth from the operating system into productivity, communications, and eventually networking, forming one of the strongest strategic partnerships and business ecosystems in history.

During the PC era regulation was essentially outsourced to industry which employed a combination of proprietary computer code, strategic alliances and the failure of others to work together in a competitive manner to establish the standard, which of course led to a monopoly. The political, social and cultural dynamics at play were very interesting at the time, dominated by the view still common today that the only other option was government which couldn’t be relied upon to regulate technology.  It turns out that many other forms of regulation exist that can be learned from in natural sciences, physical sciences, social sciences, and architecture, among others.

Among the most important business lessons in the PC era was that Microsoft bet against the abilities of others to work together which when combined with strong execution was rewarded at historic levels. At the time I clearly recall arguments from investors, customers, government officials and even competitors agreeing with Microsoft that the PC revolution was too young for external regulation (which I heard again this week regarding the web), so “let the best man win”.  I myself said much the same then—not having the benefit of observing this case (and many others) in what is a very complex, quickly evolving environment.

In hindsight I believe we were correct regarding regulation in the PC era, but only for a very brief time—less than a decade; that’s how fast the big innovation door opened, scaled, and began to close. Our society cannot respond that quickly. This was new territory, just as the web would be a decade later.

In a very similar manner to the PC era the lack of regulation on the early web fostered a highly innovative environment during the very early days, but the era peaked much quicker on the web due to the toxic flood of capital during the inflation of the dot-com bubble; and the web was very quickly taken over by entrenched interests.  Opportunity still exists of course, but the dirty secret few discuss is that the failure rate for new IT ventures is very likely at an all time high—no credible statistics exist on the entire ecosystem to my knowledge; only portions thereof.  Of course knowledge, experience, relationships, resources and luck play a big role on outcomes, as usual, but lack of effective regulation generally favors and rewards predatory behavior.

The PC was sustaining innovation; the web is disruptive innovation

It’s important to understand that in the pre-network era economic alignment in desktop computing was primarily positive for everyone except direct competitors to Microsoft (or Intel in semiconductors), which was managed masterfully by a brilliant entrepreneur who became the world’s wealthiest human, and supported by many other brilliant people.  The world needed a standard for interoperability, and since few were negatively impacted the increases in productivity from authoritarian rule were viewed largely as positive within the social regulation realm, even if only for a brief time.  In hindsight government regulation failed not only to prevent the monopoly but also in resolving it. Government was then and still is today complicit in the creation and protection of  monopolies, regardless of how they form, particularly in the U.S. and EU within the IT cluster, which is I think driving future industry leaders to other countries.

Once monopoly power sets in it can be very destructive, including to the long-term company culture within the monopoly itself, which provides a strong case to manage market share very carefully.  The largest impact, however, is invisible, which usually manifests as aborted innovation within the specific market and industry, lack of adoption of competing technologies, and failed investment, which is evident today in most consolidated industries as reflected by very poor economic performance.

Failed regulation often leads to market failure, which is a real possibility for the web unless a sustainable economic structure is formed. This is essentially the argument behind the claim the “Web is dead”—with Chris Anderson suggesting that the Internet was moving over to wireless devices where a more viable economic structure is forming; customers are far more likely to pay for services rendered in the iPhone structure than the web structure.

As I have often argued since the commercialization of the web, the advertising industry is not nearly large enough to compensate for the economic displacement of industries from the disruption, particularly in the service dominated economies in the West.  Silicon Valley, Madison Ave, Wall St. and D.C. cultures still don’t seem to fully understand this reality and equation, or presumably policy would reflect it.  China and Germany on the other hand seem to understand the issue with abundant clarity, and are exploiting the situation brilliantly, as is India and others.  A nation does not want to be dependent upon a service economy within a global economy that is increasingly delivering services over an ad supported medium; particularly a nation deep in debt that is challenged educationally.

The often misunderstood lesson here is that the PC ecosystem was not a disruptive innovation but rather a sustaining innovation—meaning that it threatened very few. In direct contrast the web is very disruptive—not only to specific companies, but to entire industry clusters, regional, and national economies, which affects everyone on the planet.

Despite this extremely important difference, regulatory schemes reacting to the two very different situations are essentially the same, and will very likely result in a similar outcome unless regulation improves quickly, particularly relating to technical standardization.  As market share becomes more dominant in corporate cultures, so does hubris—the cultures become increasingly less influenced by voluntary standard processes or social regulation.  Eventually, as we’ve seen in our recent past, the monopoly cultures can even directly challenge the authority of sovereign governments with the potential exist for global companies to actually dominate national policy.  Currently Ireland provides a fine case study of why such a situation should be avoided.

I maintain that the winner-takes-all approach of the PC era would be catastrophic for the web and the global economy, perhaps even leading indirectly to civil disruption and conflict. Many wars have been fought over far less economic disruption so in a very real sense this issue is one of national and global security.

So is the web dead or reborn?

The web is primarily lost in a sea of confusion from lack of structure, which is largely due to the lack of effective regulation, which is in turn due to spin from those who benefit from the lack of regulation, and perhaps the impact of that spin on the ideology within our culture.  As in all previous standards wars free from effective regulation, a continuous battle rages, albeit somewhat more rational given the global nature of the beast than in previous sector or geographic standards wars.

In an invited letter to the editor in an upcoming issue of a leading publication, I will argue for functional regulation of the web relating to the creation and enforcement of technical standards, which are necessary to achieve security, privacy, and a host of other essential issues, including some degree of certainty for investors and entrepreneurs like myself.  It is far more important that credible independent standards exist than what the specific standards are, which is lost on the academic CS community almost entirely.  The current scheme is without power, glacial, and entirely without dependability, the latter of which is synonymous with credibility outside of academia.

I will save my detailed suggestions on how such a regulatory body might be structured for my book, but there is an emerging regulatory scheme on the web worth noting within the largest industry.  The U.S. health care legislation, as messy as it was, did empower the HHS to determine technical standards for electronic health records, which was tied to funding and reimbursement.  While substantially less than perfect, this standards process does appear to have the ability to gain traction due to a combination of initial funding, need for interoperable data, and leverage from other governments around the world to achieve a functional global standard.

Just one example of how this may occur is the relationship between life sciences, government regulation over drugs and devices, and the delivery of health care, all of which will require interoperability in order to function with any degree of efficiency.  In the current environment the health care technical standards process appears to be the most functional regulatory path towards adoption of a more intelligent web, aka the semantic web.

While we are all aware of the messiness of democracy, this alternate path towards regulation of standards on the web should not be viewed as a substitute for a rational, long-term solution.  Welcoming luck once it occurs is one thing; depending on it for survival quite another.  Our economy is too fragile and complex to depend on luck alone.  Conflicted interests simply cannot be trusted, whether corporate, academic, or otherwise.

Realizing the Theory: Yield Management of Knowledge


Since I have now responded to a related USPTO elections/restriction letter, I feel a bit more freedom in sharing additional thoughts on the underlying theory that has served as the foundation of Kyield:

Yield Management of Knowledge: The process by which individuals and organizations manage the quality and quantity of information consumption, storage, and retrieval in order to optimize knowledge yield.–(me) 

(Please see this related post prior to attempting to categorize)

Background: The cyber lab experiment

The theory emerged gradually over several years of hyper intensive information management in my small lab on our property in northern Arizona (we are currently based in Santa Fe after a year in the Bay area). The experimental network called GWIN (Global Web Interactive Network) was designed after several years of previous high intensity work in our incubator, which followed a decade in consulting that was also drawn from. The GWIN product was entirely unique and intentionally designed to test the bleeding edge of what was then possible in computer and social sciences. We were particularly interested in filtering various forms of digitized intelligence worldwide as quality sources came online, conversion to useful knowledge, weaving through academic disciplines, and then mix with professional networking.

The network was open to anyone, but soon became sort of an online version of the World Economic Forum (photo), with quite a few of the same institutions and people, although our humble network even in nascent form was broader, deeper, larger, with less elitism and therefore more effective in some ways.

Mark Montgomery's first computer lab and incubator building (Kyield)

Our first computer lab and office

I was quite proud that membership was based primarily on interest, effort, and intellectual contributions; not social status, guilds, political views, market power, or wealth, even if the norm in our membership.

My late partner and friend Russell Borland and I learned a great deal from GWIN, as did many of our members and those who followed our work closely. The first thing one should understand is that while we worked with various teams of remote programmers to build products, and served thousands of people worldwide daily who provided about half of the content, I operated the lab solo onsite. Given the volume, work hours, lab efficiencies, and short commute, I was likely consuming as much data personally as any other human, which is fundamental to the construct of the theory; how the brain functions in dealing with overload, human-computer interaction, and what tools, languages, and architectures were needed in order to optimize knowledge yield.

Need to emphasize data quality, not quantity

The vast majority of solutions for improved decision making in the networked era have been computing versions of HazMat crews attempting to clean up the toxic waste resulting from information overload. Reliance on the advertising model for the consumer Web created a system design essentially requiring lower quality in a populist manner, aided and abetted by search and then social networking.  While the advertising model is certainly appropriate for many forms of low cost entertainment, for serious learning and important decision making envisioned in yield management of knowledge, an ounce of prevention in the form of logically structured data is worth potentially far more than a ton of cure.

It became obvious very early in our lab (1996) that the world needed a much more intelligently structured Web and Internet, for consumers as well as the enterprise. In studying search engines closely at the earliest stages, they were by necessity applying brute computing force, clever algorithms, and exploiting content providers in an attempt to deal with the unprecedented explosion of data, and noise, while providing what investors needed for such risk. What we really needed of course was logically structured data that was controlled by data owners and providers, which would then (and only then) provide the opportunity for knowledge yield. Further, the languages used for the structure must be non-proprietary due to the overwhelming global market power that would result for the winner due to the network effect.

Need for independent standards

In the enterprise market, proprietary languages can and do thrive internally, but the integration required in sharing data with essential external partners is similar to the brute force applied in search—crisis clean-up rather than prevention, complete with disempowerment of customers who create and own the data. Most organizations increasingly rely on shared data, whether regulatory or partnerships, even if private and encrypted, so proprietary data languages are not well aligned to the enterprise in the increasingly networked, global economy.

Finally, there are fundamental and unavoidable conflicts between large public companies that dominate markets with proprietary languages, their fiduciary duty, and the minimal sustainable requirements of our globally networked economy. A few examples of these conflicts can be clearly observed today in failure to deal effectively with network security, personal privacy, protection of intellectual property, and information overload. Evidence of the challenge can also be observed (and felt by millions of people) in economics where policies of multinationals favor the largest emerging markets due to market access. Of course the lack of functioning governance of what has become an essential global medium empowers these phenomenons.

It is my personal position that the intellectual competition should be intentionally focused on optimal use of the data to achieve the mission of the customer (whether individual consumer or enterprise), not protectionism, and that vendors should be the caretaker of data on the behalf of data owners, which requires a different economic model than the free ad supported model on the consumer Web.

So in order to realize the goal of the theory, we really needed a much more intelligent and highly structured Internet and Web that is based on independent languages in a similar method as the underlying protocols (TCP/IP and HTTP), and not supported by advertising alone.

I am speaking here of data communication in global networks, not individual applications. If we had the former we need not worry about the latter, at least in the context of network dynamics.

A word of caution on standards

A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines.– Tim Berners-Lee, 1999

One of the most significant risks with independent universal standards is unintended consequences.  While the stated vision of most involved is for more efficiency, transparency, empowering individuals and organizations, less bureaucracy, and lower costs, the nature of universal languages favors organizations that control data. One of the primary challenges in Web 1.0 and 2.0 has been data silos and private sector exploitation of data owned by others, which is largely driven by the revenue model. The primary challenge of Web 3.0 and 4.0 could be increased government control at the expense of individual liberty and private sector jobs, or perhaps worse; a public/private duopoly or oligopoly. From the perspective of an entrepreneur attempting to create jobs, I see such risk increasing daily.

Introducing Mawthos

Louis V. Gerstner, Jr. was perhaps most responsible for moving software towards a service model in his turn around of IBM in the mid 1990s, which was a brilliant business strategy for IBM at that time (we exchanged letters on the topic in that era), but it has not been terribly successful as a model for creative destruction, rather it has primarily seemed to exchange one extortion model (propriety code) for another (combination of proprietary code, consulting, and open source). Unlike a giant turn around, we were focused on more revolutionary models that provided value where none existed previously, so our first effort was an ASP (Application Service Provider), which emerged in the mid 1990s. In 2001, this paper by the SIIA is credited with defining and popularizing SaaS (Software as a Service), which has evolved more recently to an ‘on demand’ subscription model that is often bare bones software apps like those developed for smart phones.

While I have been a big proponent of a cultural shift in software towards service, I have rarely been a proponent of the results sold under the banner of service in the software industry, recognizing a shift in promotions and revenue modeling, not culture. In reviewing this article I recalled many public and private discussions through the years debating the misalignment of interests between vendors and the missions of customers, so thought I would introduce yet another acronym: Mawthos (Mission accomplished with the help of software), which is a slight jab at our acronym-fatigued environment, while attempting to describe a more appropriate posture and role for enterprise software, and the philosophy necessary to realize the theory Yield Management of Knowledge.