Non-Data-Driven Approach to Machine Learning: Practical AI

Source: Deep Learning on Medium

Non-Data-Driven Approach to Machine Learning: Practical AI

Perhaps more than 90% of all machine learning applications today, such as deep learning, are data-driven methods. But, data can be quite expensive and messy caused by collection, validation, annotation, and correction. Additional labor costs emerge due to long training cycles and forming an in-house team of experts. In some cases, the costs make the deep learning approach economically infeasible such that some businesses stay out of this particular AI race. Those who fall into the AI whirl relying on data, the outcomes are not as bright as explained in this recent article.

“A recent International Data Corporation survey of global organizations that are already using AI solutions found only 25% have developed an enterprise-wide AI strategy. Most organizations reported failures among their AI projects, with a quarter of them reporting up to a 50% failure rate.”

Good news is, there are alternative machine learning methods that are not data-driven (for NLP purposes). They are knowledge-driven (or model-driven) methods and are more sophisticated than their data-driven counterparts. Using these methods require understanding of multiple disciplines such as linguistics, KR, cognitive sciences, etc. In other words, they are not engineering friendly. As a result, most engineers steer away from them and gravitate toward data-driven methods (like deep learning) and miss the advantages of alternative approaches. Non-data-driven methods are not easily accessible today, but they are coming. One prominent example by exClone is illustrated in this article.

I tagged the knowledge-driven methods as “Practical AI” because of the non-data part where development and deployment are straight forward, transparent, and affordable, not affected by the uncertainties of data.

Let’s reiterate the difference between data-driven and knowledge-driven methods, which can be explained by a cognitive function: Learning by Reading!

Can Computers Learn by Reading like Us?

When we (humans) open a book, it is usually true that we understand 99% of all the words inside the book. We know what each word means, and how all the words are related to each other. Reading the book only shows us new relationships between the words (which is the formation of new knowledge).

For example, we know what alligator means, what egg means, what 34 degree Celsius means, what male means. But if we read a sentence “If alligator eggs are kept warmer than 34 C, they produce male alligators.” then we immediately form a new relationship between these words. In the human brain, that corresponds to a new cognitive map (new connections). That’s learning in the true sense. This process is Knowledge-based learning because we start reading the book with the knowledge of the words (ontological reference). Obviously, these methods rely on the availability of a well developed ontology. exClone’s technology is labeled as “instant learning” in the chart below referring to learning by reading (non-iterative).

Data-driven method, on the other hand, does not start with the knowledge of the words, and tries to learn the language in addition to learning the knowledge embedded in the book. Now, this task is colossal compared to the one above, and requires immense volumes of data (corpus) beyond what is available in the book. Finding an appropriate corpus is often a problem, and corpus validation, annotation, and correction will drive the costs up.

The trade-off between the two approaches converge to (1) availability of ontology versus training data/corpus, (2) cost of development, (3) length of deployment, and (4) complexity of maintenance.

Fundamental Differences

If we go one step further in technical details, the diagram below can be used to explain the two approaches. This particular knowledge driven method, instant learning network (shown on the right), is comprised of layers with different neuron types where their role is determined linguistically. All the neurons correspond to unique ontological concepts, and the learning is represented only by the connections. For example, the sentence “If alligator eggs are kept warmer than 34 C, they produce male alligators.” is represented with one set of connections easily traceable.

In contrast, deep learning (shown on the left) is a homogeneous architecture of neurons fully connected prior to any learning. Despite variations of deep learning, no neuron activity is designated for any linguistic role. The same sentence “If alligator eggs are kept warmer than 34 C, they produce male alligators.” is therefore not traceable, and is represented by a vast number of weights spread over the countless number of connections.

In Knowledge-driven architecture, connections represent knowledge. In Data-driven architecture, weights (numbers) represent knowledge. Obviously, former is closer to the equivalent biological process, and is faster to emulate learning by reading.

Answering Questions

Knowledge-based machine learning can answer questions from the content it learned with utmost precision using the ontological connections shown in the network picture above. This is a hypothetical case, where a question presented to the network finds its most relevant answer using those connections.

In case of partial connections, the network puts more emphasis on target, event, and instrument (in this order) and produces answers with an accuracy score. Based on the type of application, a threshold can be set to declare “no answer” if the best scoring sentence is below the threshold. With such a capability, the chatbot becomes self-aware of its performance, and can report how well it did in answering questions. This can be further expanded to social learning where chatbots can ask for feedback to learn how to answer particular questions.

Knowledge Breeding

More impressive than answering questions, knowledge-driven machine learning can breed new knowledge from the content it learned as shown here.

This is logic resolution using existing knowledge to produce possible new knowledge using the ontological connections. Obviously, breeding new knowledge is one of the most exciting aspects of learning algorithms that are not as straight forward as it looks when using data-driven models such as deep learning. One of the advantages of knowledge-driven machine learning is that the “new knowledge” is transparent (can be verified by human inspection) whereas the same cannot be said for data-driven deep learning.


Join CHATBOTS group in linkedin.

You can follow exClone in Facebook, and in LinkedIn.

Request a Demo from exClone

Download exClone App (iOS) — Ecosystem of virtual experts (beta)

Download exClone App (Android) — Ecosystem of virtual experts (beta)


#AI #Experts #Virtualexperts #Voiceassistants #practicalAI #chatbot #chatbots #artificialintelligence #ConversationalAI #Virtualassistants #bots #machinelearning #NLP #DL #deeplearning #