5 Main Types of Knowledge Representation in Machine Learning

Original article can be found here (source): Artificial Intelligence on Medium

5 Main Types of Knowledge Representation in Machine Learning

Photo by Kuma Kum on Unsplash

There are many different ways for representing the patterns that can be discovered by machine learning, and each one dictates the kind of technique that can be used to infer that output structure from data.

Once you understand how the output is represented, you have come a long way toward understanding how it can be generated.

In this article,I talk about main types of representation:

Decision tables

Decision Table Example

The simplest, most rudimentary way of representing the output from machine learning is to make it just the same as the input.

Decision trees

Decision Tree Example

A divide-and-conquer approach to the problem of learning from a set of independent instances, leads naturally to a style of representation called a decision tree.

Classification rules

classification rules example

Classification rules are a popular alternative to decision trees.

The antecedent, or precondition, of a rule is a series of tests just like the tests at nodes in decision trees, and the consequent,or conclusion, gives the class or classes that apply to instances covered by that rule, or perhaps gives a probability distribution over the classes.

Association rules

Association rules are really no different from classification rules except that they can predict any attribute, not just the class, and this gives them the freedom to predict combinations of attributes too.

To reduce the number of rules that are produced, in cases where several rules are related it makes sense to present only the strongest one to the user.

For example, with the weather data,we can extract this rule:

If temperature = cool then humidity = normal

Rules with exceptions

Returning to classification rules, a natural extension is to allow them to have exceptions.

Then incremental modifications can be made to a rule set by expressing exceptions to existing rules rather than reengineering the entire set.

Instead of changing the tests in the existing rules, an expert might be consulted to explain why the new instance violates them, receiving explanations that could be used to extend the relevant rules only.


Clusters Example

When clusters rather than a classifier is learned, the output takes the form of a diagram that shows how the instances fall into clusters.

In the simplest case this involves associating a cluster number with each instance, which might be depicted by laying the instances out in two dimensions and partitioning the space to show each cluster.


Knowledge representation is a key topic in classical artificial intelligence and is well represented by a comprehensive series of papers edited by Brachman and Levesque.

We mentioned the problem of dealing with conflict among different rules.

Various ways of doing this, called conflict resolution strategies, have been developed for use with rule-based programming systems.

These are described in books on rule-based programming, such as that by Brownstown.