Machine Learning & Big Data Blog

5 Tribes of Machine Learning

5 minute read
Jonathan Johnson

Machine learning is multi-faceted. Learning mechanisms are proving incredibly useful to many disciplines. When machine learning gets applied to a discipline, like Biology or Psychology let’s say, the methodology usually takes on a form in line with that discipline.

Master of algorithm

In Pedro Domingos’ book, The Master Algorithm: How The Quest for the Ultimate Learning Machine Will Remake Our World, he categorizes the types of machine learning algorithms into five classes, which he calls the tribes of machine learning. Each group supports a set of principles, and, from them, stem different machine learning models.

From many to one, his approach ends by saying that all tribes will center upon a single, master algorithm to know everything. This final machine learning algorithm, The Master Algorithm, would be the artificial general intelligence many predict is to come. In order to get there, Domingos wants to take a little from each of these fives tribes and mix them together.

The 5 Tribes of Machine Learning:

  1. Symbolists
  2. Connectionists
  3. Bayesians
  4. Evolutionaries
  5. Analogizers

Symbolists

Symbolists focus on logic. They develop formal systems to create an AI. From the 1950s to the 1980s, symbolic AI was the dominant paradigm. Most computers were based on this kind of thinking, and, for decades, it was the most accessible to code. Symbolists will use technologies that follow a rigid path to make a decision based on logic. Their hard-coded rules would look something like, “Always go left when you encounter a stop sign”.

Systems that produce this kind of decision-making include:

Connectionists (Neuroscience)

Connectionists are the group that create models based on the brain. They love to point out that neural networks, a key part of their internal machine learning architecture, were modeled after the way neurons work in the brain. They use models such as:

When they say connections, they do not refer to connections like analogies, like a kitten is to a cat as a puppy is to a dog. For Connectionists, they use connections to term the signals that pass from one neuron to another. There are strengths of signals and numbers of signals.

A large problem people have with the Connectionist framework is that the way that decisions are made is hidden from view—the proverbial black box. Every step of a decision made in a decision tree is known. In Connectionism, an input may have an output, but the path that input took to get to that output goes unseen. A person cannot ask, “How did the model come to that conclusion?”

model

Bayesians (Statisticians)

The Bayesian tribe values statistics. It is a different form of logic, one that exists in probabilistic outcomes, and not in hard coded or either/or outcomes. For the Bayesians, machine learning is a form of probabilistic inference. Given an input, there is some chance that the output is Y. Nothing is determined; the possible outcomes have some chance of occurrence.

Bayesians might use Connectionist or Symbolist methods to generate decision models, but the focus in their thinking lies in having only probabilistic outcomes. The infrastructure to get from X to Y isn’t as important as the way the result is interpreted. Nonetheless, the Bayesians will use models such as:

  • Hidden Markov chains
  • Graphical models
  • Causal inference

Symbolist methods
(Source)

Evolutionaries (Biologists)

Evolutionaries like to think of things, not infrastructure, or determined outcomes, or even in the interpretations of the outcomes. These guys are the process theory types. They are concerned with steps and iterations.

Looking at machine learning from a biologist stance, they are concerned with the evolution of an AI. They’re curious with how it grows, mutates…with how it becomes. Similar to how people evolve at each mutation, the human genome itself having some ability to learn and adapt, producing a human better suited to navigate its environment at every turn, Evolutionaries want to create machine learning models that, too, learn and adapt.

Evolutionaries use genetic algorithms, and evolutionary programming. A common application of Evolutionary AI is on learning tasks. Below, the DeepMind team created a little stick figure object with legs, gave it an environment and simple physics, and made the goal to go forward. After many iterations of trial and error, the figure’s intelligence evolved to be able to leap over the objects and progress forward. Evolutionary algorithms are used a lot in game play. They will be one of the primary models that get cars driving autonomously.

Analogizers (Psychologists)

Analogizers are usually the storytellers. They can create classes of entities. If an input, old or new, is identified as part of one of those classes, then Analogizers believe they can predict the outcome of the input as being like the outcome of that class.

Problems that are tackled through this approach are recommender systems. For example, if a user were to watch a romantic comedy on Netflix, the Analogizer will say, “Given your watching behavior and that you liked this rom-com, you will like these other sets of movies because other people who have watched the same movie also like these other movies.” Analogizers use classes and types to identify groups of people, and predict future outcomes based upon the outcomes of other members of the class.

The machine learning models Analogizers will use are K-Nearest Neighbor algorithms and SVMs, unsupervised learning methods, to place members of a population into their classes. The below chart places its members into two classes 0 and 1. The second picture classifies a member into one of three classes.

 

The 5 ML Tribes

Each approach has its usefulness. Combinations of these Machine Learning methods might be the answer to an AGI. Self-driving cars might learn to drive safely on the roads through an Evolutionary method, but they will use Connectionist methods to give the cars’ sensors sight. The car might get a huge boost in its user-driver interactions by utilizing the Analogizer’s way of classifying its drivers into types, Aggressive, Defensive, or Passive. Finally, there are rules to the road such as stopping at stop signs and staying between the lines, where methods from the traditional Symbolist tribe prove beneficial.

Additional resources

For more on this topic, browse the BMC Machine Learning & Big Data Blog or check out these articles and how-to’s:

Learn ML with our free downloadable guide

This e-book teaches machine learning in the simplest way possible. This book is for managers, programmers, directors – and anyone else who wants to learn machine learning. We start with very basic stats and algebra and build upon that.


These postings are my own and do not necessarily represent BMC's position, strategies, or opinion.

See an error or have a suggestion? Please let us know by emailing [email protected].

Business, Faster than Humanly Possible

BMC empowers 86% of the Forbes Global 50 to accelerate business value faster than humanly possible. Our industry-leading portfolio unlocks human and machine potential to drive business growth, innovation, and sustainable success. BMC does this in a simple and optimized way by connecting people, systems, and data that power the world’s largest organizations so they can seize a competitive advantage.
Learn more about BMC ›

About the author

Jonathan Johnson

Jonathan Johnson is a tech writer who integrates life and technology. Supports increasing people's degrees of freedom.