As was previously hinted, ML is a field that uses statistical models to allow machines to mimic human task performance without needing any kind of brute force code to help it run.
Over the decades, as the theory behind Machine Learning algorithms developed, so did their definitions.
ML [the fidget spinner-looking center] breaks down into three main categories
- Supervised Learning
- Unsupervised Learning
- Reinforcement Learning
Conveniently, the list so happened to be arranged in increasing order of complexity, beginning with Supervised Learning being the least complex to Reinforcement Learning being the most complex (relatively speaking, so to say).
So let’s dissect each of the learning approaches to get a better understanding of them.
The world of Supervised Learning (SL) is an intuitive one albeit nuanced in its own way. It is a type of learning that feeds [learns] off of data that is labeled.
How so?
Before I get into what it means for data to be labeled, I think it is important to first classify [get it?] the categories [I did it again…] of problems that come under Supervised Learning.
There are mostly two types of problems you will encounter when dealing with Supervised Learning
- Classification-type problems
- Regression-type problems
Classification is the act of classifying certain observations into a category. Usually, it is a 0/1-Dog/Cat -Yes/No -Spam/Not Spam type category which we call Binary Classification. You could, however, have multiple classes to identify such as Bird/Cat/Mouse/Dog — Yes/No/Maybe, in which case it would be called Multi-Class Classification.
There are many algorithms available to tackle classification problems, some of which are:
- Logistic Regression
- Support Vector Machines
- Neural Networks and etc.
On the other hand, Regression is a problem whereby we have to come up with a “lesser” form, hence regression, but yet representative line for the data so that we are able to interpolate or extrapolate for new, real-valued data!
As with everything, there are many algorithms that are equipped to take on regression problems, some of which are but not limited to:
- Simple & Multiple Linear Regression
- Support Vector Regression
- Neural Networks…
So now that we know what types of problems might arise, let’s go back and explore what it means for data to be labeled.
In a given dataset, you usually have a bunch of features [also called predictors] and the variable to be predicted [dependent variable]. In our case here, the dependent variable is known and used while training hence the learning is supervised and the data is labeled. A feature is a certain measured characteristic of our problem that helps describe certain instances.
So let’s say you’re trying to classify a salmon vs. bass fish, some of the features would be lightness, width, number of fins, and so forth.
Unsupervised Learning gets a tiny bit more abstract since the model we deploy learns off of data that isn’t labeled. As you’ve probably guessed, data that is not labeled refers to data that only has a bundle of features but no dependent variable to help categorize those features.
What ends up happening now, is that it’s up to the algorithm to find the hidden relation between the features. This is important because when faced with millions of data points and high-dimensional feature spaces, the structure of it all is not so evident. Therefore, we pass it onto an Unsupervised Learning model in order to capture some of the non-trivial insights that are hidden from within the data.
To deal with this, two widely used ML concepts called clustering and association are deployed.
Clustering is a way of throwing a bunch of features, with nothing to characterize them, onto your model and allowing it to come up with clusters of data that are similar in their own ways. An example of an algorithm deployed to tackle this problem is called K-Means Clustering.
Association rule learning on the other hand, while slightly seeming similar to Clustering, serves a different purpose. Let’s take a clear example from Wikipedia on Association,
“For example, the rule {onions, potatoes} => {burger} found in the sales data of a supermarket would indicate that if a customer buys onions and potatoes together, they are likely to also buy hamburger meat. Such information can be used as the basis for decisions about marketing activities.”
One of many algorithms used to tackle the problem of Association rule learning is the Apriori Algorithm, which is mainly used to determine trends in databases containing transactional (from the word transaction) histories, as seen below:
In a formal sense, association rule learning is a method used to discover interesting relationships between variables, whereby acting on a variable can infer the other through this discovered relationship.
Reinforcement Learning (R.L) is probably one of the coolest ML techniques out there. Remember, back in the day, when you couldn’t finish that level in Mario Kart in record time without failing miserably? Well…
RL is a bit of a complicated topic with quite a deep involvement in the math aspect of things, but at the core of it, it is an agent and an environment whereby the agent performs an action (A_t) and the environment updates the agent’s state and either penalizes or rewards the agent based on the agent’s previous action. [Simplistically speaking]
This is where we will leave it… for now.