Before we survey specific methods, it helps to formally define our scope. Supervised learning refers to machine learning tasks where models are trained on labeled datasets that map inputs to desired outputs.
The goal is to uncover relationships in the data that accurately predict labels for new, never-before-seen inputs. Based on what type of labels we want to predict, supervised learning divides into two subtypes…
Classification – Categorizing Observations
Classification models assign each data point one or more discrete categories. Common uses include:
- Image recognition: Is this a dog or cat?
- Medical diagnosis: Does this patient have diabetes?
- Sentiment analysis: Is this review positive or negative?
The model must choose from a fixed set of possibilities, rather than outputting a numeric score. We‘ll cover algorithms optimized for handling such categorical prediction problems.
Regression – Forecasting Continuous Values
On the other hand, regression models predict continuous, real-valued outputs like:
- House prices
- Weather forecasts
- Energy demand
- GDP growth rates
Any quantity within a range makes sense versus classification where only certain labels are valid. The algorithms below focus on accurately fitting such infinite target variables.
Now that we distinguish supervised learning tasks, let‘s explore common algorithms that excel at either classification, regression, or both…
Naive Bayes Classifiers
Algorithms don‘t get much simpler than Naive Bayes! It relies on Bayes‘ theorem of conditional probability to estimate the likelihood that an input belongs to a certain class.
The "naive" assumption gives it speed—the model posits each input as independent rather than interacting. This allows efficient calculations even on massive datasets with thousands of features.
While less accurate than fancier methods, Naive Bayes delivers surprisingly good classification performance. It shines when:
- Training data is limited
- Distinct, uncorrelated inputs exist
- Easy model interpretation is desired
Fields successfully applying Naive Bayes include:
- Spam filtering – 93% accuracy based on text features
- Sentiment analysis – 85% precision labeling reviews
- Recommender systems – 75% match rate predicting user tastes
However, raw Naive Bayes struggles with redundant, strongly correlated inputs. Advanced variants like Gaussian NB address this limitation by assuming inputs follow particular probability distributions.
Overall, the algorithm provides a simple, yet solid baseline model for classification tasks involving lots of data points and features.
