Learning Objectives

By the end of this lesson, the learner should be able to:

1. Define classification and regression in the context of machine learning.
2. Distinguish between classification and regression problems.
3. Identify common business applications of classification models.
4. Identify common business applications of regression models.
5. Explain the fundamental principles of decision trees, logistic regression and other classification approaches.
6. Explain the role of linear regression and other regression techniques in business prediction.
7. Distinguish binary, multiclass and multilabel classification.
8. Explain the importance of model training, prediction and evaluation.
9. Select an appropriate modeling approach for different business problems.
10. Interpret classification and regression outputs in a business decision-making context.

1. Introduction to Classification and Regression

In Lesson 8.2, supervised learning was introduced as an approach in which machine learning algorithms learn from historical data containing known outcomes.

Two of the most important supervised learning tasks are:

• Classification
• Regression

Both approaches use historical data to learn relationships between input variables and an outcome.

The fundamental difference is the type of outcome being predicted.

Classification

Classification predicts a category or class.

Examples:

• Fraud / Not Fraud
• Churn / No Churn
• Approved / Rejected
• High Risk / Low Risk
• Product A / Product B / Product C

Regression

Regression predicts a numerical value.

Examples:

• Monthly sales
• Property value
• Customer lifetime value
• Loan amount
• Revenue
• Delivery time

A simplified distinction is:

Classification → “Which category?”

Regression → “How much?”

2. Understanding Classification

2.1 Definition

Classification is a supervised machine learning task in which the objective is to assign an observation to one or more predefined categories.

Suppose a telecommunications company wants to predict whether customers will leave.

Historical data could look like:

Customer	Monthly Spend	Complaints	Tenure	Churn
C001	3,500	0	48 months	No
C002	2,800	4	8 months	Yes
C003	5,600	1	36 months	No

The model learns relationships between the features and the target.

For a new customer, the model might produce:

Predicted class: Churn

or potentially:

Probability of churn: 82%

Management can then use this information to prioritize retention efforts.

3. Types of Classification

Classification problems can take several forms.

3.1 Binary Classification

Binary classification involves two possible classes.

Examples:

• Fraud / Not Fraud
• Churn / No Churn
• Default / No Default
• Approved / Rejected
• Disease / No Disease

For example:

Will this customer churn?

Possible outcomes:

Yes

or

No

3.2 Multiclass Classification

Multiclass classification involves more than two possible categories.

For example, a bank could classify customers into:

• Low Risk
• Medium Risk
• High Risk

A retailer could classify products into:

• High Demand
• Medium Demand
• Low Demand

The model selects among multiple possible classes.

3.3 Multilabel Classification

In multilabel classification, one observation can be associated with multiple labels simultaneously.

For example, a customer may be classified as:

• High-value
• Frequent buyer
• Mobile user

A document could simultaneously be classified as:

• Finance
• Risk
• Compliance

This differs from multiclass classification, where an observation generally belongs to one class among several alternatives.

4. Classification Algorithms

Several machine learning algorithms can be used for classification.

Common examples include:

1. Logistic regression
2. Decision trees
3. Random forests
4. Support vector machines
5. k-nearest neighbors
6. Neural networks
7. Gradient-boosting methods

The choice depends on factors such as:

• Data characteristics
• Prediction requirements
• Interpretability
• Computational resources
• Accuracy requirements
• Regulatory considerations

No single algorithm is universally best.

5. Logistic Regression

Despite its name, logistic regression is commonly used for classification, particularly binary classification.

For example, a bank may use logistic regression to estimate the probability that a customer will default.

The model may produce:

Probability of default = 0.73

This means the model estimates a 73% probability of default, subject to the assumptions and calibration of the model.

The organization may then establish an appropriate decision threshold.

For example:

• Probability below threshold → Lower-risk classification
• Probability above threshold → Higher-risk classification

The threshold should be determined according to the business context rather than automatically assuming that 50% is always appropriate.

6. Decision Trees

A decision tree represents decisions through a sequence of questions or conditions.

For example, a simplified customer churn model might operate conceptually as:

Is customer tenure < 12 months?

→ Yes

Has the customer made more than 3 complaints?

→ Yes

High churn risk

This structure resembles a flowchart.

Decision trees are attractive in business environments because their logic can often be easier for non-technical stakeholders to understand than highly complex models.

However, individual decision trees can become overly complex and may overfit training data.

7. Random Forests

A random forest combines multiple decision trees to produce a stronger overall model.

Instead of relying on a single tree, the method creates an ensemble of trees and combines their outputs.

Conceptually:

Tree 1 → Prediction

Tree 2 → Prediction

Tree 3 → Prediction

…

Many Trees → Combined Prediction

Random forests can provide strong predictive performance and are useful for many structured business datasets.

However, they can be less straightforward to interpret than a single decision tree.

8. Understanding Regression

8.1 Definition

Regression is a supervised learning task in which the objective is to predict a continuous numerical outcome.

For example, a retailer may want to estimate next month’s sales.

Historical data could include:

• Price
• Advertising expenditure
• Promotional activity
• Season
• Previous sales

The target variable could be:

Monthly sales revenue

The model may predict:

Expected monthly sales = KSh 12.4 million

Unlike classification, the output is not simply a category.

9. Examples of Business Regression Problems

Regression can be applied to many business questions.

Sales

How much revenue will the business generate next month?

Property

What is the expected market value of this property?

Finance

What is the expected value of a financial exposure?

Marketing

What revenue might be generated from a campaign?

Operations

How long will a delivery take?

Customer Analytics

What is the expected lifetime value of this customer?

The central characteristic is that the target is numerical and generally continuous.

10. Linear Regression

Linear regression is one of the foundational regression techniques.

It attempts to model the relationship between a target variable and one or more explanatory variables.

A simplified model with one predictor can be represented as:

Y = a + bX

Where:

• Y = predicted outcome
• a = intercept
• b = coefficient
• X = predictor

For example, a business could examine the relationship between advertising expenditure and sales.

A simplified model might be:

Sales = 2,000,000 + 3.5 × Advertising Expenditure

The coefficient indicates the modeled relationship between advertising expenditure and sales, subject to the model’s assumptions.

With multiple predictors, the model can incorporate variables such as:

• Advertising
• Price
• Seasonality
• Distribution coverage
• Competitor activity

11. Multiple Linear Regression

Multiple linear regression uses several predictors to estimate a numerical outcome.

For example:

Sales = β₀ + β₁(Advertising) + β₂(Price) + β₃(Promotion) + β₄(Seasonality)

This allows the analyst to examine several factors simultaneously.

However, analysts should be careful not to interpret statistical association as proof of causation.

If advertising and sales are positively associated, this does not automatically prove that increasing advertising alone caused the increase in sales.

Other variables may influence both.

12. Other Regression Approaches

Linear regression is only one regression technique.

Other approaches include:

• Polynomial regression
• Ridge regression
• Lasso regression
• Decision-tree regression
• Random-forest regression
• Gradient-boosting regression
• Neural-network regression

The appropriate technique depends on:

• The structure of the data
• The relationship between variables
• Prediction requirements
• Interpretability requirements
• Model complexity
• Computational resources

13. Classification versus Regression

The distinction can be summarized as follows:

Dimension	Classification	Regression
Output	Category/class	Numerical value
Example	Churn/No Churn	Expected revenue
Target	Discrete	Continuous/numerical
Typical question	“Which class?”	“How much?”
Common algorithms	Logistic regression, decision trees, random forests	Linear regression, tree-based regression, random forests
Business application	Fraud detection	Sales forecasting

14. Selecting the Correct Model Type

A business analyst should begin by examining the target variable.

If the target is categorical:

Use a classification approach.

Example:

Will the customer churn?

Target:

Yes/No

→ Classification

If the target is numerical:

Use a regression approach.

Example:

What will the customer’s annual spending be?

Target:

KSh 75,000

→ Regression

This simple distinction prevents a common modeling mistake: selecting an algorithm before understanding the business outcome.

15. Example: Bank Loan Decisions

Consider a commercial bank.

Management wants to improve loan decision-making.

Problem A

Will the applicant default?

Target:

Default / No Default

This is a classification problem.

Problem B

What amount is the applicant likely to repay over the next year?

Target:

Numerical amount

This is a regression problem.

The same customer data can therefore support different machine learning tasks depending on the business question.

16. Example: Retail Customer Analytics

A retailer wants to improve its customer strategy.

Question 1

Which customers are likely to stop purchasing?

Classification

Question 2

How much will each customer spend next quarter?

Regression

Question 3

What types of customers exist?

Potentially:

Unsupervised clustering

This demonstrates how classification, regression and clustering can work together within a broader analytics programme.

17. Model Training

Before a classification or regression model can make predictions, it must be trained.

A simplified process is:

Historical Data

↓

Feature Selection and Preparation

↓

Training Dataset

↓

Algorithm

↓

Trained Model

↓

New Data

↓

Prediction

For supervised learning, the training data contains known outcomes.

The algorithm attempts to learn a relationship between the features and the target.

18. Training Error versus Generalization

A model can perform very well on its training data and still perform poorly on new observations.

For example:

Training performance: 98%

Test performance: 71%

This difference may indicate overfitting.

A business analyst should therefore avoid judging a model solely by its training performance.

The key question is:

How well does the model perform when it encounters new business data?

19. Classification Evaluation

Different metrics can be used to evaluate classification models.

Important measures include:

• Accuracy
• Precision
• Recall
• F1-score
• Specificity
• Area Under the ROC Curve (AUC)

The appropriate metric depends on the business problem.

19.1 Accuracy

Accuracy measures the proportion of predictions that are correct overall.

A simplified formula is:

Accuracy = Correct Predictions / Total Predictions

However, accuracy can be misleading when classes are highly imbalanced.

20. Precision

Precision focuses on the proportion of predicted positive cases that are actually positive.

For example, in fraud detection:

Of the transactions flagged as fraudulent, how many were actually fraudulent?

High precision means fewer false alarms among flagged cases.

This can be important where investigating each alert is costly.

21. Recall

Recall measures how many of the actual positive cases were successfully identified.

In fraud detection:

Of all genuinely fraudulent transactions, how many did the model detect?

A bank may prioritize recall when failing to detect fraudulent transactions is particularly costly.

This demonstrates an important business principle:

The “best” metric depends on the consequences of different types of errors.

22. False Positives and False Negatives

Classification decisions can produce different types of errors.

False Positive

The model predicts positive when the actual outcome is negative.

Example:

A legitimate transaction is incorrectly flagged as fraud.

False Negative

The model predicts negative when the actual outcome is positive.

Example:

A fraudulent transaction is incorrectly classified as legitimate.

These errors can have very different business costs.

For example:

Error	Possible Business Consequence
False positive fraud alert	Customer inconvenience and investigation cost
False negative fraud detection	Financial loss and reputational damage

Therefore, model evaluation should consider business consequences, not merely mathematical performance.

23. Regression Evaluation

Regression models require different evaluation measures.

Common metrics include:

• Mean Absolute Error (MAE)
• Mean Squared Error (MSE)
• Root Mean Squared Error (RMSE)
• R-squared

23.1 Mean Absolute Error

MAE represents the average absolute difference between actual and predicted values.

For example, if a sales model has an MAE of KSh 200,000, predictions differ from actual sales by approximately KSh 200,000 on average, subject to the interpretation of the dataset and metric.

MAE is relatively easy for business stakeholders to understand because it remains in the same units as the target.

24. Root Mean Squared Error

RMSE gives greater weight to larger errors because the errors are squared before averaging and then converted back to the original scale.

This can be useful when large prediction errors are particularly undesirable.

For example, a business forecasting system may want to penalize a prediction that misses actual demand by 10,000 units more heavily than several smaller errors.

25. R-Squared

R-squared provides an indication of how much of the variation in the target variable is explained by the model under the relevant modeling assumptions.

A higher R-squared does not automatically mean that a model is appropriate for every business decision.

A model can have a strong statistical fit while still being:

• Operationally impractical
• Poorly generalized
• Difficult to interpret
• Based on unreliable data
• Misaligned with the business objective

26. Model Interpretability

Model selection should consider not only predictive performance but also how easily decision-makers can understand the model.

For example:

A bank may prefer a model whose decisions can be explained to customers, regulators or internal risk committees.

A highly complex model might provide slightly better predictive performance but be difficult to explain.

Therefore, organizations sometimes accept a small reduction in predictive performance in exchange for:

• Greater transparency
• Easier auditing
• Easier governance
• Better stakeholder trust

The appropriate balance depends on the business context and regulatory environment.

27. Classification and Regression in Business Strategy

Classification and regression models can support different stages of decision-making.

Classification can answer:

• Who is likely to churn?
• Which transactions are suspicious?
• Which applicants are high risk?
• Which leads are likely to convert?

Regression can answer:

• How much will the customer spend?
• What will next month’s sales be?
• What will the expected claim cost be?
• How long will delivery take?

Both can therefore transform historical data into forward-looking information.

28. Combining Classification and Regression

A sophisticated business analytics system may use both approaches.

Consider an insurance company.

Step 1 — Classification

Estimate:

Is this customer likely to submit a claim?

Step 2 — Regression

For customers likely to submit a claim, estimate:

What is the expected claim amount?

The business can then estimate expected exposure using both outputs.

Similarly, a retailer could:

1. Classify customers according to purchase likelihood.
2. Predict expected purchase value using regression.

This illustrates how different machine learning models can complement one another.

29. International Business Example: Credit Risk

Financial institutions use predictive models to support credit-risk decisions.

A classification model can estimate the likelihood that a borrower will default.

A regression model could estimate a numerical financial outcome, such as:

• Expected loss
• Expected exposure
• Potential repayment amount

The exact model design depends on the institution’s risk framework, data, regulatory requirements and decision process.

The key analytical lesson is that different business questions require different target structures and therefore potentially different modeling approaches.

30. Practical Model Selection Framework

A business analyst can use the following process.

Step 1: Define the business decision

What decision needs to be improved?

Step 2: Define the target

What exactly should the model predict?

Step 3: Identify target type

Is it:

• Binary?
• Multiclass?
• Multilabel?
• Continuous numerical?

Step 4: Select candidate algorithms

Choose approaches appropriate to the problem and data.

Step 5: Train the models

Use historical data appropriately.

Step 6: Evaluate

Use suitable metrics and validation procedures.

Step 7: Consider business consequences

Examine the costs of different prediction errors.

Step 8: Select the appropriate model

Balance:

• Performance
• Interpretability
• Cost
• Risk
• Scalability
• Business value

Step 9: Deploy and monitor

Track performance after implementation.

31. Common Mistakes

Business analysts and organizations should avoid several common mistakes.

Mistake 1: Choosing the Algorithm First

Starting with “Which algorithm should we use?” before defining the business problem can result in inappropriate modeling.

Mistake 2: Using Accuracy Alone

Accuracy may be inadequate for highly imbalanced classification problems.

Mistake 3: Ignoring Business Costs

False positives and false negatives may have very different consequences.

Mistake 4: Overfitting

A model that performs well on historical data may fail on new observations.

Mistake 5: Confusing Correlation with Causation

A predictive relationship does not necessarily establish a causal relationship.

Mistake 6: Ignoring Data Quality

A sophisticated model cannot compensate indefinitely for fundamentally poor-quality data.

Mistake 7: Ignoring Interpretability

A technically strong model may be unsuitable when stakeholders need transparent explanations.

32. Practical Business Scenario

A supermarket chain wants to improve sales and customer retention.

Management identifies three analytical questions:

Question A

Will a customer make a purchase next month?

Classification

Possible target:

Purchase / No Purchase

Question B

How much will the customer spend next month?

Regression

Possible target:

Expected spending in KSh

Question C

What groups of customers behave similarly?

Unsupervised clustering

This example demonstrates how different machine learning approaches can address different questions within the same business environment.

Reflection Exercise

A commercial bank wants to improve its analytical decision-making.

Management provides the following objectives:

1. Predict whether a loan applicant will default.
2. Estimate the expected annual value of a customer.
3. Predict whether a transaction is fraudulent.
4. Estimate the expected value of a fraudulent loss.
5. Identify groups of customers with similar behavior.

Reflection Questions

For each objective:

1. Is the problem classification, regression or unsupervised learning?
2. What would the target variable be?
3. What features could potentially be used?
4. Which type of error might be particularly costly?
5. What evaluation measure could be appropriate?
6. What business decision would the model support?

Best Practices

Organizations developing classification and regression models should:

1. Define the business objective before selecting an algorithm.
2. Clearly define the target variable.
3. Distinguish categorical outcomes from numerical outcomes.
4. Use appropriate training, validation and test procedures.
5. Evaluate models using metrics suited to the business problem.
6. Consider false-positive and false-negative costs.
7. Do not rely on accuracy alone for imbalanced datasets.
8. Check for overfitting and poor generalization.
9. Consider interpretability and governance requirements.
10. Avoid interpreting predictive association as automatic evidence of causation.
11. Validate models using relevant business stakeholders.
12. Monitor model performance after deployment.
13. Review models when market conditions or customer behavior change.
14. Document assumptions, limitations and decision thresholds.

Lesson Summary

Classification and regression are two major forms of supervised machine learning.

Classification

Classification predicts a category.

Examples include:

• Fraud versus legitimate
• Churn versus no churn
• High-risk versus low-risk
• Approved versus rejected

Classification may involve:

• Binary classification
• Multiclass classification
• Multilabel classification

Common classification techniques include logistic regression, decision trees, random forests and other machine learning algorithms.

Regression

Regression predicts a numerical outcome.

Examples include:

• Sales revenue
• Customer lifetime value
• Property prices
• Claim costs
• Delivery times

Linear regression is a foundational regression technique, while other approaches include tree-based and regularized regression methods.

The most important distinction is:

Classification predicts “which category?” while regression predicts “how much?”

Model evaluation must also reflect the business context. Classification may require metrics such as precision and recall, while regression may use MAE, RMSE and R-squared.

Most importantly, organizations should not select models based solely on technical performance. Business objectives, data quality, interpretability, risk, cost and the consequences of prediction errors must all be considered.

References

1. scikit-learn — Supervised Learning
2. scikit-learn — Linear Models
3. scikit-learn — Decision Trees
4. Google for Developers — Machine Learning Glossary
5. IBM — Classification in Machine Learning
6. IBM — Regression in Machine Learning
7. Microsoft Learn — Machine Learning Fundamentals