Learning Objectives

By the end of this lesson, learners should be able to:

  • Explain the purpose of data cleaning.
  • Identify common data quality problems.
  • Apply techniques for handling missing values.
  • Detect duplicates and inconsistencies.
  • Prepare cleaner datasets for analysis.

Why Data Cleaning Is Important

Analytical results are only as reliable as the underlying data. Dirty data can cause incorrect reports, inaccurate forecasts, failed marketing campaigns, operational inefficiencies, financial losses, and compliance issues.

Common Data Quality Problems

Missing Values

Examples:

  • Missing customer email addresses,
  • Missing transaction dates,
  • Missing sales amounts.

Duplicate Records

The same customer appears multiple times.

Inconsistent Formats

  • 2026-03-01,
  • 01/03/2026,
  • Mar 1, 2026.

Typographical Errors

  • “Singpore” instead of “Singapore”.

Outliers

Unusually large or small values that may indicate data entry errors or exceptional business events.

Handling Missing Values

Deletion

Remove records when only a small number of observations are affected.

Imputation

Replace missing values with:

  • Mean,
  • Median,
  • Mode,
  • Predicted values.

International Example

A hotel group operating in Dubai, Singapore, and Sydney replaces missing room rates with the median rate for the same hotel category and season.

Removing Duplicates

Use unique identifiers such as customer ID, booking number, or invoice number.

Standardizing Formats

Ensure consistent formatting for dates, currencies, phone numbers, and text.

Outlier Detection

Methods include:

  • Box plots,
  • Z-scores,
  • Interquartile range (IQR).

Outliers should be investigated before removal because they may represent important business events.

Learning Materials / Reference Materials

  • Rahm & Do. Data Cleaning: Problems and Current Approaches.
  • OpenRefine Documentation.
  • ISO 8000 Data Quality Standards.

Lesson Summary

Data cleaning improves accuracy, consistency, completeness, and reliability, ensuring that business analytics results are trustworthy and useful for decision-making.

PreviousNext