Pandas for Data Analytics · Lesson 9 of 20
Handling Missing Data
Learn how to identify missing values, understand their impact, and make a reasoned decision about how they should be handled.
Concept
What Is Missing Data?
Missing data occurs when a value that should be available is absent. For example, a customer record may have a missing Age, a transaction may have no Discount value, or a survey response may leave a question unanswered.
Customer DataFrame
An em dash represents a value that is absent.
| Customer | Name | Age | Revenue |
|---|---|---|---|
| C101 | Rahul | 32 | ₹4,500 |
| C102 | Priya | — | ₹3,800 |
| C103 | Amit | 41 | ₹5,200 |
| C104 | Neha | — | ₹2,900 |
Your Turn
A customer DataFrame has 10,000 records, but some customers have no value recorded in the Age column. What does this represent?
Analyst Workflow
From Missing Value to Data Decision
Detect
Find missing values
Measure
Understand their extent
Investigate
Consider context and cause
Treat
Choose an appropriate approach
Handling Missing Data in Pandas
Missing data is a common issue in real-world datasets. Customer, sales, finance, survey, and operational data can contain fields where information was not recorded or is unavailable.
Pandas provides tools such as isna() and isnull() for identifying missing values. Analysts can then examine how much missing information exists and where it occurs.
Pandas also provides dropna() for removing rows or columns containing missing values. However, missing-data treatment should be based on the business context and analytical purpose rather than applying an automatic rule to every dataset.
A useful missing-data workflow is to detect the missing values, measure their extent, investigate the context, and then choose an appropriate treatment.