NaN or null values are undesired in any dataset. In this article, we will discuss different ways to replace nan with 0 in a pandas dataframe and series.
We can use three approaches to replace nan with 0 in a series, dataframe, or column in a dataframe. Let us discuss all the approaches one by one.
Pandas Replace NaN With 0 Using the fillna() Method
The pandas fillna method is used to replace nan values in a dataframe or series. To replace nan with 0 in a series, you can invoke the fillna() method on the series. Also, you need to pass 0 as the input argument to the fillna() method. After execution, the fillna() method returns a new pandas series with all the nan values replaced by 0.
You can observe this in the following example.
import pandas as pd
import numpy as np
numbers=[100,90,np.nan,90,pd.NA,100,None]
series=pd.Series(numbers)
print("The series is:")
print(series)
series=series.fillna(0)
print("The updated series is:")
print(series)Output:
The series is:
0 100
1 90
2 NaN
3 90
4 <NA>
5 100
6 None
dtype: object
The updated series is:
0 100
1 90
2 0
3 90
4 0
5 100
6 0
dtype: int64In the above example, the input Series contains numpy.nan, pd.NA, as well as None value. In the output you can observe that the fillna() method can replace all kinds of NaN values from the series.
Substitute Null With 0 in a Pandas Dataframe
To replace nan with 0 in pandas dataframe using the fillna() method, you can invoke the fillna() method on the dataframe as shown below.
import pandas as pd
import numpy as np
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df=df.fillna(0)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 0.0
2 3 90 80.0 70.0
3 4 0 0.0 90.0
4 5 90 90.0 0.0
5 6 80 0.0 70.0In this example, we have converted a list of dictionaries to dataframe. Then, we have used the fillna() method to replace the nan values using zeroes in the dataframe.
Replace Null With 0 in a Column in a Pandas Dataframe
To replace nan with 0 in a column in a pandas dataframe, you first need to select the column using the indexing operator. After this, you can invoke the fillna() method on the column. After execution of the fillna() method, you will get the modified column as shown below.
import pandas as pd
import numpy as np
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df["Maths"]=df["Maths"].fillna(0)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 NaN
2 3 90 80.0 70.0
3 4 0 NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0In this example, we have replaced the nan values from the "Maths" column in the given dataframe. For this, we have first selected the column using the python indexing operator. Then, we have used the fillna() method to replace null values with 0 in the column.
Replace Null With 0 in Multiple Columns in a Pandas Dataframe
Instead of a single column, you can replace nan with 0 in multiple columns of pandas dataframe using the fillna() method as shown in the following example.
import pandas as pd
import numpy as np
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df[["Maths","Physics","Chemistry"]]=df[["Maths","Physics","Chemistry"]].fillna(0)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 0.0
2 3 90 80.0 70.0
3 4 0 0.0 90.0
4 5 90 90.0 0.0
5 6 80 0.0 70.0In this example, we have replaced null values with 0 in the columns "Maths", "Physics", and "Chemistry" using the fillna() method.
Pandas Replace NaN With 0 Using the replace() Method
Instead of using the fillna() method, you can also use the replace() method to replace nan with 0 in a dataframe column, a series, or a dataframe.
To replace nan with 0 in a series using the replace() method, you first need to invoke the replace() method on the series. Here, we need to give numpy.nan value as the first input argument and 0 as the second input argument. After execution, the pandas replace method will return a series having all the nan values replaced by 0s. You can observe this in the following example.
import pandas as pd
import numpy as np
numbers=[100,90,np.nan,90,pd.NA,100,None]
series=pd.Series(numbers)
print("The series is:")
print(series)
series=series.replace(np.nan, 0)
print("The updated series is:")
print(series)Output:
The series is:
0 100
1 90
2 NaN
3 90
4 <NA>
5 100
6 None
dtype: object
The updated series is:
0 100
1 90
2 0
3 90
4 0
5 100
6 0
dtype: int64You can also use the replace() method to replace the nan value with a 0 in the pandas dataframe as shown below.
import pandas as pd
import numpy as np
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df=df.replace(np.nan,0)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 0.0
2 3 90 80.0 70.0
3 4 0 0.0 90.0
4 5 90 90.0 0.0
5 6 80 0.0 70.0Instead of the entire dataframe, you can also replace nan values from a specific column using the replace() method as shown below.
import pandas as pd
import numpy as np
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df["Maths"]=df["Maths"].replace(np.nan,0)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 NaN
2 3 90 80.0 70.0
3 4 0 NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0You can also replace nan values with 0 in multiple columns using the replace() method. For this, you first need to select the columns where you want to replace the nan values with 0 and invoke the replace() method on the selected columns as shown below.
import pandas as pd
import numpy as np
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df[["Maths","Physics","Chemistry"]]=df[["Maths","Physics","Chemistry"]].replace(np.nan,0)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 0.0
2 3 90 80.0 70.0
3 4 0 0.0 90.0
4 5 90 90.0 0.0
5 6 80 0.0 70.0Replace Null Values in Pandas With 0 Using the apply() Method
The pandas apply method is used to apply a specific function to the series or a column in the pandas dataframe. We will first create a function with the following characteristics to replace nan values in a series or a dataframe column using the apply() method.
It takes a value as its input argument and checks for NaN value using pd.isna() function. If it gets a NaN value, it returns 0. Otherwise, it returns the input value.
To replace the nan value with 0 in a series using the apply() method, we will invoke the apply() method on the series and pass the function as an input argument. After execution of the apply() method, we will get the modified series as shown below.
import pandas as pd
import numpy as np
def replacenan(value):
if pd.isna(value):
return 0
else:
return value
numbers=[100,90,np.nan,90,pd.NA,100,None]
series=pd.Series(numbers)
print("The series is:")
print(series)
series=series.apply(replacenan)
print("The updated series is:")
print(series)Output:
The series is:
0 100
1 90
2 NaN
3 90
4 <NA>
5 100
6 None
dtype: object
The updated series is:
0 100
1 90
2 0
3 90
4 0
5 100
6 0
dtype: int64In this example, we have first defined the replacenan() function which takes the parameter value as its input argument. It first checks for nan value using the pd.isna() method. If the value is a null value, it returns 0. Otherwise, it returns the value itself.
Then, we have passed the replacenan() function to the apply() method to replace nan values from the series.
In a similar manner, you can replace nan with 0 in a dataframe column using the apply() method as shown in the following example.
import pandas as pd
import numpy as np
def replacenan(value):
if pd.isna(value):
return 0
else:
return value
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df["Maths"]=df["Maths"].apply(replacenan)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 NaN
2 3 90 80.0 70.0
3 4 0 NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0The apply() method doesn’t work with a dataframe or multiple columns of the same if the function passed to the apply() method is not serializable. Therefore, to replace nan values with 0 from a dataframe or multiple columns, you can use the pandas applymap method as discussed in the following section.
Pandas Replace Nan With 0 Using the applymap() Method
To replace the nan value with 0 in a dataframe using the applymap() method, we will invoke the applymap() method on the dataframe and pass the above function as an input argument. After execution of the applymap() method, we will get the modified dataframe as shown below.
import pandas as pd
import numpy as np
def replacenan(value):
if pd.isna(value):
return 0
else:
return value
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df=df.applymap(replacenan)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 0.0
2 3 90 80.0 70.0
3 4 0 0.0 90.0
4 5 90 90.0 0.0
5 6 80 0.0 70.0In a similar manner, you can replace nan with 0 in multiple columns in a dataframe using the applymap() method as shown in the following example.
import pandas as pd
import numpy as np
def replacenan(value):
if pd.isna(value):
return 0
else:
return value
myDicts=[{"Roll":1,"Maths":100, "Physics":80, "Chemistry": 90},
{"Roll":2,"Maths":np.nan, "Physics":100, "Chemistry": np.nan},
{"Roll":3,"Maths":90, "Physics":80, "Chemistry": 70},
{"Roll":4,"Maths":pd.NA, "Physics":np.nan, "Chemistry": 90},
{"Roll":5,"Maths":90, "Physics":90, "Chemistry": None},
{"Roll":6,"Maths":80, "Physics":np.nan, "Chemistry": 70}]
df=pd.DataFrame(myDicts)
print("The input dataframe is:")
print(df)
df[["Maths","Physics","Chemistry"]]=df[["Maths","Physics","Chemistry"]].applymap(replacenan)
print("The updated dataframe is:")
print(df)Output:
The input dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 NaN 100.0 NaN
2 3 90 80.0 70.0
3 4 <NA> NaN 90.0
4 5 90 90.0 NaN
5 6 80 NaN 70.0
The updated dataframe is:
Roll Maths Physics Chemistry
0 1 100 80.0 90.0
1 2 0 100.0 0.0
2 3 90 80.0 70.0
3 4 0 0.0 90.0
4 5 90 90.0 0.0
5 6 80 0.0 70.0Conclusion
In this article, we have discussed different approaches to replace null values in a pandas dataframe and series.
To learn more about python programming, you can read this article on how to sort a pandas dataframe. You might also like this article on how to drop columns from a pandas dataframe.
I hope you enjoyed reading this article. Stay tuned for more informative articles.
Happy Learning!
Recommended Python Training
Course: Python 3 For Beginners
Over 15 hours of video content with guided instruction for beginners. Learn how to create real world applications and master the basics.