我想找出我的数据的每一列中NaN的数量。
当前回答
对于你的任务,你可以使用pandas.DataFrame.dropna (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.dropna.html):
import pandas as pd
import numpy as np
df = pd.DataFrame({'a': [1, 2, 3, 4, np.nan],
'b': [1, 2, np.nan, 4, np.nan],
'c': [np.nan, 2, np.nan, 4, np.nan]})
df = df.dropna(axis='columns', thresh=3)
print(df)
使用thresh参数,您可以声明DataFrame中所有列的NaN值的最大计数。
代码输出:
a b
0 1.0 1.0
1 2.0 2.0
2 3.0 NaN
3 4.0 4.0
4 NaN NaN
其他回答
请使用以下方法计算特定的列数
dataframe.columnName.isnull().sum()
你可以从非nan值的计数中减去总长度:
count_nan = len(df) - df.count()
你应该根据你的数据计算时间。与isnull解相比,小级数的速度提高了3倍。
你可以试试:
In [1]: s = pd.DataFrame('a'=[1,2,5, np.nan, np.nan,3],'b'=[1,3, np.nan, np.nan,3,np.nan])
In [4]: s.isna().sum()
Out[4]: out = {'a'=2, 'b'=3} # the number of NaN values for each column
如果需要nan的总和:
In [5]: s.isna().sum().sum()
Out[6]: out = 5 #the inline sum of Out[4]
根据给出的答案和一些改进,这是我的方法
def PercentageMissin(Dataset):
"""this function will return the percentage of missing values in a dataset """
if isinstance(Dataset,pd.DataFrame):
adict={} #a dictionary conatin keys columns names and values percentage of missin value in the columns
for col in Dataset.columns:
adict[col]=(np.count_nonzero(Dataset[col].isnull())*100)/len(Dataset[col])
return pd.DataFrame(adict,index=['% of missing'],columns=adict.keys())
else:
raise TypeError("can only be used with panda dataframe")
import pandas as pd
import numpy as np
# example DataFrame
df = pd.DataFrame({'a':[1,2,np.nan], 'b':[np.nan,1,np.nan]})
# count the NaNs in a column
num_nan_a = df.loc[ (pd.isna(df['a'])) , 'a' ].shape[0]
num_nan_b = df.loc[ (pd.isna(df['b'])) , 'b' ].shape[0]
# summarize the num_nan_b
print(df)
print(' ')
print(f"There are {num_nan_a} NaNs in column a")
print(f"There are {num_nan_b} NaNs in column b")
给出输出:
a b
0 1.0 NaN
1 2.0 1.0
2 NaN NaN
There are 1 NaNs in column a
There are 2 NaNs in column b