我有一个pandas数据框架,其中一列文本字符串包含逗号分隔的值。我想拆分每个CSV字段,并为每个条目创建一个新行(假设CSV是干净的,只需要在','上拆分)。例如,a应该变成b:

In [7]: a
Out[7]: 
    var1  var2
0  a,b,c     1
1  d,e,f     2

In [8]: b
Out[8]: 
  var1  var2
0    a     1
1    b     1
2    c     1
3    d     2
4    e     2
5    f     2

到目前为止,我已经尝试了各种简单的函数,但是.apply方法在轴上使用时似乎只接受一行作为返回值,而且我不能让.transform工作。任何建议都将不胜感激!

示例数据:

from pandas import DataFrame
import numpy as np
a = DataFrame([{'var1': 'a,b,c', 'var2': 1},
               {'var1': 'd,e,f', 'var2': 2}])
b = DataFrame([{'var1': 'a', 'var2': 1},
               {'var1': 'b', 'var2': 1},
               {'var1': 'c', 'var2': 1},
               {'var1': 'd', 'var2': 2},
               {'var1': 'e', 'var2': 2},
               {'var1': 'f', 'var2': 2}])

我知道这不会起作用,因为我们通过numpy丢失了DataFrame元数据,但它应该给你一个我试图做的感觉:

def fun(row):
    letters = row['var1']
    letters = letters.split(',')
    out = np.array([row] * len(letters))
    out['var1'] = letters
a['idx'] = range(a.shape[0])
z = a.groupby('idx')
z.transform(fun)

当前回答

使用赋值和爆炸的一行代码:

    col1  col2
0  a,b,c     1
1  d,e,f     2

df.assign(col1 = df.col1.str.split(',')).explode('col1', ignore_index=True)

输出:

  col1  col2
0    a     1
1    b     1
2    c     1
3    d     2
4    e     2
5    f     2

其他回答

Try:

vals = np.array(a.var1.str.split(",").values.tolist())    
var = np.repeat(a.var2, vals.shape[1])

out = pd.DataFrame(np.column_stack((var, vals.ravel())), columns=a.columns)
display(out)

      var1 var2
    0   1   a
    1   1   b
    2   1   c
    3   2   d
    4   2   e
    5   2   f

我提出了一个具有任意列数的数据框架的解决方案(同时一次仍然只分离一列的条目)。

def splitDataFrameList(df,target_column,separator):
    ''' df = dataframe to split,
    target_column = the column containing the values to split
    separator = the symbol used to perform the split

    returns: a dataframe with each entry for the target column separated, with each element moved into a new row. 
    The values in the other columns are duplicated across the newly divided rows.
    '''
    def splitListToRows(row,row_accumulator,target_column,separator):
        split_row = row[target_column].split(separator)
        for s in split_row:
            new_row = row.to_dict()
            new_row[target_column] = s
            row_accumulator.append(new_row)
    new_rows = []
    df.apply(splitListToRows,axis=1,args = (new_rows,target_column,separator))
    new_df = pandas.DataFrame(new_rows)
    return new_df

我有一个类似的问题,我的解决方案是将数据帧转换为字典列表,然后进行转换。函数如下:

import re
import pandas as pd

def separate_row(df, column_name):
    ls = []
    for row_dict in df.to_dict('records'):
        for word in re.split(',', row_dict[column_name]):
            row = row_dict.copy()
            row[column_name]=word
            ls.append(row)
    return pd.DataFrame(ls)

例子:

>>> from pandas import DataFrame
>>> import numpy as np
>>> a = DataFrame([{'var1': 'a,b,c', 'var2': 1},
               {'var1': 'd,e,f', 'var2': 2}])
>>> a
    var1  var2
0  a,b,c     1
1  d,e,f     2
>>> separate_row(a, "var1")
  var1  var2
0    a     1
1    b     1
2    c     1
3    d     2
4    e     2
5    f     2

您还可以稍微更改该函数以支持分离列表类型行。

一种使用.apply()改变列格式的简单方法,以便.explod()可以使用它:

import string
import pandas as pd
from io import StringIO

file = StringIO("""    var1  var2
0  a,b,c     1
1  d,e,f     2""")

df = pd.read_csv(file, sep=r'\s\s+')

df['var1'] = df['var1'].apply(lambda x : str(x).split(','))

df.explode('var1')

输出:

  var1  var2
0   a   1
0   b   1
0   c   1
1   d   2
1   e   2
1   f   2

有可能在不改变数据框架结构的情况下拆分和爆炸数据框架

拆分和展开特定列的数据

输入:

    var1    var2
0   a,b,c   1
1   d,e,f   2



#Get the indexes which are repetative with the split 
df['var1'] = df['var1'].str.split(',')
df = df.explode('var1')

Out:

    var1    var2
0   a   1
0   b   1
0   c   1
1   d   2
1   e   2
1   f   2

Edit-1

对多列的行进行拆分和展开

Filename    RGB                                             RGB_type
0   A   [[0, 1650, 6, 39], [0, 1691, 1, 59], [50, 1402...   [r, g, b]
1   B   [[0, 1423, 16, 38], [0, 1445, 16, 46], [0, 141...   [r, g, b]

基于参考列重新索引,并将列值信息与堆栈对齐

df = df.reindex(df.index.repeat(df['RGB_type'].apply(len)))
df = df.groupby('Filename').apply(lambda x:x.apply(lambda y: pd.Series(y.iloc[0])))
df.reset_index(drop=True).ffill()

Out:

                Filename    RGB_type    Top 1 colour    Top 1 frequency Top 2 colour    Top 2 frequency
    Filename                            
 A  0       A   r   0   1650    6   39
    1       A   g   0   1691    1   59
    2       A   b   50  1402    49  187
 B  0       B   r   0   1423    16  38
    1       B   g   0   1445    16  46
    2       B   b   0   1419    16  39