过滤嵌套列表

Question

我有如下嵌套列表：-

 list = [['A:1','B:(null)','C:3','D:4'],
        ['A:1','B:abc','C:6','D:7'],
        ['A:1','B:def','C:2','G:44','E: 600','F: 6600'],
        ['A:1','B:ghi','C:33','D:44']]

我想把它转换成一个数据框，这样before : 就是column name 和after : 的值

这里我有两种类型的数据，一种是：-

[['A:1','B:(null)','C:3','D:4'],
  ['A:1','B:abc','C:6','D:7'],
  ['A:1','B:ghi','C:33','D:44']]

其中一项是不同的，即

['A:1','B:def','C:2','G:44','E: 600','F: 6600']

预期输出：-

df1 =

和 df2 =

Q.1) 到目前为止，我只有两种数据，所以我想要两个数据框。 Q.2）我们可以让它动态化，以便它根据列表中的项目创建多个 dfs。

Answer 1

IIUC，第一次将您的list 转换为dict 的list（也不要将您的列表命名为列表，它会覆盖python 函数），第二次创建数据框，使用isnull 和dot创建组键，然后构建字典，我不建议动态创建数据框，您可以将它们放入dict，如果有问题请查看local

[dict(tuple(y.split(":")) for y in x )for x in l] # make you list to list of dict 
Out[11]: 
[{'A': '1', 'B': '(null)', 'C': '3', 'D': '4'},
 {'A': '1', 'B': 'abc', 'C': '6', 'D': '7'},
 {'A': '1', 'B': 'def', 'C': '2', 'E': ' 600', 'F': ' 6600', 'G': '44'},
 {'A': '1', 'B': 'ghi', 'C': '33', 'D': '44'}]
newl=[dict(tuple(y.split(":")) for y in x )for x in l]
pd.DataFrame(newl)
Out[13]: 
   A       B   C    D     E      F    G
0  1  (null)   3    4   NaN    NaN  NaN
1  1     abc   6    7   NaN    NaN  NaN
2  1     def   2  NaN   600   6600   44
3  1     ghi  33   44   NaN    NaN  NaN
newdf=pd.DataFrame(newl)
s=newdf.isnull().dot(newdf.columns)# using dot create the groupby key 
s
Out[16]: 
0    EFG
1    EFG
2      D
3    EFG
dtype: object

---

{x: y for x , y  in newdf.groupby(s)}# using group by create the dict 
Out[17]: 
{'D':    A    B  C    D     E      F   G
 2  1  def  2  NaN   600   6600  44, 'EFG':    A       B   C   D    E    F    G
 0  1  (null)   3   4  NaN  NaN  NaN
 1  1     abc   6   7  NaN  NaN  NaN
 3  1     ghi  33  44  NaN  NaN  NaN}
d={x: y for x , y  in newdf.groupby(s)}
d['D'].dropna(1,thresh=1)
# result can using dict selection
# dropna here means atleast one column should have at least one not null value , 
# if it is all null , then we drop the entire columns  
Out[19]: 
   A    B  C    E      F   G
2  1  def  2  600   6600  44

d['EFG'].dropna(1,thresh=1)
Out[21]: 
   A       B   C   D
0  1  (null)   3   4
1  1     abc   6   7
3  1     ghi  33  44

---

不推荐local

d={x: y.dropna(1,thresh=1) for x , y  in newdf.groupby(s)}
variables = locals()
for i,j in enumerate(d.values()):
    variables["df{0}".format(i+1)] = j
df1
Out[26]: 
   A    B  C     E      F   G
2  1  def  2   600   6600  44
df2
Out[27]: 
   A       B   C   D
0  1  (null)   3   4
1  1     abc   6   7
3  1     ghi  33  44

Answer 2

你可以：

1. 从您的列表中创建字典（我选择将 "(null)" 替换为 None）
2. 通过collections.defaultdict按排序键分组dicts
3. 从组中创建和生成数据框

---

from collections import defaultdict
import pandas as pd

# convert to dictionaries        
def makeDict(inner): 
    return {k: (v if v!= "(null)" else None) for k,v in (p.split(":") for p in inner)}

# group and yield dfs
def makeIt(l):
    # collect data as dicts
    dicts = []
    for inner in l:
        dicts.append( makeDict(inner))

    # group by sorted keys
    t = defaultdict(list)
    for d in dicts:
        t[tuple(sorted(d.keys()))].append(d)

    # create dataframes from groups and yield them
    for k in t:
        df = pd.DataFrame(t[k])
        yield df

用法：

l = [['A:1','B:(null)','C:3','D:4'],
     ['A:1','B:abc','C:6','D:7'],
     ['A:1','B:def','C:2','G:44','E: 600','F: 6600'],
     ['A:1','B:ghi','C:33','D:44']]

dfs = list(makeIt(l))

for df in dfs:
    print("-"*20)
    print(df)

输出：

--------------------
   A     B   C   D
0  1  None   3   4
1  1   abc   6   7
2  1   ghi  33  44

--------------------
   A    B  C     E      F   G
0  1  def  2   600   6600  44