python 3.x - list of dictionary of list of dictionary to a DataFrame with dynamic header

Question

data = [
    {
        "node_id"   : "1212",
        "name"      : "Adam",
        "country"   : "USA",
        "parent_ids": [
            {"parent_id": "123abc", "relation": "member"},
            {"parent_id": "124abc", "relation": "previous_member"},
        ],
    },
    {
        "node_id"   : "1213",
        "name"      : "Jane",
        "country"   : "USA",
        "parent_ids": [
            {"parent_id": "125abc", "relation": "member"},
        ],
    },
    {
        "node_id"   : "1214",
        "name"      : "Harry",
        "country"   : "UK",
        "parent_ids": [
            {"parent_id": "126abc", "relation": "member"},
            {"parent_id": "127abc", "relation": "previous_member"},
            {"parent_id": "128abc", "relation": "previous_member"},
        ],
    },
    {
        "node_id"   : "1215",
        "name"      : "Jimmy",
        "country"   : "USA",
        "parent_ids": [
            {"parent_id": "129abc", "relation": "member"},
            {"parent_id": "129abd", "relation": "previous_member"},
            {"parent_id": "129abe", "relation": "previous_member"},
            {"parent_id": "129abf", "relation": "member"},
        ],
    },
]

desired dataframe

parent_id_1	relation_1	parent_id_2	relation_2	parent_id_3	relation_3	parent_id_4	relation_4	name	node_id
123abc	member	124abc	previous_member	NaN	NaN	NaN	NaN	Adam	1212
125abc	previous_member	NaN	NaN	NaN	NaN	NaN	NaN	Jane	1213
126abc	member	127abc	previous_member	128abc	previous_member	NaN	NaN	Harry	1214
129abc	member	129abd	previous_member	129abe	previous_member	129abf	previous_member	Jimmy	1215

question from:https://stackoverflow.com/questions/66065917/list-of-dictionary-of-list-of-dictionary-to-a-dataframe-with-dynamic-header

Answer 1

What about

df = pd.DataFrame(
    [
        {
            **d,
            **dict(sum(
                [
                    [
                        (f'{sub_k}_{i}', sub_v)
                        for sub_k, sub_v in sub_d.items()
                    ] for i, sub_d in enumerate(
                        d.pop('parent_ids'), start=1
                    )
                ],
                []
            ))
        } for d in data
    ]
)

which gives

  node_id   name country parent_id_1 relation_1 parent_id_2       relation_2  
0    1212   Adam     USA      123abc     member      124abc  previous_member   
1    1213   Jane     USA      125abc     member         NaN              NaN   
2    1214  Harry      UK      126abc     member      127abc  previous_member   
3    1215  Jimmy     USA      129abc     member      129abd  previous_member   

  parent_id_3       relation_3 parent_id_4 relation_4  
0         NaN              NaN         NaN        NaN  
1         NaN              NaN         NaN        NaN  
2      128abc  previous_member         NaN        NaN  
3      129abe  previous_member      129abf     member 

You can type pd.set_option('display.max_columns', None) to make df fit entirely within your screen.