python - Get nested arrays out of a dictionary

Question

I'm trying to parse the results of a simulation to extract all of the results that are numpy arrays. When simulating over a simple material, we might get a single dictionary with array values:

{'material1':array, 'material2':array, ...}

In more complex material simulations, we end up with nested dictionaries like:

{'material1': {'shellmaterial':array, 'corematerial':array}}

The depth of the nesting is arbitrary, and what I want to do is create a plot where all of the available arrays are returned to the user, named by their nesting. So for example,the above structure would end up like:

{'material1.shellmaterial' : array, 'material1.corematerial' : array}

We'd then put these in a dropdown menu. for easy viewing in a plot. Does anyone have a good way to iterate through an arbitrarily nested dictionary and return only the array type values with the new keys as shown above?

Results have to be stored this way for json compatibility, so I can't really go back and refactor to avoid this.

Answer 1

Here is a function I used with some decision trees for language processing, it's not what you want, but it's the same basic idea.

def nodeMapForDict(d):
    node_map = []
    node_path = [] 
    def nodeRecursiveMap(d, node_path): 
        for key, val in d.items():
            if type(val) is not dict: node_map.append(node_path + [key]) 
            if type(val) is dict: 
                nodeRecursiveMap(val, node_path + [key])
    nodeRecursiveMap(d, node_path)
    return node_map

And here is one that should fit your use case:

def flattenDict(d):
    node_map = {}
    node_path = [] 
    def nodeRecursiveMap(d, node_path): 
        for key, val in d.items():
            if type(val) is not dict: node_map['.'.join(node_path + [key])] = val 
            if type(val) is dict: 
                nodeRecursiveMap(val, node_path + [key])
    nodeRecursiveMap(d, node_path)
    return node_map

example:

d= {'d': [1, 2, 3, 4], 'e': {'b': {'c': 1}}, 'a': {'b': 'c'}}
In [49]: flattenDict(d)
Out[49]: {'d': [1, 2, 3, 4], 'e.b.c': 1, 'a.b': 'c'}