json - how to match an array value by it's key in a key value pair elasticsearch array?

Question

I have an array of key value pairs. Is it possible to exact match value of key & then do a check on it's value's range value?

Example: In below doc oracle_props is an array with name, value pairs. I need to check if it has "oracle_cursors" key and then check if it's value is less than 1000.

GET /eg/message/_percolate
{
   "doc": {
      "client": {
         "name": "Athena",
         "version": 1,
         "db": {
            "@type": "Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 64bit",
            "oracle_props": [
               {
                  "@name": "open_cursors",
                  "@value": 4000
               },
               {
                  "@name": "USER_ROLE_PRIVS_COUNT",
                  "@value": 1
               },
               {
                  "@name": "CREATE_PERMISSION",
                  "@value": "Y"
               }
            ]
         }
      }
   }
}

Below is my percolator.

I also need to check the following so that it gives back 3 as my result

1. "client.name" must be "Athena"
2. "client.db.@type" must be "Oracle" then only go ahead and check below points
3. "client.db.oracle_props.@name" field is not found
4. check if it has "oracle_cursors" key and then check if it's value is < 1000

1 & 2 are and operations and any of 3 or 4 satisfies it should result 3. I need help with point 4, below is my query. Also please suggest if there is a better way.

PUT /eg/.percolator/3
{
   "query": {
      "filtered": {
         "filter": {
            "or": [
               {
                  "missing": {
                     "field": "client.db.oracle_props.@name"
                  }
               }
            ]
         },
         "query": {
            "bool": {
               "must": [
                  {
                     "match": {
                        "client.name": "Athena"
                     }
                  },
                  {
                     "match": {
                        "client.db.@type": "Oracle"
                     }
                  }
               ]
            }
         }
      }
   }
}

Update

Can I have something like below

{
     "match": {
                    "client.db.oracle_props[name='open_cursors'].value": 4000
                 }
              }

More tries

I followed elasticsearch nested query and changed the mapping to nestedtype by re-indexing. Can anyone find problem why am i getting nested: NullPointerException;?

PUT /eg/.percolator/3
{
   "nested" : {
        "path" : "client.db.oracle_props",
        "score_mode" : "avg",
        "query" : {
            "bool" : {
                "must" : [
                    {
                        "match" : {"client.db.oracle_props.@name" : "open_cursors"}
                    },
                    {
                        "range" : {"client.db.oracle_props.@value" : {"lt" : 4000}}
                    }
                ]
            }
        }
    }
}

mapping change

...
"properties": {
               "@type": {
                  "type": "string"
               },
               "oracle_props": {
                   "type" : "nested",
                  "properties": {
                     "@name": {
                        "type": "string"
                     },
                     "@value": {
                        "type": "long"
                     }
                  }
               }
            }
...

Answer 1

Let's get into it:

1. You seem to map your nested path wrong, oracle_props is a child item of db in your example document, but not in your mapping, where it appears directly as child of your root.
2. You are mapping oracle_props.@value as long, but assign a text Y to it at the CREATE_PERMISSION nested doc
3. You query for range lt 4000, which excludes 4000, lte would fit for you

I didn't get your requirement for the missing value, hence I skipped that.

To get you to the right path, I has to simplify it a bit (since I couldn't follow all the mess in your question, sorry)

I'm not going into percolation either, and renamed everything to twitter/tweet, since this was easier for me to copy from my examples.

1) Create empty index "twitter"

curl -XDELETE 'http://localhost:9200/twitter/'
curl -XPUT 'http://localhost:9200/twitter/'

2) create geo_point mapping for the actual "tweet"

curl -XPUT 'http://localhost:9200/twitter/tweet/_mapping' -d '
{
    "tweet": {
        "properties": {
            "db": {
                "type": "object",
                "properties": {
                    "@type": {
                        "type": "string"
                    },
                    "oracle_props": {
                        "type": "nested",
                        "properties": {
                            "@name": {
                                "type": "string"
                            },
                            "@value": {
                                "type": "string"
                            }
                        }
                    }
                }
            }
        }
    }
}'

3) Let's check if the mapping was set

curl -XGET 'http://localhost:9200/twitter/tweet/_mapping?pretty=true'

4) Post some tweets, with nested data

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '{
    "name": "Athena",
    "version": 1,
    "db": {
        "@type": "Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 64bit",
        "oracle_props": [
            {
                "@name": "open_cursors",
                "@value": 4000
            },
            {
                "@name": "USER_ROLE_PRIVS_COUNT",
                "@value": 1
            },
            {
                "@name": "CREATE_PERMISSION",
                "@value": "Y"
            }
        ]
    }
}'

5) Query nested only

curl -XGET localhost:9200/twitter/tweet/_search -d '{
    "query": {
        "nested" : {
            "path" : "db.oracle_props",
            "score_mode" : "avg",
            "query" : {
                "bool" : {
                    "must" : [
                        {
                            "term": {
                                "db.oracle_props.@name": "open_cursors"
                            }
                        },
                        {
                            "range": {
                                "db.oracle_props.@value": {
                                    "lte": 4000
                                }
                            }
                        }
                    ]
                }
            }
        }
    }
}';

6) Query "Athena" and "Oracle"

curl -XGET localhost:9200/twitter/tweet/_search -d '{
    "query" : {
        "bool" : {
            "must" : [
                {
                    "match" : {"tweet.name" : "Athena"}
                },
                {
                    "match" : {"tweet.db.@type" : "Oracle"}
                }
            ]
        }
    }
}'

7) Combine the former two queries

curl -XGET localhost:9200/twitter/tweet/_search -d '{
    "query" : {
        "bool" : {
            "must" : [
                {
                    "match" : {"tweet.name" : "Athena"}
                },
                {
                    "match" : {"tweet.db.@type" : "Oracle"}
                },
                {
                    "nested" : {
                        "path" : "db.oracle_props",
                        "score_mode" : "avg",
                        "query" : {
                            "bool" : {
                                "must" : [
                                    {
                                        "term": {
                                            "db.oracle_props.@name": "open_cursors"
                                        }
                                    },
                                    {
                                        "range": {
                                            "db.oracle_props.@value": {
                                                "lte": 4000
                                            }
                                        }
                                    }
                                ]
                            }
                        }
                    }
                }
            ]
        }
    }
}'

Results as

{
    "took": 2,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 2.462332,
        "hits": [
            {
                "_index": "twitter",
                "_type": "tweet",
                "_id": "1",
                "_score": 2.462332,
                "_source": {
                    "name": "Athena",
                    "version": 1,
                    "db": {
                        "@type": "Oracle Database 10g Enterprise Edition Release 10.2.0.4.0 64bit",
                        "oracle_props": [
                            {
                                "@name": "open_cursors",
                                "@value": 4000
                            },
                            {
                                "@name": "USER_ROLE_PRIVS_COUNT",
                                "@value": 1
                            },
                            {
                                "@name": "CREATE_PERMISSION",
                                "@value": "Y"
                            }
                        ]
                    }
                }
            }
        ]
    }
}