Elasticsearch: Null-value fields in custom_score queries causes 'No field' errors

Created on 9 Feb 2011 · 5Comments · Source: elastic/elasticsearch

There seems to be a gotcha when using custom_score queries and trying to score based on fields that can contain null values.

Given I have:

{"user": {"name": "John", "position": null}}

When I query using a custom_score and a script like:

{"script": "_score + doc['position'].value"}

I get the following error:

ElasticSearchIllegalArgumentException[No field found for [position]]; }]

If I add another user:

{"user": {"name": "Jane", "position": 1}}

Everything works fine.

I think the default behavior should be for null-value fields to return null - and not raise an error.

The examples above might seem lame, but imagine filling the indices with data from a database, where no guarantees can be made that a column won't contain anything other than null-values. You can get around this by typecasting your fields to strings by using mapping, but I definitely suggest that ElasticSearch is able to handle the script value more intelligently.

Source

pfeiffer

Most helpful comment

This should work:

{"script": "_score + (doc.containsKey('position') ? doc['position'].value : 0)"}

Or maybe even this:

{"script": "_score + (doc['position'] ? doc['position'].value : 0)"}

Even so, with enough data set, where at least one doc has a value for position on each shard, you should not need this check, and then the performance of your script will be much better.

kimchy on 11 Feb 2011

👍3

All 5 comments

Oh - and thanks for an awesome product! :-)

pfeiffer on 9 Feb 2011

Heya,

This happens because that field has not been introduced yet (it has a null value). Once it is introduced (with a single value, into the cluster), then it will work fine. You can also define null-value in the mappings that will index a specific null value when that field has is null. Since its not introduced, its type can't be derived.

kimchy on 9 Feb 2011

Yes, I understand that. Wouldn't it be better for the script value to return null or at least to have some kind of function to check for it's existence, instead of raising an exception and causing no results to be returned?

Eg:

{"script": "_score + (doc['position'].present ? doc['position'] : 0)"}

pfeiffer on 10 Feb 2011

This should work:

{"script": "_score + (doc.containsKey('position') ? doc['position'].value : 0)"}

Or maybe even this:

{"script": "_score + (doc['position'] ? doc['position'].value : 0)"}

Even so, with enough data set, where at least one doc has a value for position on each shard, you should not need this check, and then the performance of your script will be much better.

kimchy on 11 Feb 2011

👍3

Fatal error: Uncaught [NoFieldFound]No field found: Contact._PloiciesLastApplicationDate0 Attempted: 1 time(s). thrown

I am getting the following error while updating the contact.

for example:-

$checkExist = Infusionsoft_DataService::query(new Infusionsoft_Contact(), array('Phone1' => '(618)-795-0304'));

I am fetching the contact and updating the contact based on the phone number, But it give this error?

Thanks!