ElasticSearch regex query doesn't work

Question

I am using ES 2.4.6 with Java 8, and i created a document object as following:

@Document(indexName = "airports", type = "airport")
public class Airport {

  @Id
  private String id;

  @Field(type = String)
  private String name;
}

And i successfully search several airport objects to ES, with following names: "San Francisco", "San Mateo", "Santiago", "Palo Alto", "Big San" The JSON content inside ES looks like following:

{
  "took": 2,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "failed": 0
  },
  "hits": {
    "total": 5,
    "max_score": 1,
    "hits": [
      {
        "_index": "airports",
        "_type": "airport",
        "_id": "SSMlsTWIYefbXHCnYEwEY",
        "_score": 1,
        "_source": {
          "id": "SSMlsTWIYefbXHCnYEwEY",
          "name": "Santiago"
        }
      },
      {
        "_index": "airports",
        "_type": "airport",
        "_id": "LlDcKuywPjURNeIISjXLjC",
        "_score": 1,
        "_source": {
          "id": "LlDcKuywPjURNeIISjXLjC",
          "name": "San Mateo"
        }
      },
      {
        "_index": "airports",
        "_type": "airport",
        "_id": "CVIjEHYphSmZIjYbHCMwtkqfKWtEHVh",
        "_score": 1,
        "_source": {
          "id": "CVIjEHYphSmZIjYbHCMwtkqfKWtEHVh",
          "name": "San Francisco"
        }
      },
      {
        "_index": "airports",
        "_type": "airport",
        "_id": "gbntKR",
        "_score": 1,
        "_source": {
          "id": "gbntKR",
          "name": "Palo Alto"
        }
      },
      {
        "_index": "airports",
        "_type": "airport",
        "_id": "bKosUdHeseMMboyaejv",
        "_score": 1,
        "_source": {
          "id": "bKosUdHeseMMboyaejv",
          "name": "Big San"
        }
      }
    ]
  }
}

Then i have following curl command to use regex query to find all airport names staring with "san" ignoring case, i did:

curl -XGET 'localhost:9200/airports/airport/_search?pretty' -H 'Content-Type: application/json' -d'
{
    "query": {
        "regexp":{
            "name": "^(?i)san"
        }
    }
}
'

I use the regex "^(?i)san" directly match against those airport names, it works as expect:

String regex = "^(?i)san";
assertTrue("San Francisco".matches(regex));
assertTrue("San Mateo".matches(regex));
assertTrue("Santiago".matches(regex));
assertTrue(!"Big San".matches(regex));

So does anyone know why ES regex query returns empty result back? Now, if i use "san" as regex, all 4 names return back, and if i use "San", nothing returns back.

If you want to get names starting with san or San, try "name": "[sS]an.*" — Wiktor Stribiżew
– Wiktor Stribiżew, Commented Sep 18, 2017 at 20:30
It does not work. i only want the names starting with "san" ignoring case. And why normal java regex does not work here? And in real production, "san" will be runtime parameter passed in. I will prefer "^(?i)" +name+".*" as regex — Yongqin
– Yongqin, Commented Sep 18, 2017 at 22:47
You cannot expect ES regex that is of Lucene flavor to support java.util.regex. Its patterns are always anchored and you should not use ^. (?i) is not supported in Lucene regex flavor. "name": "[sS][aA][nN].*" is your only option with that regex. — Wiktor Stribiżew
– Wiktor Stribiżew, Commented Sep 18, 2017 at 22:48
"Big San" got returned when i used "[sS]an.*". To get starting with, i have to use anchor "^" according to the ES doc: elastic.co/guide/en/elasticsearch/reference/current/… — Yongqin
– Yongqin, Commented Sep 18, 2017 at 23:06
That means the pattern is not handled by the ES (Lucene) regex engine. Acc. to the docs you provided, Lucene’s patterns are always anchored. The pattern provided must match the entire string. and the ^ is not supposed to be there. See the examples. And that means your question is now unclear. — Wiktor Stribiżew
– Wiktor Stribiżew, Commented Sep 18, 2017 at 23:22

Richa · Accepted Answer · 2017-09-19 03:57:46Z

1

You can use Match Phrase Prefix for the problem mentioned above.

 {
  "query": {
    "match_phrase_prefix": {
       "name": "San"
      }
    }
 }

See if it resolves your problem.

answered Sep 19, 2017 at 3:57

Richa

7,6796 gold badges30 silver badges36 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

Yongqin Over a year ago

I tried this query before. It doesn't work. it returns empty list back.

Richa Over a year ago

I tried and it worked.. What is the mapping of your index? and What ES version are you using?

Yongqin Over a year ago

Please see the top of this question.

Richa Over a year ago

Sorry I missed it. So name is of type String which is analyzed by default. So ideally above query should work. I have the same mapping and its is working

Yongqin Over a year ago

See my command line result:curl -XGET 'localhost:9200/airports/airport/_search?pretty' -H 'Content-Type: application/json' -d' > { > "query": { > "match_phrase_prefix":{ > "name": "san" > } > } > } > ' { "took" : 2, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 0, "max_score" : null, "hits" : [ ] } }

|

Collectives™ on Stack Overflow

ElasticSearch regex query doesn't work

1 Answer 1

8 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Your Answer

Sign up or log in

Post as a guest

Related