4

I'm currently using Laravel v7.2, have the babenkoivan/scout-elasticsearch-driver installed (4.2) and am using AWS Elasticsearch 7.1. I have several tables mapped in my application that are working fine but am having issues with a nested mapping that was previously working and is now broken.

I'm saving data into a table and having that table data copied into AWS Elasticsearch. I'm using MySQL 5.6 so I am using a TEXT column to store JSON data. Data in the table looks as follows:

'id' => 1,
'results' => [{"finish":1,"other_id":1,"other_data":1}]

I have my model setup with the following mapping:

protected $mapping = [
        'properties' => [
            'results' => [
                'type' => 'nested',
                'properties' => [
                    'finish' => [
                        'type' => 'integer'
                    ],
                    'other_id' => [
                        'type' => 'integer'
                    ],
                    'other_data' => [
                        'type' => 'integer'
                    ]
                ]
            ],
        ]
    ];

And if it's of any use, the toSearchableArray:

public function toSearchableArray()
    {
        $array = [
            'id' => $this->id,
            'results' => $this->results
        ];

        return $array;
    }

I have no problem creating this index and it worked up until about a couple of months ago. I don't know exactly when, as it wasn't a high priority item and may have occurred around an AWS ES update but not sure why this in particular would break. I receive the following error now:

{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"object mapping for [results] tried to parse field [results] as object, but found a concrete value"}],"type":"mapper_parsing_exception","reason":"object mapping for [results] tried to parse field [results] as object, but found a concrete value"},"status":400}

I've tried also storing the data in the table as such, thinking it was breaking due to the potential array, but it was to no avail:

'id' => 1,
'results' => {"finish":1,"other_id":1,"other_data":1}

I'm at a loss for what else to try to get this working again.

EDIT: Here is the entire model:

<?php
namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use ScoutElastic\Searchable;


class ResultsModel extends Model
{

    use Searchable;

    protected $indexConfigurator = \App\MyIndexConfiguratorResults::class;

    protected $searchRules = [
        //
    ];

    protected $mapping = [
        'properties' => [
            'results' => [
                'type' => 'nested',
                'properties' => [
                    'finish' => [
                        'type' => 'integer'
                    ],
                    'other_id' => [
                        'type' => 'integer'
                    ],
                    'other_data' => [
                        'type' => 'integer'
                    ]
                ]
            ],
        ]
    ];

    public function searchableAs()
    {
        return 'results_index';
    }

    public function toSearchableArray()
    {
        $array = [
            'id' => $this->id,
            'results' => $this->results
        ];

        return $array;
    }
    /**
     * The database table used by the model.
     *
     * @var string
     */
    protected $table = 'results_table';

}

Here is the \App\MyIndexConfiguratorResults::class

<?php

namespace App;

use ScoutElastic\IndexConfigurator;
use ScoutElastic\Migratable;

class MyIndexConfiguratorResults extends IndexConfigurator
{
    use Migratable;

    protected $name = "results_index";
    /**
     * @var array
     */
    protected $settings = [
        //
    ];
}

This is all that is needed to have Laravel automatically update AWS ES each time the table is updated. For the initial load, I would SSH in and run the following command to have it create the index. This, as well as elastic:migrate and any update/insert into the table produces the mapping error.

php artisan elastic:create-index results_index

4
  • Can you share the snippet where you put $mapping as an attribute? How do you pass it? Commented Oct 16, 2020 at 8:04
  • @Joe I've edited the answer to add more details. Hope this helps! Commented Oct 16, 2020 at 15:48
  • It sort of did but I need to see where you pass the $mapping variable. Commented Oct 16, 2020 at 16:18
  • I don't actually pass it anywhere. I believe it's Scout that handles that behind the scenes. I believe RisingSun is correct with the unsupported mapping types but I am currently testing to confirm and determine solution moving forward. Commented Oct 16, 2020 at 17:18

2 Answers 2

1

Finally figured this out so will share the solution for anyone that runs into this. Turns out to be a fairly simple fix, though I'm not sure how it even worked in the first place so that part is still baffling.

I created a brand new index and update the mappings accordingly to add "id" and remove the type "nested" from the "results" piece. (Adding the "nested" type was adding two "results" to the index - one that contained all my nested data, the other just being "object".)

protected $mapping = [
        'properties' => [
            'id' => [
                'type' => 'integer'
            ],
            'results' => [
                'properties' => [
                    'finish' => [
                        'type' => 'integer'
                    ],
                    'other_id' => [
                        'type' => 'integer'
                    ],
                    'other_data' => [
                        'type' => 'integer'
                    ]
                ]
            ],
        ]
    ];

Then I simply added json_decode to the toSearchableArray() function as so:

public function toSearchableArray()
    {
        $array = [
            'id' => $this->id,
            'results' => json_decode($this->results, true)
        ];

        return $array;
    }

Voila. It successfully created the index and imported the data in a manner with which I can query the nested object.

Sign up to request clarification or add additional context in comments.

Comments

0

Reading through the docs, the type field seems to have been removed. Scroll down to 7.x to see. Also, it seems you need to delete the index and re-add it in order for the new map to work according to this page.

1 Comment

This makes sense but when I checked through my version control for composer.json and my ES on AWS, I actually just noticed that the versions have not changed since the initial implementation. So, this was working and has seemingly stopped without a version update to ES. I have tried dropping "type" from my mapping but still running into the same issue. I did also try dropping all the nested properties for "results" and it loaded but just stored everything as flat json inside the "results" field instead of breaking it out.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.