Why can't Azure Search import JSON blobs?

Question

When importing data using the configuration found below, Azure Cognitive Search returns the following error:

Error detecting index schema from data source: ""

Is this configured incorrectly? The files are stored in the container "example1" and in the blob folder "json". When creating the same index with the same data in the past there were no errors, so I am not sure why it is different now.

Import data:

Data Source: Azure Blob Storage
Name: test-example
Data to extract: Content and metadata
Parsing mode: JSON
Connection string:
DefaultEndpointsProtocol=https;AccountName=EXAMPLESTORAGEACCOUNT;AccountKey=EXAMPLEACCOUNTKEY;
Container name: example1
Blob folder: json

.json file structure.

{
  "string1": "vaule1",
  "string2": "vaule2",
  "string3": "vaule3",
  "string4": "vaule4",
  "string5": "vaule5",
  "string6": "vaule6",
  "string7": "vaule7",
  "string8": "vaule8",
  "list1": [
    {
      "nested1": "value1",
      "nested2": "value2",
      "nested3": "value3",
      "nested4": "value4"
    }
  ],
  "FileLocation": null
}

Here is an image of the screen with the error when clicking "Next: Add cognitive skills (Optional)" button:

It looks like something may be wrong with the connection string. It looks suspicious that the data source name in the error message would be empty. — Bertrand Le Roy
– Bertrand Le Roy, Commented Jan 7, 2020 at 19:41
I used the "Choose an existing connection" wizard to choose the storage account and container name. See the blue link in provided image under the "Connection string" field. — crayden
– crayden, Commented Jan 7, 2020 at 19:46
I just went through the exact same steps, using the same JSON you have there, and it went through without a problem, so the JSON is not the issue. The issue is probably something to do with the connectivity between your search service and your storage service. My connection string is "DefaultEndpointsProtocol=https;AccountName=REDACTED;AccountKey=REDACTED==;EndpointSuffix=REDACTED.windows.net". You may also want to check resource group, access control, etc. If there's a setting you'd like to compare with mine, let me know. — Bertrand Le Roy
– Bertrand Le Roy, Commented Jan 7, 2020 at 23:04

Sophiac · Accepted Answer · 2020-01-16 21:05:56Z

To clarify there are two problems:

1) There is a bug in the portal where the actual error message is not showing up for errors, hence we are observing the unhelpful empty string "" as an error message. A fix is on the way and should be rolled out early next week.

2) There is an error when the portal attempts to detect index schema from your data source. It's hard to say what the problem is when the error message is just "". I've tried your sample data and it works fine with importing.

I'll update the post once the fix for displaying the error message is out. In the meantime (again we're flying blind here without the specific error string) here are a few things to check:

1) Make sure your firewall rules allow the portal to read from your blob storage

2) Make sure there are no extra characters inside your JSON files. Check the whitespace charcters are whitespace (you should be able to open the file in VSCode and check).

Update: The portal fix for the missing error messages has been deployed. You should be able to see a more specific error message should an error occur during import.

Thiago Custodio · Accepted Answer · 2020-01-07 18:58:33Z

0

Seems to me that is a problem related to the list1 data type. Make sure you're selecting: "Collection(Edm.String)" for it during the index creation.

more info, please check step 5 of the following link: https://learn.microsoft.com/en-us/azure/search/search-howto-index-json-blobs

answered Jan 7, 2020 at 18:58

Thiago Custodio

18.4k9 gold badges50 silver badges99 bronze badges

8 Comments

crayden Over a year ago

This is before I can even get to the screen to create the index. It is the first screen after choosing "import data", and the error displays when clicking add cognitive services. Basically the next button.

Thiago Custodio Over a year ago

better if you provide screenshots of "new data source" step

crayden Over a year ago

Screenshot has been added to the bottom of the question.

Thiago Custodio Over a year ago

try removing json from the blob folder field

crayden Over a year ago

Did that too. I am wondering if there is something wrong with the JSON? If I change the parsing mode to "text", it will work. But it should use JSON as done before.

|

crayden · Accepted Answer · 2020-01-08 20:14:31Z

0

I have been in contact with Microsoft, and this is a bug in the Azure Portal. The issue is the connection string wizard does not append the Endpoint suffix correctly. They have recommeded to manually pasting the connection string, but this still does not work for me. So this is a suggested answer by Microsoft, but I don't believe is completely correct because the portal outputs the same error message:

Error detecting index schema from data source: ""

answered Jan 8, 2020 at 20:14

crayden

2,3206 gold badges43 silver badges86 bronze badges

2 Comments

Bertrand Le Roy Over a year ago

FWIW, yes, I did copy-paste the querystring when I tried to repro yesterday.

Thiago Custodio Over a year ago

I've faced the same problem and also reported this to them. There should be a fix on the way...

Collectives™ on Stack Overflow

Why can't Azure Search import JSON blobs?

Import data:

.json file structure.

3 Answers 3

Comments

8 Comments

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

Import data:

.json file structure.

3 Answers 3

Comments

8 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related