0

I'm trying to get more information on an error in jq.

I have two files, let's call them file1.json and file2.json, both of which contain an array of objects that I want to join by a common property, .id. I'm using a combination of the INDEX and JOIN built-in functions. (Shout-out to this incredibly helpful blog post for helping me understand how INDEX and JOIN work.)

I'm running the following command:

jq '[JOIN(INDEX(input[], .id); .[]; (.id|tostring); add)]' file1.json file2.json > joined.json

which gives me the following error:

jq: error (at file2.json:11329): break

Line 11329 is the last line of file2.json, which contains the closing ] for the JSON array. Here is the end of file2.json

 1      [
 ...      ...,
 11304    {
 11305      "count_by_file_type": {
 11306        "gitignore": 1,
 11307        "json": 1
 11308      },
 11309      "id": 972
 11310    },
 11311    {
 11312      "count_by_file_type": {
 11313        "html": 3
 11314      },
 11315      "id": 975
 11316    },
 11317    {
 11318      "count_by_file_type": {
 11319        "html": 3
 11320      },
 11321      "id": 994
 11322    },
 11323    {
 11324      "count_by_file_type": {
 11325        "html": 2
 11326      },
 11327      "id": 999
 11328    }
 11329  ]

I have no idea what's causing the break to happen (I'm assuming it's the JOIN function but really I don't know that for sure). I know there's a debug function but I don't believe it's possible to use it to "step into" a function. Because it's erroring on the last line of the file I thought maybe it found something unexpected instead of EOF, or that somehow it was invalid JSON. I checked with a validator and file2.json is valid JSON.

If I could drill down into the JOIN function and debug the intermediate inputs, maybe jq would give me a more helpful error message. I've tried try-catch but it's not producing the output I'd want (if I even knew what I wanted it to print on failure). I feel like I don't even have enough information to know what to try next.

Does anybody have ideas for how I could debug this further? Thanks!!

Improve this question
4
  • 1
    It would help others to understand your problem if you could provide minimal example of file[12].json and expected output. Commented Jul 8 at 19:11
  • Try inputs[] instead Commented Jul 8 at 19:12
  • inputs[] allows jq to exit successfully instead of erroring, but it does not perform the JOIN, as the objects from file2.json (excerpted in my post), so unfortunately it doesn't work. Commented Jul 9 at 20:03
  • Any chance that there is a control or other non-printable characters at the end of the file? That "error" doesn't look familiar to me, can't think of a reason that either JOIN or INDEX would raise that specifically, unless the function definitions changed since I last saw it. They're using basic streaming operations so it might just be that one of the control functions triggered a break out of the stream (reduce technically), probably due to an error in the data. But I would have expected a syntax error for those cases. Commented Oct 13 at 22:37
Related questions
1534
Is there a built-in function to print all the current properties and values of an object?
1291
Reference - What does this error mean in PHP?
309
I keep getting "Uncaught SyntaxError: Unexpected token o"
Load 7 more related questions Show fewer related questions

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.