Command line bash how to extract json text from a curl output of a webpage

Question

Currently, I have a curl request that looks like this:

curl -k -X GET -H "Accept: application/json" 'https://somesite.com?id=12345' | recode html..ascii

I pipe this request into recode to encode the html ascii characters. The produced output results looks like this:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4270  100  4270    0     0   6713      0 --:--:-- --:--:-- --:--:--     0




0

-  
-  
:    
-  
-
:
-
-
 <html>
--  <head>
:-    <title>Test Cars</title>
-  </head>
:  <body>
2020-05-11 15:03:34,462 INFO test_123 - Ending import of test.
2020-05-11 15:03:34,462 INFO test_123 - Sending message to the UMB.
2020-05-11 15:03:34,989 INFO test_123 - Sending import message:
Message Headers:
  JMSExpiration: 0
  JMSPriority: 0
  JMSMessageID: null
  JMSTimestamp: 0
  JMSCorrelationID: null
  JMSReplyTo: null
  JMSRedelivered: false
  JMSType: application/json
Message Properties:
  id: 12345
  type: test
Message Content:
{
  "cars" : [ {
    "make" : "honda",
    "model" : "accord",
    "trim" : "ex"
  } ],
  "status" : "passed",
  "log-url" : "https://somesite.com/logurl"
}
2020-05-11 15:03:35,572 INFO test_123 - Message sent.

    </pre>
  </body>
</html>

I am trying to parse out this portion of the output:

{
      "cars" : [ {
        "make" : "honda",
        "model" : "accord",
        "trim" : "ex"
      } ],
      "status" : "passed",
      "log-url" : "https://somesite.com/logurl"
    }

and pipe it into jq to turn it into json.

Any idea on how to parse out that json so I can pipe it into jq?

SiegeX · Accepted Answer · 2020-05-13 22:57:05Z

2

Assuming your output is consistent to what you gave above, use:

sed -n '/^{/,/^}/p'

Proof of Concept

$ sed -n '/^{/,/^}/p' ./tojq
{
  "cars" : [ {
    "make" : "honda",
    "model" : "accord",
    "trim" : "ex"
  } ],
  "status" : "passed",
  "log-url" : "https://somesite.com/logurl"
}

answered May 13, 2020 at 22:57

SiegeX

142k25 gold badges150 silver badges159 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Command line bash how to extract json text from a curl output of a webpage

1 Answer 1

Proof of Concept

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Proof of Concept

Comments

Your Answer

Sign up or log in

Post as a guest

Related