Filter json and extract specific value with python

Question

I am trying to extract Size of multiple artifacts by hitting an api. Below json data is of a single artifact and has size repeated at.

{
    "name" : "Payload",
    "type" : "table",
    "value" : {
      "Content-Type" : "",
      "Size" : -1
    }

and

{
    "name" : "Payload",
    "type" : "table",
    "value" : {
      "Content-Type" : "application/zip",
      "Size" : 3369
    }

In this example data: 3369 is the right value which I need, but I couldn't extract it despite trying multiple filters.

Complete json output of single artifact

  "parameters" : {
    "path" : "/mysql.odbc/5.1.14",
    "nexusUrl" : "http://fqdn"
  },
  "items" : [ {
    "name" : "Request",
    "type" : "topic",
    "value" : "Request"
  }, {
    "name" : "Details",
    "type" : "table",
    "value" : {
      "Action" : "GET",
      "path" : "/mysql.odbc/5.1.14"
    }
  }, {
    "name" : "Parameters",
    "type" : "table",
    "value" : {
      "describe" : "json"
    }
  }, {
    "name" : "Headers",
    "type" : "table",
    "value" : {
      "Accept" : "application/json",
      "User-Agent" : "curl/7.78.0",
      "Host" : "fqdn"
    }
  }, {
    "name" : "Attributes",
    "type" : "table",
    "value" : {
      "org.apache.shiro.subject.support.DefaultSubjectContext.SESSION_CREATION_ENABLED" : false,
      "Key[type=org.sonatype.nexus.security.SecurityFilter, annotation=[none]].FILTERED" : true,
      "authcAntiCsrf.FILTERED" : true,
      "nx-apikey-authc.FILTERED" : true
    }
  }, {
    "name" : "Payload",
    "type" : "table",
    "value" : {
      "Content-Type" : "",
      "Size" : -1
    }
  }, {
    "name" : "Response",
    "type" : "topic",
    "value" : "Response"
  }, {
    "name" : "Status",
    "type" : "table",
    "value" : {
      "Code" : 200,
      "Message" : ""
    }
  }, {
    "name" : "Headers",
    "type" : "table",
    "value" : {
      "ETag" : "\"df4f013db18103f1b9541cdcd6ba8632\"",
      "Content-Disposition" : "attachment; filename=mysql.odbc.5.1.14.nupkg",
      "Last-Modified" : "Tue, 13 Oct 2015 03:54:48 GMT"
    }
  }, {
    "name" : "Attributes",
    "type" : "table",
    "value" : { }
  }, {
    "name" : "Payload",
    "type" : "table",
    "value" : {
      "Content-Type" : "application/zip",
      "Size" : 3369
    }
  } ]
}

I am new to python, below is my incomplete code and its output for reference

import requests
import json
import re

repo_name = "repo"

file_list = ["/mysql.odbc/5.1.11","/mysql.odbc/5.1.14"]
for i in file_list:
   url = "http://fqdn/repository/{0}/{1}?describe=json".format(repo_name, i)
   response = requests.get(url)
   json_data = response.text
   data = json.loads(json_data)
   for size in data['items']:
       if size['name'] == 'Payload':
           print(size['value'])

{'Content-Type': '', 'Size': -1}
{'Content-Type': 'application/zip', 'Size': 3109}
{'Content-Type': '', 'Size': -1}
{'Content-Type': 'application/zip', 'Size': 3369}

Any help is appreciated.

You have added the python tag, but you have added no code to your question. — quamrana
– quamrana, Commented Apr 14, 2022 at 20:56
Response for every artifact has a dummy content with size "-1" and the other one has right size. I have validated the sizes of the artifacts in the repository. — reddy0969
– reddy0969, Commented Apr 14, 2022 at 21:00
I see you have updated your question with some python code. What is the current output of this code? — quamrana
– quamrana, Commented Apr 14, 2022 at 21:05

quamrana · Accepted Answer · 2022-04-14 21:19:34Z

1

So to find the non -1 values, just detect those and only print the others:

for i in file_list:
   url = "http://fqdn/repository/{0}/{1}?describe=json".format(repo_name, i)
   response = requests.get(url)
   json_data = response.text
   data = json.loads(json_data)
   for size in data['items']:
       if size['name'] == 'Payload':
           value_size = size['value']['Size']
           if value_size != -1:
               print(value_size)

Please note that I'm not an expert at requests, but I have seen other code which extracts json information and the code is like this:

response = requests.get(url)
data = response.json()

I don't know if this will work in your case.

answered Apr 14, 2022 at 21:19

quamrana

39.5k13 gold badges57 silver badges77 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

reddy0969 Over a year ago

It does, but it is returning just response code:200

Pierre D · Accepted Answer · 2022-04-14 21:17:18Z

0

Perhaps this will get you on the right track?

valid_values = [
    v for v in [
        e.get('value').get('Size', -1)
        for e in data['items']
        if isinstance(e.get('value'), dict)
    ]
    if v > 0
]

On your data, there is only one valid value:

>>> valid_values
[3369]

answered Apr 14, 2022 at 21:17

Pierre D

26.6k8 gold badges71 silver badges108 bronze badges

Collectives™ on Stack Overflow

Filter json and extract specific value with python

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related