Python json.dumps using new dictionary not returning valid json format

Question

I'm currently using a python module that helps with the tenable API to export asset data from tenable. The export function returns an "ExportIterator" type object to walk through the results of the export.

Essentially this returns too much data per asset, and I'm having difficulty figuring out how to filter out the data being returned so I can use it.

This returns thousands of json objects with hundreds of keys (I've removed and obfuscated several) like this:

{
    "id": "1a2b3c",
    "has_plugin_results": true,
    "created_at": "xxx",
    "terminated_at": null,
    "terminated_by": null,
    "updated_at": "xxx",
    "deleted_at": null,
    "deleted_by": null,
    "first_seen": "",
    "last_seen": "",
    "first_scan_time": "xxx",
    "last_scan_time": "xxx",
    "last_authenticated_scan_date": "xxx",
    "last_licensed_scan_date": "xxx,
    "last_scan_id": "xxx,
    "last_schedule_id": "xxx",
    "azure_vm_id": null,
    "azure_resource_id": null,
    "gcp_project_id": null,
    "gcp_zone": null,
    "gcp_instance_id": null,
    "aws_ec2_instance_ami_id": null,
    "aws_ec2_instance_id": null,
    "agent_uuid": "xxx",
    "bios_uuid": "xxx",
    "network_id": "xxx",
    "network_name": "Default",
    "aws_owner_id": null,
    "aws_availability_zone": null,
    "aws_region": null,
    "aws_vpc_id": null,
    "aws_ec2_instance_group_name": null,
    "aws_ec2_instance_state_name": null,
    "aws_ec2_instance_type": null,
    "aws_subnet_id": null,
    "aws_ec2_product_code": null,
    "aws_ec2_name": null,
    "mcafee_epo_guid": "{xxx}",
    "mcafee_epo_agent_guid": "{xxx}",
    "servicenow_sysid": null,
    "agent_names": [
        "aaabbbccc123"
    ],
    "installed_software": [],
    "ipv4s": [
        "1.1.1.1",
        "2.2.2.2"
    ],
    "ipv6s": [],
    "fqdns": [
        "aaabbbbccc"
    ],
    "mac_addresses": [
        "aa:bb:cc"
    ],
    "netbios_names": [
        "aaabbbccc123"
    ],
    "operating_systems": [
        "foobar 10"
    ],
    "system_types": [
        "general-purpose"
    ],
    "hostnames": [
        "aaabbbccc123"
    ],
    "sources": [
        {
            "name": "AGENT",
            "first_seen": "xxx",
            "last_seen": "xxx"
        }
    ],
}

This module function for exporting doesn't support any arguments for filtering the json object itself.

To filter, I'm using this to map the "hostnames": value to a new key named "vmName" in a new dictioary:

from tenable.io import TenableIO
import json

tio = TenableIO()

wr = open('tioasset.json','w')

for asset in tio.exports.assets():
    new_data = {'vmName' : asset['hostnames'],},
    wr.write(json.dumps(new_data, indent = 2, separators=(',', ':')))

wr.close()

This drops all the unnecessary keys from the api response , but the formatting seems to be all wrong:

output from code:

][
  {
    "vmName":[
      "aaabbbccc123"
    ]
  }
][
  {
    "vmName":[
      "dddeeefff123"
    ]
  }
][
  {
    "vmName":[
      "ggghhhiii123"
    ]
  }
][
  {
    "vmName":[
      "jjjkkklll123"
    ]
  }
][
  {
    "vmName":[
      "mmmnnooo123"
    ]
  }
][

Any idea how to make the code return appropriately formatted json data dictionaries? something like this:

[
      {
        "vmName":"aaabbbccc123"
      },
      {
        "vmName":"dddeeefff123"
      },
      {
        "vmName":"ggghhhiii123"
      },
      {
        "vmName":"jjjkkklll123"
      }
]

sbabti zied · Accepted Answer · 2020-12-10 18:29:14Z

1

that's because hostnames is an array: if you want just take the first element (just replace this):

new_data = {'vmName' : asset['hostnames'][0]}

or you can do this if you have many hostnames in each array :

for asset in tio.exports.assets():
    for a in asset['hostnames']:
        new_data = {'vmName' : a,},
        wr.write(json.dumps(new_data, indent = 2, separators=(',', ':')))

answered Dec 10, 2020 at 18:29

sbabti zied

8457 silver badges17 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Fancy Schmancy Over a year ago

I tried the first bit, and returned: new_data = {'vmName' : asset['hostnames'][0]}, IndexError: list index out of range and the second code snippet still had the brackets and incorrect format for some reason

sbabti zied · Accepted Answer · 2020-12-10 18:34:20Z

1

from tenable.io import TenableIO
import json

tio = TenableIO()

wr = open('tioasset.json','w')

result = []
for asset in tio.exports.assets():
    for a in asset['hostnames']:
        new_data = {'vmName' : a}
        result.append(new_data)

wr.write(json.dumps(result))

wr.close()

answered Dec 10, 2020 at 18:34

sbabti zied

8457 silver badges17 bronze badges

1 Comment

Fancy Schmancy Over a year ago

I only returned a single hostname after running this, whereas before there were many hundreds.

Collectives™ on Stack Overflow

Python json.dumps using new dictionary not returning valid json format

2 Answers 2

1 Comment

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related