Accessing JSON items in python

Question

I'm attempting to iterate through each section and print out the information but I keep getting TypeError: list indices must be integers or slices, not str as my error. Here is my data:

  [
   {
      "_id":"5be584e807b6f93c2cee00c4",
      "updated":1541785331000,
      "series":null,
      "price":11,
      "site":"cigar_auctioneer",
      "d_no":null,
      "wrapper_code":"OSC",
      "shape":"petite corona",
      "link":"https://www.cigarauctioneer.com/tatuaje-tattoo-needles-5-cigars/item-61689?auctionnum=3058524",
      "item_id":61689,
      "path":"tatuaje-tattoo-needles-5-cigars/item-61689?auctionnum=3058524",
      "id":3058524,
      "size":"4 1/2 x 38",
      "sku":"CI-TTA-NEEDMZ-75",
      "closes":"2018-11-12T23:13:00.000Z",
      "is_cigar":true,
      "number":null,
      "name":"Tatuaje Tattoo Needles",
      "package":"pack",
      "vintage":null,
      "wrapper":"oscuro",
      "state":"open",
      "num_units":5,
      "closed":1542064380000,
      "type":"pack",
      "quantity":1
   },
   {
      "_id":"5be4221207b6f92342f0ae23",
      "updated":1541785907000,
      "series":null,
      "price":39,
      "site":"cigar_auctioneer",
      "d_no":null,
      "wrapper_code":"OSC",
      "shape":"torpedo",
      "link":"https://www.cigarauctioneer.com/tatuaje-tattoo-bonita-10-pk-cigars/item-53476?auctionnum=3052556",
      "item_id":53476,
      "path":"tatuaje-tattoo-bonita-10-pk-cigars/item-53476?auctionnum=3052556",
      "id":3052556,
      "size":"6 1/4 x 52",
      "sku":"CI-TTA-BONM10PK-75",
      "closes":"2018-11-12T03:22:00.000Z",
      "is_cigar":true,
      "number":null,
      "name":"Tatuaje Tattoo Bonita",
      "package":"pack",
      "vintage":null,
      "wrapper":"oscuro",
      "state":"open",
      "num_units":5,
      "closed":1541992920000,
      "type":"pack",
      "quantity":10
   },
   {
      "_id":"5bde7e2a07b6f92cc36a75fb",
      "origin":"nicaragua",
      "updated":1541693111000,
      "series":null,
      "price":null,
      "site":"cbid",
      "d_no":null,
      "number":null,
      "shape":"robusto",
      "is_cigar":true,
      "link":"https://www.cigarbid.com/a/tatuaje-tattoo-caballero/3030156/",
      "path":"a/tatuaje-tattoo-caballero/3030156/",
      "id":3030156,
      "unit":"box of 50",
      "size":"5 x 50",
      "sku":"TAT-PM-1008",
      "closes":"2018-11-11T04:58:00.000Z",
      "strength":"medium",
      "name":"Tatuaje Tattoo Caballero",
      "package":"box of 50",
      "vintage":null,
      "wrapper":"ecuador habano",
      "exdata":{
         "skuId":"TAT-PM-1008",
         "list":"PLP>Search",
         "isImpression":true,
         "lot":{
            "category":"Free Fall Cigar Auction",
            "closes":"2018-11-10T23:58:00-05:00",
            "lotId":3030156,
            "name":"Tatuaje Tattoo Caballero (Robusto) (5.0\"x50) Box of 50",
            "engine":"DutchLoop",
            "status":"Open",
            "opens":"2018-11-04T00:13:00-04:00"
         },
         "fullName":"Tatuaje Tattoo Caballero (50)",
         "type":"Handmade Cigars",
         "id":154551,
         "pack":"Box of 50"
      },
      "state":"open",
      "num_units":null,
      "closed":1541912280000,
      "type":"box of 50",
      "quantity":50
   },
   {
      "_id":"5be2d68207b6f90b492ae0fd",
      "updated":1541786158000,
      "series":null,
      "price":195,
      "site":"cigar_auctioneer",
      "d_no":null,
      "wrapper_code":"OSC",
      "shape":"torpedo",
      "link":"https://www.cigarauctioneer.com/tatuaje-tattoo-bonito-cigars/item-45795?auctionnum=3045899",
      "item_id":45795,
      "path":"tatuaje-tattoo-bonito-cigars/item-45795?auctionnum=3045899",
      "id":3045899,
      "size":"6 1/4 x 52",
      "sku":"CI-TTA-BONM-75",
      "closes":"2018-11-11T02:45:00.000Z",
      "is_cigar":true,
      "number":null,
      "name":"Tatuaje Tattoo Bonito",
      "package":"box",
      "vintage":null,
      "wrapper":"oscuro",
      "state":"open",
      "num_units":1,
      "closed":1541904300000,
      "type":"box",
      "quantity":50
   },
   {
      "_id":"5be07b8d07b6f95b9a151a83",
      "origin":"nicaragua",
      "updated":1541787982000,
      "series":null,
      "price":3,
      "site":"cbid",
      "d_no":null,
      "number":null,
      "shape":"robusto",
      "is_cigar":true,
      "link":"https://www.cigarbid.com/a/tatuaje-tattoo-caballero/3029330/",
      "path":"a/tatuaje-tattoo-caballero/3029330/",
      "id":3029330,
      "unit":"single",
      "size":"5 x 50",
      "sku":"TAT-PM-1001",
      "closes":"2018-11-10T04:48:00.000Z",
      "strength":"medium",
      "name":"Tatuaje Tattoo Caballero",
      "package":"single",
      "vintage":null,
      "wrapper":"ecuador habano",
      "exdata":{
         "skuId":"TAT-PM-1001",
         "list":"PLP>Search",
         "isImpression":true,
         "lot":{
            "category":"Cigar Singles",
            "closes":"2018-11-09T23:48:00-05:00",
            "minimumOffer":3,
            "lotId":3029330,
            "name":"Tatuaje Tattoo Caballero (Robusto) (5.0\"x50) Single",
            "engine":"English",
            "status":"Open",
            "units":2,
            "opens":"2018-11-05T12:00:00-05:00"
         },
         "fullName":"Tatuaje Tattoo Caballero (robusto) (Single)",
         "type":"Handmade Cigars",
         "id":161996,
         "pack":"Single"
      },
      "state":"open",
      "num_units":2,
      "closed":1541825280000,
      "type":"single",
      "quantity":1
   },
   {
      "_id":"5be190b607b6f9711c751df4",
      "updated":1541786729000,
      "series":null,
      "price":19,
      "site":"cigar_auctioneer",
      "d_no":null,
      "wrapper_code":"OSC",
      "shape":"torpedo",
      "link":"https://www.cigarauctioneer.com/tatuaje-tattoo-bonita-5-pk-cigars/item-53475?auctionnum=3048388",
      "item_id":53475,
      "path":"tatuaje-tattoo-bonita-5-pk-cigars/item-53475?auctionnum=3048388",
      "id":3048388,
      "size":"6 1/4 x 52",
      "sku":"CI-TTA-BONM5PK-75",
      "closes":"2018-11-10T03:05:00.000Z",
      "is_cigar":true,
      "number":null,
      "name":"Tatuaje Tattoo Bonita",
      "package":"pack",
      "vintage":null,
      "wrapper":"oscuro",
      "state":"open",
      "num_units":5,
      "closed":1541819100000,
      "type":"pack",
      "quantity":5
   },
   {
      "_id":"5be08ae107b6f95e788e0999",
      "origin":"nicaragua",
      "updated":1541786947000,
      "series":null,
      "price":14.5,
      "site":"cbid",
      "d_no":null,
      "number":null,
      "shape":"robusto",
      "is_cigar":true,
      "link":"https://www.cigarbid.com/a/tatuaje-tattoo-caballero/3025943/",
      "path":"a/tatuaje-tattoo-caballero/3025943/",
      "id":3025943,
      "unit":"pack of 5",
      "size":"5 x 50",
      "sku":"TAT-PM-1003",
      "closes":"2018-11-10T00:58:00.000Z",
      "strength":"medium",
      "name":"Tatuaje Tattoo Caballero",
      "package":"pack of 5",
      "vintage":null,
      "wrapper":"ecuador habano",
      "exdata":{
         "skuId":"TAT-PM-1003",
         "list":"PLP>Search",
         "isImpression":true,
         "lot":{
            "category":"5-Packs of Cigars",
            "closes":"2018-11-09T19:58:00-05:00",
            "minimumOffer":14.5,
            "lotId":3025943,
            "name":"Tatuaje Tattoo Caballero (Robusto) (5.0\"x50) Pack of 5",
            "engine":"English",
            "status":"Open",
            "units":9,
            "opens":"2018-11-05T12:35:00-05:00"
         },
         "fullName":"Tatuaje Tattoo Caballero - 5-Pack",
         "type":"Handmade Cigars",
         "id":154550,
         "pack":"Pack of 5"
      },
      "state":"open",
      "num_units":9,
      "closed":1541811480000,
      "type":"pack of 5",
      "quantity":5
   },
   {
      "_id":"5be18ff707b6f9711c751d87",
      "updated":1541786468000,
      "series":null,
      "price":12,
      "site":"cigar_auctioneer",
      "d_no":null,
      "wrapper_code":"OSC",
      "shape":"robusto",
      "link":"https://www.cigarauctioneer.com/tatuaje-tattoo-caballero-5-pack-cigars/item-48230?auctionnum=3051568",
      "item_id":48230,
      "path":"tatuaje-tattoo-caballero-5-pack-cigars/item-48230?auctionnum=3051568",
      "id":3051568,
      "size":"5 x 50",
      "sku":"CI-TTA-CABM5PK-75",
      "closes":"2018-11-10T00:17:00.000Z",
      "is_cigar":true,
      "number":null,
      "name":"Tatuaje Tattoo Caballero",
      "package":"pack",
      "vintage":null,
      "wrapper":"oscuro",
      "state":"open",
      "num_units":1,
      "closed":1541809020000,
      "type":"pack",
      "quantity":5
   }
]

And here is my code attempting to access it

import os
import time
import re
import json
import requests, urllib.parse
from bs4 import BeautifulSoup, element

if __name__ == "__main__":
    with requests.Session() as se:
        searchTerm = "tatuaje+tattoo"
        se.headers = {
            "X-Requested-With": "XMLHttpRequest",
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.92 Safari/537.36",
            "Referer": "https://herf.io/bids?search="+searchTerm,
            "Content-Type": "application/x-www-form-urlencoded; charset=UTF-8",
            "Accept-Encoding":"gzip, deflate, br",
        }
        data = [
            "search="+searchTerm,
            "types=",
            "sites=",
        ]

        cookies = {
            "Cookie": "connect.sid=s%3ANYNh5s6LzCVWY8yE9Gra8lxj9OGHPAK_.vGiBmTXvfF4iDScBF94YOXFDmC80PQxY%2FX9FLQ23hYI"}

        url = "https://herf.io/bids/search/open"

        price = "https://herf.io/bids/search/stats"

        openAuctions = "https://herf.io/bids/search/open"

        closedAuctions = "https://herf.io/bids/search/closed"

        #req = se.post(price,data="&".join(data),cookies=cookies,verify=False)
        reqAuctions = se.post(openAuctions,data="&".join(data),cookies=cookies,verify=False)
        #str = req.json()
        strAuctions = reqAuctions.json()

        print(strAuctions['_id'])
        print(strAuctions['link'])

I'm assuming the issue is the fact that the top level doesn't have a dict key? Looking around other people are talking about using for loops iterating through the top level dicts but I'm not sure how to access without it?

Thanks.

wilkben · Accepted Answer · 2018-11-09 21:03:13Z

2

Your guess is right. The top level is an array, not a dictionary. If you just want the first entry you can do

print(strAuctions[0]['_id'])
print(strAuctions[0]['link'])

To output information for each entry use

for auction in strAuctions:
    print(auction['_id'])
    print(auction['link'])

answered Nov 9, 2018 at 21:03

wilkben

6673 silver badges13 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

ramseys1990 Over a year ago

Thanks! this is as simple of an approach as I was hoping for! Works perfectly!

omair issa · Accepted Answer · 2018-11-09 20:59:48Z

0

use that code print(strAuctions.keys()) with this code you are see all the keys if you want to use it you can try for loop

answered Nov 9, 2018 at 20:59

omair issa

14 bronze badges

2 Comments

ramseys1990 Over a year ago

it throws AttributeError: 'list' object has no attribute 'keys' error

omair issa Over a year ago

strAuctions[0].keys()

Harikrishna · Accepted Answer · 2018-11-10 00:02:35Z

Since the outermost element in the object you have is a list, you should access it using indexes and then use key to access the dictionary elements.

Example:

list1=[
   {
      "_id":"5be584e807b6f93c2cee00c4",
      "updated":1541785331000,
      "series":11,
      "price":11,
      "site":"cigar_auctioneer",
      "d_no":11,
      "wrapper_code":"OSC",
      "shape":"petite corona",
      "link":"https://www.cigarauctioneer.com/tatuaje-tattoo-needles-5-cigars/item-61689?auctionnum=3058524",
      "item_id":61689,
      "path":"tatuaje-tattoo-needles-5-cigars/item-61689?auctionnum=3058524",
      "id":3058524,
      "size":"4 1/2 x 38",
      "sku":"CI-TTA-NEEDMZ-75",
      "closes":"2018-11-12T23:13:00.000Z",
      "is_cigar":True,
      "number":[1,2,3],
      "name":"Tatuaje Tattoo Needles",
      "package":"pack",
      "vintage":11,
      "wrapper":"oscuro",
      "state":"open",
      "num_units":5,
      "closed":1542064380000,
      "type":"pack",
      "quantity":1
   }]

To access the quantity value of 1, you can do this:

list1[0]['quantity']

If your values in the dictionary contains list you need to further access using indexes.

To access elements in the list under the key number, you should do this

list1[0]['number'][0]

where 0 refers to the index

Note: modified original data to give the explanation

Collectives™ on Stack Overflow

Accessing JSON items in python

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

1 Comment

2 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related