Insert dict into dataframe with loop

Question

Fetching data from API with for loop, but only last row is showing. If i put print statement instead of d=, I get all records for some reason. How to populate a Dataframe with all values?

I tried with for loop and with append but keep getting wrong results

for x in elements:
    url = "https://my/api/v2/item/" + str(x[number"]) + "/"
    get_data = requests.get(url)
    get_data_json = get_data.json()
    d = {'id': [x["enumber"]], 
         'name': [x["value1"]["name"]],
         'adress': [value2["adress"]],
         'stats': [get_data_json["stats"][5]["rating"]]
        }
df = pd.DataFrame(data=d)
df.head()

Result:

id name  order  adress  rating

Only last row is showing, probably because it's overwriting until it comes to last element. Should I put another for loop somewhere or there is some obvious solution that I cannot see?

You need to look into df.append just create the dataframe once before adding all the data then append the new data — Jab
– Jab, Commented Feb 16, 2019 at 0:26
@Jaba generally that is not the way to go. Append the results into a list and then use that list to build the data-frame. .append with a data-frame is needlessly inefficient — juanpa.arrivillaga
– juanpa.arrivillaga, Commented Feb 16, 2019 at 0:27

Keatinge · Accepted Answer · 2019-02-16 00:26:55Z

3

Put all your data into a list of dictionaries, then convert to a dataframe at the very end

At the top of your code write:

all_data = []

Then in your loop, after d = {...}, write

all_data.append(d)

Finally at the end (after the loop has finished)

df = pd.DataFrame(all_data)

answered Feb 16, 2019 at 0:26

Keatinge

4,3616 gold badges28 silver badges44 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Mr.Mister. Over a year ago

thanks, it's populated now, just all values are in [], eg [3] [2] [100] , how to remove that or it doesn't matter?

Mr.Mister. Over a year ago

is it because I used these parentheses in [x["entry_number"]] etc?

Keatinge Over a year ago

@Mr.Mister. It does matter, you should use x["entry_number"] instead of [x["entry_number"]], there's no need for the outside brackets []

Collectives™ on Stack Overflow

Insert dict into dataframe with loop

1 Answer 1

3 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Your Answer

Sign up or log in

Post as a guest

Related