Pandas - iterate over dataframe rows and update df (one line of code)

Question

having the following code below. Scenario description:

Iterate over the dataframe having the list of URLs Send GET request. Create initially 3 new columns and update them based on the results from the previous get request.

Question: Is there any option how to do write the 3 "df.set_value" in one line of code??

Many thanks in advance

import pandas as pd, numpy as np

d = {'ListOfURLs': ['URL1', 'URL2', 'URL3']}
df = pd.DataFrame(data=d)

#print(df)
s = requests.session()
s.post(login_url, login_data)
for index, row in df.iterrows():
    r = s.get(row['ListOfURLs'])
    r.status_code
    if r.status_code == 200:
        # Update Dataframe , create initially 3 new columns and update them based on the results from the previous get request
        df.set_value(index, 'Status Code', r.status_code)
        df.set_value(index, 'Result', '[OK]')
        df.set_value(index, 'Error', np.nan)

Why don't extract the data first and then build the DataFrame? — Dani Mesejo
– Dani Mesejo, Commented Nov 29, 2020 at 13:57
Data would be extracted from cvs file, the main goal is to update the df based on the results coming from the GET requests. I’m trying to figure out how to update the df ideally with one line of code. — Tomas Nemeth
– Tomas Nemeth, Commented Nov 29, 2020 at 14:40
You can omit this part, actually.. I’m interested the most how to write the last 3 lines into 1 line. — Tomas Nemeth
– Tomas Nemeth, Commented Nov 29, 2020 at 14:48
@hd1 This is not a request for review, but a request for specific rewrite. Wouldn't do well there. — Mast
– Mast, Commented Nov 29, 2020 at 15:26

Dani Mesejo · Accepted Answer · 2020-11-29 14:52:56Z

You could do:

import pandas as pd
import requests
import numpy as np

d = {'ListOfURLs': ['https://stackoverflow.com/q/65060875/4001592',
                    'https://stackoverflow.com/q/65060875/4001592',
                    'https://stackoverflow.com/q/65060875/4001592']}
df = pd.DataFrame(data=d)

for index, row in df.iterrows():
    r = requests.get(row['ListOfURLs'])
    if r.status_code == 200:
        df.at[index, ['Status Code', 'Result', 'Error']] = (r.status_code, '[OK]', np.nan)

print(df)

Output

                                     ListOfURLs  Status Code Result  Error
0  https://stackoverflow.com/q/65060875/4001592        200.0   [OK]    NaN
1  https://stackoverflow.com/q/65060875/4001592        200.0   [OK]    NaN
2  https://stackoverflow.com/q/65060875/4001592        200.0   [OK]    NaN

Don't use set_value:

Deprecated since version 0.21.0: Use .at[] or .iat[] accessors instead.

Notice that some details from your original question were omitted to produce an actual output.

Collectives™ on Stack Overflow

Pandas - iterate over dataframe rows and update df (one line of code)

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related