Dynamically Generating Variables for Dataframe values Assignment

Question

I am trying to make the below code more dynamic. I have left out all code that is not relevant for simplicity. What follows is the whole concept for the program. The objective is to build linear models from random columns. The way it working is x numbers of random columns are selection and then those columns are used to build a linear model. That model is used on a test dataset and relevant information is captured in a dataframe. This will continues a large number of times. What I would like to do is to be able to generation the code that is used to assign the values to the dataframe dynamical based on the number of columns selected. Otherwise I need to keep the number of columns selected static. With a consequence that I babysit the program while it runs and I manual index the number of columns selected.

The following code is what I would like to generate dynamically: test_df.loc[i,asignment_list[ii]] = i.

The example code below is only calling for 3 random columns. import pandas as pd

test_df = pd.DataFrame(columns = {'a','b','c','d','e','f','g','h','i','j'})


for i in range(10):
    asignment_list = list(test_df.sample(n = 3, replace = True, axis = 1))

    test_df.loc[i,asignment_list[0]] = i
    test_df.loc[i,asignment_list[1]] = i
    test_df.loc[i,asignment_list[2]] = i

print(test_df)

Output:

I did trying the below piece of code but it requires that I call the variable name which can't be done dynamically.

for ii in range(0,3,1):
    globals()[f'test_df.loc[{i},asignment_list[{ii}]'] = i

If python does not have this functionality could I build it into python with C?

Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking. — Community
– Community Bot, Commented Jul 22, 2022 at 5:53

sitting_duck · Accepted Answer · 2022-07-22 02:49:06Z

1

Is this what you are looking for?

tgt_cols = list('abcdef')
dfs = pd.DataFrame(index=range(10))

for c in tgt_cols:
    r = [random.randint(0,9) for _ in range(3)]
    s = pd.Series(r, index=r).drop_duplicates()
    dfs[c] = s
    
print(dfs)

Result

     a    b    c    d    e    f
0  0.0  0.0  0.0  NaN  NaN  NaN
1  NaN  NaN  NaN  NaN  1.0  NaN
2  NaN  NaN  NaN  NaN  NaN  NaN
3  NaN  3.0  NaN  NaN  NaN  NaN
4  NaN  NaN  NaN  NaN  NaN  4.0
5  NaN  NaN  NaN  NaN  NaN  5.0
6  6.0  NaN  6.0  NaN  NaN  NaN
7  NaN  NaN  NaN  7.0  7.0  NaN
8  NaN  8.0  8.0  NaN  8.0  NaN
9  9.0  NaN  NaN  9.0  NaN  NaN

answered Jul 22, 2022 at 2:49

sitting_duck

3,7901 gold badge17 silver badges20 bronze badges

Sign up to request clarification or add additional context in comments.

Comments

Frank · Accepted Answer · 2022-07-24 16:07:02Z

Sorry for any confusion and thanks for the questions and responses. I found a solution. If anyone finds another way I would like to see it because this answer was a stretch for me. It is below with a note on application. The solution uses the zip function that is feed into the dict() function and then into a Pandas Dataframe. After the first iteration it concatenated the previous Pandas Dataframe from the last iteration with the newly constructed Pandas Dataframe. The newly concatenated Pandas Dataframe is assigned to the previous Pandas Dataframe variable name.

Use Application Comment: Given a wide table of data and a desire to build a model from the data set it would be optimal to select a random set of columns for each model that is built. This will prevent over fitting and show which variables have the model consistent effect.

import pandas as pd
import random

test_df = pd.DataFrame(columns = {'a','b','c','f','j'})


for i in range(0,10,1):
    columns_number_selected = len(test_df.columns)
    Randum_Columns = random.randrange(columns_number_selected)
    while Randum_Columns == 0:
        Randum_Columns = random.randrange(columns_number_selected)
        
    assignment_list = list(test_df.sample(n = columns_number_selected, replace = True, axis = 1))
    data = dict(zip(assignment_list, [i]))
    
    if i == 0:
        answer1 = pd.DataFrame(data, index = [i])
    
    else:
        answer2 = pd.DataFrame(data, index = [i])
        answer1 = pd.concat([answer1, answer2])
        
print(answer1)

Example of a possible output:

Collectives™ on Stack Overflow

Dynamically Generating Variables for Dataframe values Assignment

2 Answers 2

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related