10

I am creating the list as follows:

myList = []
for i in range(0,10):
    val0 = 1  # some formula for obtaining values
    val1 = 2.5
    val2 = 1.8
    myList.append([val0,val1,val2])

How can I do the same loop for pandas DataFrame (i.e. myList must be a DataFrame).

3
  • concat if you really need to, but it's better to do all the concatenation in a list as pandas has O(n) insertion). Commented Feb 24, 2016 at 20:57
  • @Alex: Could you please give an example relevant to my particular case? Commented Feb 24, 2016 at 21:02
  • Are the other answers clear? Commented Feb 24, 2016 at 21:30

2 Answers 2

18

Ideally you want to create your DataFrame once you have all the data in place. Slightly modifying your example:

my_df = []
for i in range(0,10):
    d = {
        'val0' : 1,  # some formula for obtaining values
        'val1' : 2.5,
        'val2' : 1.8
    }
    my_df.append(d)

my_df = pd.DataFrame(my_df)

So now my_df is a DataFrame with val0, val1 and val2 as columns

Sign up to request clarification or add additional context in comments.

5 Comments

Shouldn't there be commas at the end of each of the dictionary elements?
@HeathRaftery there's no reason to enforce that, I took this data from the OP's question. Don't think I'm qualified to add my own comments to his/her data
The OP had them as statements in a loop. You have them as items in a dictionary. AFAIK, items have to be separated by commas when creating a dictionary using that syntax. I'm not widely experienced enough to know if there are exceptions, but all the versions of Python I have at hand report a syntax error when running your code. I had to change it to d = { 'val0' : 1, 'val1' : 2.5, 'val2' : 1.8 }.
I apologize, I totally misread your initial comment (I read "commas" as "comments"). You are correct about the syntax. Will adjust the answer
This is very memory inefficient, and infeasible when the data is more than half your available memory. My only thought is to store in an easy file format (eg: csv) and read that directly as a DataFrame..
5

if i got your question right:

import pandas as pd

myList = []

for i in range(0,10):
    val0 = 1  # some formula for obtaining values
    val1 = 2.5
    val2 = 1.8
    myList.append([val0,val1,val2])

df = pd.DataFrame(myList, columns=['val0','val1','val2'])
print(df)

PS you don't want to do append data to the DataFrame in the loop - it won't be very efficient.

Output:

   val0  val1  val2
0     1   2.5   1.8
1     1   2.5   1.8
2     1   2.5   1.8
3     1   2.5   1.8
4     1   2.5   1.8
5     1   2.5   1.8
6     1   2.5   1.8
7     1   2.5   1.8
8     1   2.5   1.8
9     1   2.5   1.8

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.