1

I'm trying to create a simple backtester on python which allows me to assess the performance of a trading strategy. As part of the backtester, I need to record the transactions which occur. E.g., 

Day     Stock    Action     Quantity
1       AAPL     BUY        20
2       CSCO     SELL       30
2       AMZN     SELL       50

During the trading simulation, I'll need to add more transactions.

What's the most efficient way to do this. Should I create a transactions list at the start of the simulation and append lists such as [5, 'AAPL', 'BUY', 20] as I go. Should I instead use a dictionary, or a numpy array? Or just a Pandas DataFrame directly?

Thanks,

Jack

Improve this question
12
  • 1
    some sort of ? The question is what sort of ? Hows does the data from that sort of simulation look like? Commented Jan 17, 2018 at 12:14
  • 1
    More probably for a question like this answer usually would be it depends. Commented Jan 17, 2018 at 12:14
  • Just generic rows of data. One row might contain the values ['a', 'b', 'c', 'd'], for instance. Commented Jan 17, 2018 at 12:15
  • you cant ask for opinions here. As a data science question we like to see the data and type of operation you want to perform on the data. This is too broad. Commented Jan 17, 2018 at 12:16
  • 1
    Use a list. Append operations are very costly in numpy and pandas. Convert to a DataFrame once you are done. Commented Jan 17, 2018 at 12:22

1 Answer 1

Reset to default
2

list.append operations are amortised constant time operations, because it just involves shifting pointers around.

OTOH, numpy.ndarray and pd.DataFrame objects are internally represented as arrays in C, and those are immutable. Each time you "append" to an array/dataframe, you have to reallocate new memory for an entire copy of the old data plus the appended, and ends up being linear in complexity.

So, as @ayhan said in a comment, accumulate your data in a list, and then load into a dataframe once you're done.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thanks for your help!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.