Numpy matrix doesn't update value

Question

So I have a dataset with (business) daily data from 2008 to 2011 for 5 variables stored in a pandas DataFrame called storeddata. I want to compute a exponentially-weighted moving average covariance matrix for each day. I have an initiation period (let's say from 2008-01-01 to 2009-01-01) for which I compute the covariance matrix, a 5x5 matrix, called covmat1.

Now I want to iterate through the storeddata-DataFrame and after 2009-01-01 update a dict called covfinaldict with a 5x5 matrix for each business day. I have tried with this block of code:

from pandas.tseries.offsets import BDay
import pandas as pd
import numpy as np
import datetime as dt

zeromatrix=np.mat(np.zeros((5,5)))

for row in storeddata.iterrows():
    if row[0]>dt.datetime(2009,01,01):
        mat1=covfinaldict[row[0]-BDay(1)]  
        for x in range(5):
            for y in range(5):
                if x==y:
                    zeromatrix[x,y]=sqrt(0.93*(mat1[x,y]**2)+0.07*(row[1][x+1]**2))
                else:
                    zeromatrix[x,y]=0.93*mat1[x,y]+0.07*row[1][x+1]*row[1][y+1]
        covfinaldict[row[0]]=zeromatrix  
    else:
        covfinaldict[row[0]]=covmat1

The formulas are an approximation of an exponentially-weighted moving average, the details of which are irrelevant. The computation does not work in the sense that I get the same 5x5 matrix (although it is different from covmat1) for each day after 2009-01-01. Furthermore, if I run sqrt(0.93*(mat1[x,y]**2)+0.07*(row[1][x+1]**2)) in console I get 0.1613, if I run zeromatrix[x,y] I get 0.158 (x=4, y=4). This makes no sense to me. How can the left hand side of an equation not be equal to the right hand side? Explanations are much appreciated.

Raul Andres · Accepted Answer · 2014-09-04 09:31:55Z

1

You should copy your zeromatrix object:

import copy

and

    covfinaldict[row[0]]=copy.copy(zeromatrix)

Otherwise, all covfinaldict rows will point to the same object.

From Python docs; Assignment statements in Python do not copy objects, they create bindings between a target and an object. For collections that are mutable or contain mutable items, a copy is sometimes needed so one can change one copy without changing the other. This module provides generic shallow and deep copy operations (explained below).

edited Sep 4, 2014 at 9:31

answered Sep 4, 2014 at 9:23

Raul Andres

3,81617 silver badges24 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user2928681 Over a year ago

Nice, worked! You wouldn't happen to have an explanation for why this is a fix or perhaps a link that does?

Collectives™ on Stack Overflow

Numpy matrix doesn't update value

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related