12

I was unable to find anything describing how to do this, which leads to be believe I'm not doing this in the proper idiomatic Python way. Advice on the 'proper' Python way to do this would also be appreciated.

I have a bunch of variables for a datalogger I'm writing (arbitrary logging length, with a known maximum length). In MATLAB, I would initialize them all as 1-D arrays of zeros of length n, n bigger than the number of entries I would ever see, assign each individual element variable(measurement_no) = data_point in the logging loop, and trim off the extraneous zeros when the measurement was over. The initialization would look like this:

[dData gData cTotalEnergy cResFinal etc] = deal(zeros(n,1));

Is there a way to do this in Python/NumPy so I don't either have to put each variable on its own line:

dData = np.zeros(n)
gData = np.zeros(n)
etc.

I would also prefer not just make one big matrix, because keeping track of which column is which variable is unpleasant. Perhaps the solution is to make the (length x numvars) matrix, and assign the column slices out to individual variables?

EDIT: Assume I'm going to have a lot of vectors of the same length by the time this is over; e.g., my post-processing takes each log file, calculates a bunch of separate metrics (>50), stores them, and repeats until the logs are all processed. Then I generate histograms, means/maxes/sigmas/etc. for all the various metrics I computed. Since initializing 50+ vectors is clearly not easy in Python, what's the best (cleanest code and decent performance) way of doing this?

Improve this question

4 Answers 4

Reset to default
24

If you're really motivated to do this in a one-liner you could create an (n_vars, ...) array of zeros, then unpack it along the first dimension:

a, b, c = np.zeros((3, 5))
print(a is b)
# False

Another option is to use a list comprehension or a generator expression:

a, b, c = [np.zeros(5) for _ in range(3)]   # list comprehension
d, e, f = (np.zeros(5) for _ in range(3))   # generator expression
print(a is b, d is e)
# False False

Be careful, though! You might think that using the * operator on a list or tuple containing your call to np.zeros() would achieve the same thing, but it doesn't:

h, i, j = (np.zeros(5),) * 3
print(h is i)
# True

This is because the expression inside the tuple gets evaluated first. np.zeros(5) therefore only gets called once, and each element in the repeated tuple ends up being a reference to the same array. This is the same reason why you can't just use a = b = c = np.zeros(5).

Unless you really need to assign a large number of empty array variables and you really care deeply about making your code compact (!), I would recommend initialising them on separate lines for readability.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

This gets the closest to what I'm trying to do. In some of my other Matlab code - the stuff used to analyze the output of the data logger - I wind up creating a very large number of vectors to store computed metrics for each run. They all get initialized as part of a big deal(zeros(n,1)), and I prefer to not have to keep track of how many there are on the left side of the assignment. I rather get the feeling I should use a structure instead, but Matlab's syntax makes this somewhat painful. See the edit to my question.
Sorry, you're going to have to keep track of how many variables you're assigning on the LHS yourself. The problem is that in Python the function that's being called has no idea what's going to be done with the return value, so it can't know how many are needed on the LHS. Think of it as the price you pay for having the flexibility to do things like indexing into the result of a function call.
You might consider using a recarray to keep track of multiple vectors empty vectors of the same length with different 'column names'
5

Nothing wrong or un-Pythonic with 

dData = np.zeros(n)
gData = np.zeros(n)
etc.

You could put them on one line, but there's no particular reason to do so.

dData, gData = np.zeros(n), np.zeros(n)

Don't try dData = gData = np.zeros(n), because a change to dData changes gData (they point to the same object). For the same reason you usually don't want to use x = y = [].

The deal in MATLAB is a convenience, but isn't magical. Here's how Octave implements it

function [varargout] = deal (varargin)
  if (nargin == 0)
    print_usage ();
  elseif (nargin == 1 || nargin == nargout)
    varargout(1:nargout) = varargin;
  else
    error ("deal: nargin > 1 and nargin != nargout");
  endif

endfunction

In contrast to Python, in Octave (and presumably MATLAB)

one=two=three=zeros(1,3)

assigns different objects to the 3 variables.

Notice also how MATLAB talks about deal as a way of assigning contents of cells and structure arrays. http://www.mathworks.com/company/newsletters/articles/whats-the-big-deal.html

Improve this answer

1 Comment

Thanks, I appreciate this discussion on deal(); I marked the other answer, but I found this one very helpful as well and wish I didn't have to pick just one.
0

If you put your data in a collections.defaultdict you won't need to do any explicit initialization. Everything will be initialized the first time it is used.

import numpy as np
import collections
n = 100
data = collections.defaultdict(lambda: np.zeros(n))
for i in range(1, n):
    data['g'][i] = data['d'][i - 1]
    # ...
Improve this answer

Comments

0

How about using map:

import numpy as np
n = 10  # Number of data points per array
m = 3   # Number of arrays being initialised
gData, pData, qData = map(np.zeros, [n] * m)
Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.