numpy array conversion to pairs

Question

I'm not sure how it is called in python i think its a pair Anyway I have a huge numpy array, its format is like

FFnetlayer0 =   [ 0,  243,    9,  243,   18,  243,    4,  244,   13,  244, ....etc.]

I need this numpy array format to be:

FFnetlayer0 =   [ (0,  243),    (9,  243),   (18,  243),    (4,  244),   (13,  244), .....]

Nodepairs needs to be between ( ) for building manually a neural net FFnet for python I'm building a huge neural net, so i use a function to create the array but i dont get the ( ) symbols included

conec =[]
for i in range (3):
    conec = numpy.append(conec,[(i,243),(i+9,243),(i+18,243)])
    d = 4
    conec = numpy.append(conec,[(i+d,244),(i+9+d,244),(i+18+d,244)])
    d = 7
    ...
    ..
    .

Thanks for all your answers; there is truth in all of them, and pywisdom for me to learn. Meanwhile i found that FFnet, also accepts arrays like [[0,0],[1,2]] besides [(0,0),(.....] Best of all i think is the reshape option, so i dont have to change all my previous code. — user613326
– user613326, Commented Feb 15, 2011 at 5:45
Thanks for all your answers; there is truth in all of them, and pywisdom for me to learn. Meanwhile i found that FFnet, also accepts arrays like [[0,0],[1,2]] besides [(0,0),(.....] Best of all i think is the reshape option, so i dont have to change all my previous code. — user613326
– user613326, Commented Feb 15, 2011 at 5:46

Sven Marnach · Accepted Answer · 2011-02-11 16:19:29Z

7

One way is to just convert it to a two-dimensional NumPy array:

FFnetlayer0 = FFnetlayer0.reshape(-1, 2)

Now, accessing FFnetlayer0[i] for some i will give you a NumPy array with two entries.

answered Feb 11, 2011 at 16:19

Sven Marnach

608k123 gold badges968 silver badges865 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

user613326 Over a year ago

well this is what i'm going to use in my code so i vote it as answer; all other answers where great too, and i had to decide on something, i'm not sure who is best here. and other answers gave me better insight on numpy arrays. and others on python arrays; i have to thank you all.

dr jimbob · Accepted Answer · 2011-02-12 15:58:47Z

4

Well your FFnetlayer0 isn't an numpy array, its still just a python list. You can slice it

from numpy import array
FFnetlayer = [0,243, 9,243, 18,243]
first_array = array(FFnetlayer[0::2]) # array([0,9,18])
second_array = array(FFnetlayer[1::2]) # array([243,243,243])

Unless its a matrix, I don't see the benefit of making it a two-dimensional array rather than two separate arrays.

You can also zip the two lists together if you didn't need to use them in numpy as a:

list_of_tuples = zip(FFnetlayer0[0::2], FFnetlayer0[1::2]) # [(0,243), (9,243), (18,243)]
array_of_list_of_tuples = array(list_of_tuples) # array([[0,243],[9,243],[18,243]])

For more into extended slices (or strides) see: http://docs.python.org/release/2.3/whatsnew/section-slices.html

For more on zip see: http://docs.python.org/library/functions.html#zip

Noticed in the comments that you created the numpy array via append. Noted that numpy.append doesn't append in-place, so isn't an efficient way of extending long arrays.

E.g.,

ff_list = [(0,243), (9,243)]
orig_id = id(ff_list)
for i in range(1000):
    ff_list.append((i,243))
    assert(orig_id == id(ff_list)) # Assertion is always True
ff_array = numpy.array(ff_list) # This will copy the list into an array; but does this only once rather than N times.

whereas

ff_array = numpy.array([(0,243), (9,243)])
last_id = id(ff_array)
for i in range(1000):
    ff_array = numpy.append(ff_array, (i,243))
    assert(last_id != id(ff_array)) # Assertion is True as array is always different.
    last_id = id(ff_array)

id tells the memory location of a python object. Note, this may not be a big difference unless your arrays are large and frequently appended. Also if at all possible its best to do array math to construct large arrays, rather than element by element for loops or appending.

edited Feb 12, 2011 at 15:58

answered Feb 11, 2011 at 16:23

dr jimbob

17.8k7 gold badges63 silver badges84 bronze badges

3 Comments

senderle Over a year ago

You're right that FFnetlayer0 is a plain list; but the numpy.append() lines do in fact return numpy arrays.

dr jimbob Over a year ago

FYI if your arrays are large, numpy.append is not efficient. Each numpy.append takes the old array copies the first part of it to a new array with the second part appended to the end of the array (check yourself with id -- the memory location will change after each numpy.append call). This is inefficient. Note list.append is in-place (e.g., a_list.append(b); though note a_list + [b] is not in-place).

senderle Over a year ago

+1, good point, though I think you should mention this in the body of your answer.

akira · Accepted Answer · 2011-02-11 16:26:28Z

2

a = [ 0,  243,    9,  243,   18,  243]
zip(a[::2],a[1::2])

answered Feb 11, 2011 at 16:26

akira

6,19732 silver badges37 bronze badges

Comments

senderle · Accepted Answer · 2011-02-11 22:16:44Z

1

The default behavior when appending to numpy arrays is to flatten them. But once you have a 2-d numpy array, you can append to it without flattening it; you just have to specify the axis argument:

>>> conec = []
>>> for i in range(3):
...     conec = numpy.append(conec,[(i,243),(i+9,243),(i+18,243)])
...     conec = conec.reshape(-1, 2)
...     d = 4
...     conec = numpy.append(conec,[(i+d,244),(i+9+d,244),(i+18+d,244)], axis=0)
... 
>>> conec
array([[  0,   0], [  0, 243], [  9, 243], [ 18, 243], [  4, 244],
       [ 13, 244], [ 22, 244], [  1, 243], [ 10, 243], [ 19, 243],
       [  5, 244], [ 14, 244], [ 23, 244], [  2, 243], [ 11, 243],
       [ 20, 243], [  6, 244], [ 15, 244], [ 24, 244]])

Still, it may be easier to simply fill the array and then reshape.

EDIT: As jimbob correctly points out, this isn't the most efficient way to build a large array. As an alternative, consider numpy.fromiter()

edited Feb 11, 2011 at 22:16

answered Feb 11, 2011 at 17:14

senderle

152k36 gold badges218 silver badges244 bronze badges

1 Comment

user613326 Over a year ago

Thanks; i'm trying to build aneural filter for video; (and i will be over 255 nodes); all your answers are realy helpfull to me/

Heini Høgnason · Accepted Answer · 2011-02-11 16:46:34Z

You can achieve this using numpy records.

It seems that all of your numbers are under 255; therefore I assume that you will not need a datatype that can handle numbers larger than that. Should you require another datatype, you can replace np.int8 with the appropriate type.

import numpy as np

structure_type = np.dtype([('field1', np.int8), ('field2', np.int8)])
values = np.array([(0, 243), (9, 243), (18, 243)], dtype=structure_type)
print "Entire array:", values
print "First field only:", values['field1']
print "Second element:", values[1]

Output:

Entire array: [(0, 243) (9, 243) (18, 243)]
First field only: [  0   9  18]
Second element: (9, 243)

As an aside, you do not seem to be using numpy arrays in your code; rather, you are using the function numpy.append to extend a Python list.

Collectives™ on Stack Overflow

numpy array conversion to pairs

5 Answers 5

1 Comment

3 Comments

Comments

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

1 Comment

3 Comments

Comments

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related