Fill values in a numpy array given a condition

Question

Currently I have an array as follows:

myArray = np.array(
    [[ 976.77 ,  152.95 ,  105.62 ,   53.44 ,   0 ],
    [ 987.61 ,  156.63 ,  105.53 ,   51.1  ,    0 ],
    [1003.74 ,  151.31 ,  104.435,   52.86 ,    0 ],
    [ 968.   ,  153.41 ,  106.24 ,   58.98 ,    0 ],
    [ 978.66 ,  152.19 ,  103.28 ,   57.97 ,    0 ],
    [1001.9  ,  152.88 ,  105.08 ,   58.01 ,    0 ],
    [1024.93 ,  146.59 ,  107.06 ,   59.94 ,    0 ],
    [1020.01 ,  148.05 ,  109.96 ,   58.67 ,    0 ],
    [1034.01 ,  152.69 ,  107.64 ,   59.74 ,    0 ],
    [   0.   ,  154.88 ,  102.   ,   58.96 ,    0 ],
    [   0.   ,  147.46 ,  100.69 ,   54.95 ,    0 ],
    [   0.   ,  149.7  ,  102.439,   53.91 ,    0 ]]
)

I would like the fill in the zeros in the first column with the previous last value (1034.01) however if the 0's start from index 0, for it to remain as 0.

Example of end result:

myArrayEnd = np.array(
    [[ 976.77 ,  152.95 ,  105.62 ,   53.44 ,   0 ],
    [ 987.61 ,  156.63 ,  105.53 ,   51.1  ,    0 ],
    [1003.74 ,  151.31 ,  104.435,   52.86 ,    0 ],
    [ 968.   ,  153.41 ,  106.24 ,   58.98 ,    0 ],
    [ 978.66 ,  152.19 ,  103.28 ,   57.97 ,    0 ],
    [1001.9  ,  152.88 ,  105.08 ,   58.01 ,    0 ],
    [1024.93 ,  146.59 ,  107.06 ,   59.94 ,    0 ],
    [1020.01 ,  148.05 ,  109.96 ,   58.67 ,    0 ],
    [1034.01 ,  152.69 ,  107.64 ,   59.74 ,    0 ],
    [1034.01 ,  154.88 ,  102.   ,   58.96 ,    0 ],
    [1034.01 ,  147.46 ,  100.69 ,   54.95 ,    0 ],
    [1034.01 ,  149.7  ,  102.439,   53.91 ,    0 ]]
)

I would like the code to be applicable to any array not just this one, where the situation may be different. (Column 3 might be all 0's and Column 4 might have 0's in the middle which should be filled with the last previous value).

Will the 0s only exist at the end of a column or that 0s could exist in the middle of a column? Will there be cases like [1, 0, 1] for a column? If so, should the result be [1, 1, 1] or just [1, 0, 1]? — Tai
– Tai, Commented Jun 1, 2018 at 14:18
@Tai yes there will be arrays in my actual code where cases like [1, 0, 1] would exist — lee.edward01
– lee.edward01, Commented Jun 1, 2018 at 14:20

jpp · Accepted Answer · 2018-06-01 14:30:36Z

Here's a vectorised way with pandas. This is also possible with numpy. In any case, you should not need explicit loops for this task.

import pandas as pd
import numpy as np

df = pd.DataFrame(myArray)\
       .replace(0, np.nan)\
       .ffill().fillna(0)

res = df.values

print(res)

[[  976.77    152.95    105.62     53.44      0.   ]
 [  987.61    156.63    105.53     51.1       0.   ]
 [ 1003.74    151.31    104.435    52.86      0.   ]
 [  968.      153.41    106.24     58.98      0.   ]
 [  978.66    152.19    103.28     57.97      0.   ]
 [ 1001.9     152.88    105.08     58.01      0.   ]
 [ 1024.93    146.59    107.06     59.94      0.   ]
 [ 1020.01    148.05    109.96     58.67      0.   ]
 [ 1034.01    152.69    107.64     59.74      0.   ]
 [ 1034.01    154.88    102.       58.96      0.   ]
 [ 1034.01    147.46    100.69     54.95      0.   ]
 [ 1034.01    149.7     102.439    53.91      0.   ]]

AGN Gazer · Accepted Answer · 2018-06-01 14:56:13Z

2

Staying within numpy:

for k, c in enumerate(myArray.T):
    idx = np.flatnonzero(c == 0)
    if idx.size > 0 and idx[0] > 0:
        myArray[idx, k] = myArray[idx[0] - 1, k]

edited Jun 1, 2018 at 14:56

answered Jun 1, 2018 at 14:50

AGN Gazer

8,4272 gold badges31 silver badges49 bronze badges

1 Comment

jpp Over a year ago

Nice solution if you want to stick with numpy. Worth commenting these loopy operations can be optimised with numba.

End genocide - save Gaza · Accepted Answer · 2018-06-01 14:20:37Z

1

Assuming I've understood you correctly, this should do the trick:

def fill_zeroes(array):
    temp_array = array
    for i in xrange(1, len(temp_array)):
        if temp_array[i][0] == 0:
            temp_array[i][0] = temp_array[i-1][0]
    return temp_array

answered Jun 1, 2018 at 14:20

End genocide - save Gaza

25k10 gold badges113 silver badges133 bronze badges

Comments

Hein Wessels · Accepted Answer · 2018-06-01 14:20:25Z

0

How about something like this (in psuedo code)?

for each col in array
    for each row in col
        if array[col,row] == 0 && row>0
            array[col,row] = array[col,row-1]

edit: Combined with @ukemi, who has a quicker solution, but does not loop over the various columns. Also, you need to make sure to not try to index array[0][-1].

answered Jun 1, 2018 at 14:20

Hein Wessels

9475 silver badges15 bronze badges

Comments

Gomes J. A. · Accepted Answer · 2018-06-01 14:30:08Z

0

The code below requires testing:

values = myArray.to_list()    # don't remember if nd_array.to_list is a method or property
result = []
last = None
for i,item in enumerate(values):
    if i == 0 and item[0] == 0:
        last = item
    elif item[0] == 0 and last is not None:
        item[0] = last
    else:
        last = item[0]

    result.append(item)

answered Jun 1, 2018 at 14:30

Gomes J. A.

1,4852 gold badges10 silver badges9 bronze badges

Collectives™ on Stack Overflow

Fill values in a numpy array given a condition

5 Answers 5

Comments

1 Comment

Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

1 Comment

Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related