0

So I have a source array like this:

 [[  9  85  32 100]
 [  7  80  30 100]
 [  2  90  16 100]
 [  6 120  22 100]
 [  5 105  17 100]
 [  0 100  33 100]
 [  3 110  22 100]
 [  4  80  22 100]
 [  8 115  19 100]
 [  1  95  28 100]]

and I want to update the array with this one, depend on the first column

[[  3 110  22 105]
 [  5 105  17 110]
 [  1  95  28 115]]

to be like this

 [[  9  85  32 100]
 [  7  80  30 100]
 [  2  90  16 100]
 [  6 120  22 100]
 [  5 105  17 110]
 [  0 100  33 100]
 [  3 110  22 105]
 [  4  80  22 100]
 [  8 115  19 100]
 [  1  95  28 115]]

but I can't find a function in NumPy can do this directly, so currently have no way to do that better than this method I wrote:

def update_ary_with_ary(source, updates):
    for x in updates:
        index_of_col = np.argwhere(source[:,0] == x[0])
        source[index_of_col] = x

This function makes a loop so it's not professional and not have high performance so I will use this until some-one give me a better way with NumPy laps, I don't want a solution from another lap, just Numpy

Improve this question
0

2 Answers 2

Reset to default
1

Assuming your source array is s and update array is u, and assuming that s and u are not huge, you can do:

update_row_ids = np.nonzero(s[:,0] == u[:,0].reshape(-1,1))[1]
s[update_row_ids] = u

Testing:

import numpy as np
s = np.array(
    [[  9,  85,  32, 100],
     [  7,  80,  30, 100],
     [  2,  90,  16, 100],
     [  6, 120,  22, 100],
     [  5, 105,  17, 100],
     [  0, 100,  33, 100],
     [  3, 110,  22, 100],
     [  4,  80,  22, 100],
     [  8, 115,  19, 100],
     [  1,  95,  28, 100]])
u = np.array(
    [[  3, 110,  22, 105],
     [  5, 105,  17, 110],
     [  1,  95,  28, 115]])

update_row_ids = np.nonzero(s[:,0] == u[:,0].reshape(-1,1))[1]
s[update_row_ids] = u

print(s)

This prints:

[[  9  85  32 100]
 [  7  80  30 100]
 [  2  90  16 100]
 [  6 120  22 100]
 [  5 105  17 110]
 [  0 100  33 100]
 [  3 110  22 105]
 [  4  80  22 100]
 [  8 115  19 100]
 [  1  95  28 115]]

Edit: OP has provided the following additional details:

  • The "source array" is "huge".
  • Each row in the "update array" matches exactly one row in the "source array".

Based on this additional detail, the following alternative solution might provide a better performance, especially if the source array does not have its rows sorted on the first column:

sorted_idx = np.argsort(s[:,0])
pos = np.searchsorted(s[:,0],u[:,0],sorter=sorted_idx)
update_row_ids = sorted_idx[pos]

s[update_row_ids] = u
Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

0

fountainhead your answer works correctly and yes it's full used Numpy laps, but in the performance test, it's rise the time on processing 50K rows in my simulation program in double!! from 22 seconds to 44 seconds!! I don't know why!! but your answer helps me to get the right answer on only this line:

source[updates[:,0]] = updates
# or 
s[u[:,0]] = u

so when I use this its lower processing time from for 100K rows to only 0.5 seconds and then let me process more like 1M rows for only 5 seconds, am already learning python and data mining am shocked from these numbers, it's never happing before on other languages I play on the huge array like regular variables. you can see that on my GitHub.

https://github.com/qahmad81/war_simulation

fountainhead you should take the answer but visited should know the best answer to use.

Improve this answer

4 Comments

I double-checked again. Your solution of s[u[:,0]] = u will not produce the result that you asked for. So, either your question is wrong, or your solution of s[u[:,0]] = u is wrong. As for speed, your question did not mention anything about the sizes involved. I made some assumptions (which I did mention, though in non-quantitative terms). If you're still looking for some answers, or, some explanation of your speed observations, I think you should provide a lot more detail about the sizes involved.
your right, in my code the order of source array and col(0) is the same so its works for me, so your answer is right until we get an answer with higher performance
One more question about your application/data -- will each row of u match exactly one row of s, or is it possible that it may match (and update) more than one row of s ? And what could be the potential (worst-case) lengths of s and u ?
no its should not be in my case its unique rows by col(0) in (s) and (u), currently s[u[:,0]] = u is working well as rows in (s) are sorted, but the problem stays in future when I need to update array to array depend on col in both.. this not fixed yet

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.