Vectorizing numpy Multiple Condition Nested Loops

Question

On attempting to produce Automatic Peak Detection in Noisy Periodic and Quasi-Periodic Signals, by Felix Scholkmann, Jens Boss and Martin Wolf in Python, I've hit a stumbling block in the implementation.

Upon attempting to optimise, I've noticed that the nested for loops are creating a bottleneck in processing time (taking 115394 ms on average to complete). Is there a more efficient means of constructing the nested for loop?

N.B: The parameter, signal, is a list of co-ordinates to which the algorithm will process which is of the form

-48701.0 -20914.0 -1757.0 -49278.0 -106781.0 -88139.0 -13587.0 28071.0 11880.0 -13375.0 -18056.0 -15248.0 -12476.0 -9832.0 -26365.0 -65734.0 -81657.0 -41566.0 6382.0 872.0 -30666.0 -20261.0 17543.0 6278.0 ...

The list is 32768 lines long.

The function returns the indexes of the peaks detected to which is processed in another function.

def ampd(signal):

    s_time = range(1, len(signal)+1)

    [fitPolynomial, fitError] = np.polyfit(s_time, signal, 1)
    fitSignal = np.polyval([fitPolynomial, fitError], s_time)

    dtrSignal = signal - fitSignal

    N = len(dtrSignal)
    L = math.ceil(N/2.0)-1

    creation_start = time.time()
    np.random.seed(1969)

    LSM = np.random.uniform(0, 2, size=(L, N))
    creation_elapsedTime = time.time() - creation_start
    print('LSM created in %s ms' % int(creation_elapsedTime * 1000))

    loop_start = time.time()
    for k in range(1, L):
        for i in range(k+2, N-k+1):
            if signal[i-1]>signal[i-k-1] and signal[i-1]>signal[i+k-1]:
                    LSM[k,i] = 0

    loop_elapsedTime = time.time() - loop_start
    print('Loop completed in %s ms' % int(loop_elapsedTime * 1000))

    G = np.sum(LSM, axis=1)

    l = min(enumerate(G), key=itemgetter(1))[0]

    MLSM = LSM[0:l]

    S = np.std(MLSM, ddof=1)

    found_indices = np.where(MLSM == ((S-1) == 0))
    del LSM
    del MLSM

    return found_indices[1]

Is the PyDSTool module useful to you? mail.scipy.org/pipermail/scipy-user/2009-September/022576.html — Alan
– Alan, Commented Mar 10, 2016 at 12:44
It will be as an alternate option if I can't find an adequate solution to the problem. Thank you! — AGreatPigeon
– AGreatPigeon, Commented Mar 10, 2016 at 13:04
The linked data file is locked (and external linkage is discouraged because of access and link-rot). Can you add a short sample of the data to the question? — msw
– msw, Commented Mar 10, 2016 at 13:59

JPG · Accepted Answer · 2016-03-10 17:35:34Z

2

Here is a solution which uses only one loop

for k in range(1, L):
    mat=1-((signal[k+1:N-k]>signal[1:N-2*k]) & (signal[k+1:N-k]>signal[2*k+1:N]))
    LSM[k,k+2:N-k+1]*=mat

it's faster and seems do give the same solutions. You compare slices (as suggested by Ami Tavory), combine the comparisons with a &, which gives a True/False array; with 1-operation, you transform it to zeros and ones, the zeros corresponding to where the conditions are met. And lastly you multiply the row by the result.

answered Mar 10, 2016 at 17:35

JPG

2,2642 gold badges16 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

AGreatPigeon Over a year ago

Thank you Ami and JPG for your assistance. Between the both of you, it is now running superbly. Thank you once again

Collectives™ on Stack Overflow

Vectorizing numpy Multiple Condition Nested Loops

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related