Increase speed of array population when using nested for loops - matlab

Question

I have written this function to find indices based on certain criteria. It should work, the problem is that it will take 2-3 days to run on my pc. Is there any way to get it down below an hour (or faster at all) ? This really doesn't need to be very fast. But 2 days is unacceptably slow.

I don't expect an in depth analysis on the function (Though it would be nice). Just some general improvements.

All it essentially is is 3 for-loops used to populate 8 large 3d arrays using another 256x8 matrix Logic. Then a few logic tests to find the desired index.

%These are sample values from the g.u.i. and other functions - 
%ignore up til the loops unless you need it to understand something in the loops.

PriceMat=[58867 55620 16682 97384 11660 18175 25896 16300];
CapMat=[1400 1200 450 3600 150 1330 2000 250];
RepMat=[58 53 31 127 15 164 242 27];
DesiredRep=293.04;
DesiredCap=2600;

prevmin=99999999;
P=perms(0:7);

D=zeros(256,8,40320);
Cap=zeros(size(D,3),8);
Rep=zeros(size(D,3),8);
Price=zeros(size(D,3),8);
SufRep=zeros(1,size(D,3));
SufCap=zeros(1,size(D,3));
CapTot=zeros(1,size(D,3));
RepTot=zeros(1,size(D,3));
PriceTot=zeros(1,size(D,3));


for i=1:40320
    for x=1:8

        for   j=1:256
            D(j,x,i)=P(i,x)*Logic(j,x);

            Cap(i,x)=D(j,x,i)*CapMat(x);
            Price(i,x)=D(j,x,i)*PriceMat(x);
            Rep(i,x)=D(j,x,i)*RepMat(x);
            CapTot=sum(Cap,2);
            RepTot=sum(Rep,2);
            PriceTot=sum(Price,2);

            if CapTot(i)>=DesiredCap
                SufCap(i)=true;
            else 
                SufCap(i)=false;
            end

            if RepTot(i)>=DesiredRep
                SufRep(i)=true;
            else 
                SufRep(i)=false;
            end

            if SufRep(i)==true && SufCap(i)==true

                if PriceTot(i)<=prevmin
                    prevmin=i;
                end
            end

        end

    end

end

return prevmin

Logic is another array. The 256 by 8 matrix I mentioned at the start is Logic, not D. Sorry about that. Will edit it now — Lkz
– Lkz, Commented Jul 11, 2013 at 16:16
Should be D(j,x,i). Copied from a previous attempt earlier and didn't notice that, thanks. — Lkz
– Lkz, Commented Jul 11, 2013 at 16:35
Can you please update your code so all "known bugs" are out? Thanks. — Floris
– Floris, Commented Jul 11, 2013 at 16:43

Shai · Accepted Answer · 2013-07-11 16:50:49Z

1

use bsxfun it's so much FUN!

Here's how you can compute matrix D in a single line (no loops):

 D = bsxfun( @times, permute( P, [3 2 1] ), Logic );

I guess you can take it from here and compute the rest of the matrices this way - no loops.

edited Jul 11, 2013 at 16:50

answered Jul 11, 2013 at 16:32

Shai

115k39 gold badges259 silver badges398 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Floris Over a year ago

I didn't realize you could permute a 2D matrix into a 3D one with the trick you used - that is classy. I don't think you want Logic to be transposed though... order of its dimensions is (j,x) same as for D

Shai Over a year ago

@Floris check out the bsxfun tag wiki for more nice tips and tricks.

Floris · Accepted Answer · 2013-07-11 18:12:19Z

0

You said "it would be nice" to get some in depth analysis of your function. It's pretty complex -and can be greatly simplified. I am a little bit worried about the amount of memory that my solution would take - one of your 256x8x40320 arrays is about 660 MB, and I create four. If that's not a problem, great. Otherwise you might have to choose a more conservative data type to keep memory requirements down - if you start swapping to disk you are dead, timing wise.

So let's assume you are not limited by RAM, then the following will speed things up considerably (note - I am stealing Shai's suggestion to use bsxfun). Note also that I am clearing the "really big" arrays after taking their sum - this could all be done in one line but it would be even harder for you to follow:

D = bsxfun( @times, permute( P, [ 3 2 1] ), Logic );
Cap = bsxfun( @times, D, CapMat );
CapTot = sum( Cap, 2 );
clear Cap

Price = bsxfun( @times, PriceMat );
PriceTot = sum( Price, 2 );
clear Price

Rep = bsxfun( @times, D, RepMat ); % <<<<< STRONGLY recommend not to use RepMat -
                                   % <<<<< to avoid confusion with built in repmat()
RepTot = sum( Rep, 2 );
clear Rep

CapRepOK = ( CapTot >= DesiredCap && RepTot >= DesiredRep ); % logical array - fast, small

[minPrice minPriceInd ] = min(PriceTot(CapRepOK)); % find minimum value and index

% convert index to correct value of `i` in original code:
cs = cumsum(ok(:)); % increases by one for every value that meets criteria
                    % but we need to know which original index that corresponds to:
possibleValues = find( cs == minPriceInd );
[j i] = ind2sub( size(CapRepOK), possibleValues(1) );
prevmin = i;

Obviously I don't have your data so it's a bit hard to be sure this replicates your functionality exactly - but I believe it does. If not - that's what comments are for.

I suspect it is possible never to create the largest arrays (D etc) with some careful thought - if you are truly memory starved that may be needed.

edited Jul 11, 2013 at 18:12

answered Jul 11, 2013 at 17:49

Floris

46.6k7 gold badges73 silver badges128 bronze badges

2 Comments

Lkz Over a year ago

Unfortunately memory is quite a big problem. Im running a 32bit with 4GB RAM - . The maximum array size my pc can handle is about 700-1200MB. Total Memory available for all arrays is roughly 1500-1700MB with everything I know of done to free up memory. I could run it on a colleague's better PC I suppose. Thank you very much for your effort, will definitely try some of your proposals and experiment with bsxfun.

Floris Over a year ago

For many applications it is sufficient to use float as the data type - it depends a bit on the range of values and the required accuracy. If so, then making P, Logic, CapMat, PriceMat and RepMat float from the outset will halve the memory requirement and may well give a speed boost.

Collectives™ on Stack Overflow

Increase speed of array population when using nested for loops - matlab

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related