0

So, my python script generates a (25,8) variable "latent_vars". I save this variable as a .mat file. Then, from python, I run a matlab script that runs a trained kriging model with "latent_var" as input. The kriging model (in matlab) outputs "comp_vals", a 1x25 double variable, which I then save as a .mat file and open back up in python. The problem is that this process takes a while (about a minute and 15 seconds), but I need it to occur in under 3 seconds.

Is there any other way I can go about this to speed the process up?

This is they python code:

import scipy.io as sio   
import numpy as np
sio.savemat('latent_vars.mat', {'latent_vars':noise})

import matlab.engine
eng = matlab.engine.start_matlab()
eng.Kriging_to_python(nargout=0)
eng.quit()

comp_vals=sio.loadmat('comp_vals.mat')

And this is the matlab code:

%Load matlab workspace with kriging model
krig_workspace=load('C:\Users\User\Documents\MATLAB\krig_posttrain_workspace2.mat');
%Import latent variable values from python
latent_from_python=load('/Users/User/Documents/project/Interactive Framework/latent_vars.mat');
latent_vars=latent_from_python.latent_vars;
latent_vars=single(latent_vars);
%Save kriging model from workspace as new variable
krig_model=krig_workspace.dmodel;
%Run kriging model with latent variables generated from python
[comp_vals MSEpredict] = predictor(latent_vars, krig_model);
comp_vals=transpose(comp_vals);
save('/Users/User/Documents/project/Interactive Framework/comp_vals.mat','comp_vals');

Any help/insight would be appreciated! Thanks.

4
  • 2
    Have you tried checking which line your python code actually takes bulk of the time? Answering this question may help to know where to optimize. Commented Sep 21, 2021 at 22:40
  • 1
    I’m guessing it is starting up MATLAB that takes most of the time. I would suggest you start MATLAB once at the start of your program, and leave it open to repeatedly run your code there (I presume you repeatedly run the code because of your time limit). A second thing to optimize is to put the array directly into MATLAB, then pull the result back out. You don’t need files for that. If you leave MATLAB open, you can load your model only once, and keep it in memory. Commented Sep 22, 2021 at 0:38
  • 1
    If you repeatedly run your code in a new Python session, consider connecting to a running MATLAB session. It is fast to start up Python, but it takes for ever to start up MATLAB, you really want to reuse the MATLAB session as much as you can. Commented Sep 22, 2021 at 0:42
  • Another alternative, since your MATLAB script doesn’t require any interaction with Python, is to run it by matlab -batch scriptname, which you can do from within Python with os.system. Commented Sep 22, 2021 at 0:47

1 Answer 1

1

In addition to nice comments by Cris Luengo:

  1. One simple (and hopefully applicable) option is to do it vice-versa. That is, run Python from Matlab, not Matlab from Python, much like in the official tutorial. According to my own experience, this way it generally works better. In this case, you only start Matlab once (and Matlab also only initializes Python interpreter once) you do not suffer from any startup overhead each time, if you run your workflow repeatedly. This approach would guarantee that the communication overhead would be below 1 second if the size of data you send is below few megabytes.

  2. Another possible approach involves Matlab Compiler SDK toolbox (example) - it allows to build a package out of your Matlab code, that is usually significantly more lightweight to initialize than the whole Matlab. The initialization of this Matlab package is not guaranteed to be below 3 seconds (but might be pretty close). Once initialized, there would be an easy and natural way to reuse your Matlab function without Matlab reinitialization.

  3. matlab -batch scriptname could be further improved with -nojvm option a bit (because, AFAIU, you do not use the related functionality). That is,

    C:\Program Files\MATLAB\R2021a\bin>matlab.exe -nojvm -batch "disp(1+1)"
    

    should be a bit faster (takes about 3 seconds on my PC), compared to

    C:\Program Files\MATLAB\R2021a\bin>matlab.exe -batch "disp(1+1)"
    

    (takes about 6 seconds on my PC).

Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.