Execute Fortran subroutine through parallel Python impossible to execute

Question

I am trying to execute a Fortran subroutine in Python 2.7 using parallel python (PP package). But when I execute it using pp.server().submit(...) nothing happens. I might have implemented it wrong, I have used the numpy.f2py.compile() as explained here. Is this correct? And if not, what should I change?

Just to mention that the code is almost surely correct given that it is taken from a doctorate thesis (code, paper).

The subroutine implemented in a Python 2.7 module called "design_operation" is:

import numpy.f2py
fsource = '''
subroutine matrix_op(grid_x,grid_t,eval_grid,pas,K,L,M,C)
  COMPLEX :: i=(0.0,1.0)
  INTEGER , intent(in) :: K,L,M
  REAL , intent(in) :: pas
  INTEGER :: u,v,w
  REAL , dimension(1:M) , intent(in) :: grid_x
  REAL , dimension(1:K) , intent(in) :: grid_t
  REAL , dimension(1:L) , intent(in) :: eval_grid
  COMPLEX, dimension(1:L,1:M) , intent(out) :: C

  do u=1,L
     do v=1,M
        do w=1,K
           C(u,v) = C(u,v) - i*pas*grid_t(w)*grid_x(v)*exp(-i*grid_t(w)*grid_x(v)*eval_grid(u))
        end do
     end do
  end do
  end subroutine matrix_op
  '''
numpy.f2py.compile(fsource, modulename='design_operation', verbose=0)

Then, I call it this way:

job_server.submit(func=list_append,
                                  args=(grid_x, grid_t, sub_eval[k,:], pas_f,),
                                  modules=('numpy as np','design_operation as fdp',)

which is actually in a loop and should be executed in:

job_server = pp.Server()
thread_number = job_server.get_ncpus()
...some unimportant code ...
jobs = []
for k in range(thread_number): 
    jobs.append(job_server.submit(func=list_append,
                                  args=(grid_x, grid_t, sub_eval[k,:], pas_f,),
                                  modules=('numpy as np','design_operation as fdp',)))

for i,job in enumerate(jobs):
    if i == 0:
        dM = job() 
    else:
        dM = np.concatenate((dM, job()))

job_server.destroy()
return dM

I always get the following error:

zero-dimensional arrays cannot be concatenated.

Therefore I suppose that the error comes from the incorrect execution of the tasks, but perhaps am I mistaking.

The stack error is:

An error has occured during the function execution
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\project\lib\site-packages\ppworker.py", line 90, in run
    __result = __f(*__args)
  File "<string>", line 2, in list_append
AttributeError: 'module' object has no attribute 'matrix_op'
An error has occured during the function execution
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\project\lib\site-packages\ppworker.py", line 90, in run
    __result = __f(*__args)
  File "<string>", line 2, in list_append
AttributeError: 'module' object has no attribute 'matrix_op'
Traceback (most recent call last):

  File "<ipython-input-89-6cb5b50fd813>", line 5, in <module>
    dM = np.concatenate((dM, job()))#

ValueError: zero-dimensional arrays cannot be concatenated

PS: I supposed there is some unnecessary code and therefore I didn't include it for more clarity + the arguments of func= list_append are correct.

I'm not familiar with the parallel python package, but somewhat with F2PY. You get an error message that "'module' object has no attribute 'matrix_op'", that could indicate that the function is not being loaded. You import the F2PY module design_operation as fdp, but you do not show where the fdp.matrix_op is being used? How is list_append defined in your code? — jbdv
– jbdv, Commented Mar 11, 2019 at 8:15

Matt P · Accepted Answer · 2019-03-09 17:59:02Z

1

I notice your code doesn't seem to pass the K, L, M and C variables to the Fortran routine. However, K, L, M are used to dimension some arrays and are also used as loop counters. It's quite possible that these values are set to a default value of 0 by the compiler, or maybe more likely as None by Python itself. That would explain your error message `ValueError: zero-dimensional arrays cannot be concatenated'.

answered Mar 9, 2019 at 17:59

Matt P

2,4071 gold badge15 silver badges28 bronze badges

Sign up to request clarification or add additional context in comments.

6 Comments

jbdv Over a year ago

F2PY creates wrappers so that a Fortran subroutine can be called like a Python function (i.e. the intent(out) argument C becomes the return value of the function), and dimensions for automatic arrays do not need to be passed explicitly, see this example in the documentation. So I don't believe that is the problem in this case.

Matt P Over a year ago

@jbdv That surpises me a bit based on my reading of the docs here (see 'intent', especially 'in', 'out', and 'hide'). Not to say that things don't 'work out' in some cases even without doing so. I'd have to experiment to know more.

jbdv Over a year ago

As I read that, C which is intent(out) in Fortran, the F2PY intent out and hence hide implies that "The argument is considered as a return variable" and "The argument is removed from the list of required or optional arguments", so when called from Python, C should not be passed to the function, but is the function's return value.

jbdv Over a year ago

While the arguments defining array dimensions K, L and M are Fortran intent(in) and will get the F2PY attributes intent in and optional, so that they can, but don't have to, be passed to the wrapped function (but are inferred from and checked against the passed array arguments grid_t, eval_grid and grid_x). I may very well be wrong, but I believe this is not what's causing the OP's problems

Matt P Over a year ago

@jbdv Regarding C, I believe we are in agreement - particularly since C is the only intent(out) variable then hide is implied. However, it seems to me that even the example you linked illustrates how intent(in) array dims must be handled explicitly to avoid issues (in that example, the interface defines how the array length is to be handled, which allows it to be optional). That's not done in the OP's code. Aside: By now, I think I could have just run an experiment, but I haven't :)

|

Collectives™ on Stack Overflow

Execute Fortran subroutine through parallel Python impossible to execute

1 Answer 1

6 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

6 Comments

Your Answer

Sign up or log in

Post as a guest

Related