Handle Fortran Character Arrays from Python with F2PY

Question

I have a legacy Fortran library I've wrapped with F2PY. However, I'm at a loss for how to properly read character arrays declared as module data, from Python. The data data comes through, but the array is transposed in such a way that it is indiscernible. How can I get Numpy to correctly handle my array? I'd be satisfied with a 2 dimensional array of characters if they were in an intelligible order.

The character arrays are declared and populated in Fortran like so:

module plot_mod
    implicit none

    CHARACTER*4, JSP(39)

    ...

    JSP = (/ &
      'SF ',   'WF ',   'GF ',   'AF ',   'RF ',   'SS ',   'NF ', &
      'YC ',   'IC ',   'ES ',   'LP ',   'JP ',   'SP ',   'WP ', &
      'PP ',   'DF ',   'RW ',   'RC ',   'WH ',   'MH ',   'BM ', &
      'RA ',   'WA ',   'PB ',   'GC ',   'AS ',   'CW ',   'WO ', &
      'WJ ',   'LL ',   'WB ',   'KP ',   'PY ',   'DG ',   'HT ', &
      'CH ',   'WI ',   '   ',   'OT '/)

end module plot_mod

In Python 2.7 (previous version of numpy) I could do this:

x = numpy.frombuffer(fvslib.plot_mod.jsp.data, numpy.dtype('a4'))

But now Python (3.4.4) and Numpy (1.10.4) raises an error, BufferError: memoryview: underlying buffer is not C-contiguous.

I know I should be able to get Numpy to handle this for me by reshaping, or using stride tricks, but I can't seem to figure it out. The array is reported as F-contiguous, so at least that seems correct.

If I simply print the array it looks like this:

array([[b'S', b' ', b' ', b'L'],
   [b'F', b'L', b' ', b' '],
   [b' ', b'P', b'B', b' '],
   [b' ', b' ', b'M', b'W'],
   [b'W', b' ', b' ', b'B'],
   [b'F', b'J', b' ', b' '],
   [b' ', b'P', b'R', b' '],
   [b' ', b' ', b'A', b'K'],
   [b'G', b' ', b' ', b'P'],
   [b'F', b'S', b' ', b' '],
   [b' ', b'P', b'W', b' '],
   [b' ', b' ', b'A', b'P'],
   [b'A', b' ', b' ', b'Y'],
   [b'F', b'W', b' ', b' '],
   [b' ', b'P', b'P', b' '],
   [b' ', b' ', b'B', b'D'],
   [b'R', b' ', b' ', b'G'],
   [b'F', b'P', b' ', b' '],
   [b' ', b'P', b'G', b' '],
   [b' ', b' ', b'C', b'H'],
   [b'S', b' ', b' ', b'T'],
   [b'S', b'D', b' ', b' '],
   [b' ', b'F', b'A', b' '],
   [b' ', b' ', b'S', b'C'],
   [b'N', b' ', b' ', b'H'],
   [b'F', b'R', b' ', b' '],
   [b' ', b'W', b'C', b' '],
   [b' ', b' ', b'W', b'W'],
   [b'Y', b' ', b' ', b'I'],
   [b'C', b'R', b' ', b' '],
   [b' ', b'C', b'W', b' '],
   [b' ', b' ', b'O', b' '],
   [b'I', b' ', b' ', b' '],
   [b'C', b'W', b' ', b' '],
   [b' ', b'H', b'W', b' '],
   [b' ', b' ', b'J', b'O'],
   [b'E', b' ', b' ', b'T'],
   [b'S', b'M', b' ', b' '],
   [b' ', b'H', b'L', b' ']],
  dtype='|S1')

What I would like an array like this:

[['SF ']
, ['WF ']
, ['GF ']
, ['AF ']
, ['RF ']
, ['SS ']
, ['NF ']
, ['YC ']
, ['IC ']
, ['ES ']
, ['LP ']
, ['JP ']
, ['SP ']
, ['WP ']
, ['PP ']
, ['DF ']
, ['RW ']
, ['RC ']
, ['WH ']
, ['MH ']
, ['BM ']
, ['RA ']
, ['WA ']
, ['PB ']
, ['GC ']
, ['AS ']
, ['CW ']
, ['WO ']
, ['WJ ']
, ['LL ']
, ['WB ']
, ['KP ']
, ['PY ']
, ['DG ']
, ['HT ']
, ['CH ']
, ['WI ']
, ['   ']
, ['OT ']]

Warren Weckesser · Accepted Answer · 2016-10-05 01:11:53Z

I haven't tried running f2py on your module, but if I define the array you show as:

In [11]: s = array([[b'S', b' ', b' ', b'L'],
    ...:    [b'F', b'L', b' ', b' '],
    ...:    [b' ', b'P', b'B', b' '],
    ...:    [b' ', b' ', b'M', b'W'],
    ...:    [b'W', b' ', b' ', b'B'],
    ...:    [b'F', b'J', b' ', b' '],
    ...:    [b' ', b'P', b'R', b' '],
    ...:    [b' ', b' ', b'A', b'K'],
    ...:    [b'G', b' ', b' ', b'P'],
    ...:    [b'F', b'S', b' ', b' '],
    ...:    [b' ', b'P', b'W', b' '],
    ...:    [b' ', b' ', b'A', b'P'],
    ...:    [b'A', b' ', b' ', b'Y'],
    ...:    [b'F', b'W', b' ', b' '],
    ...:    [b' ', b'P', b'P', b' '],
    ...:    [b' ', b' ', b'B', b'D'],
    ...:    [b'R', b' ', b' ', b'G'],
    ...:    [b'F', b'P', b' ', b' '],
    ...:    [b' ', b'P', b'G', b' '],
    ...:    [b' ', b' ', b'C', b'H'],
    ...:    [b'S', b' ', b' ', b'T'],
    ...:    [b'S', b'D', b' ', b' '],
    ...:    [b' ', b'F', b'A', b' '],
    ...:    [b' ', b' ', b'S', b'C'],
    ...:    [b'N', b' ', b' ', b'H'],
    ...:    [b'F', b'R', b' ', b' '],
    ...:    [b' ', b'W', b'C', b' '],
    ...:    [b' ', b' ', b'W', b'W'],
    ...:    [b'Y', b' ', b' ', b'I'],
    ...:    [b'C', b'R', b' ', b' '],
    ...:    [b' ', b'C', b'W', b' '],
    ...:    [b' ', b' ', b'O', b' '],
    ...:    [b'I', b' ', b' ', b' '],
    ...:    [b'C', b'W', b' ', b' '],
    ...:    [b' ', b'H', b'W', b' '],
    ...:    [b' ', b' ', b'J', b'O'],
    ...:    [b'E', b' ', b' ', b'T'],
    ...:    [b'S', b'M', b' ', b' '],
    ...:    [b' ', b'H', b'L', b' ']],
    ...:   dtype='|S1')

I can get an array that looks like what you want with:

In [12]: s.T.reshape(-1, 4).view('S4')
Out[12]: 
array([[b'SF  '],
       [b'WF  '],
       [b'GF  '],
       [b'AF  '],
       [b'RF  '],
       [b'SS  '],
       [b'NF  '],
       [b'YC  '],
       [b'IC  '],
       [b'ES  '],
       [b'LP  '],
       [b'JP  '],
       [b'SP  '],
       [b'WP  '],
       [b'PP  '],
       [b'DF  '],
       [b'RW  '],
       [b'RC  '],
       [b'WH  '],
       [b'MH  '],
       [b'BM  '],
       [b'RA  '],
       [b'WA  '],
       [b'PB  '],
       [b'GC  '],
       [b'AS  '],
       [b'CW  '],
       [b'WO  '],
       [b'WJ  '],
       [b'LL  '],
       [b'WB  '],
       [b'KP  '],
       [b'PY  '],
       [b'DG  '],
       [b'HT  '],
       [b'CH  '],
       [b'WI  '],
       [b'    '],
       [b'OT  ']], 
      dtype='|S4')

Note that the data type is 'S4', to match the declared size of the Fortran array.

That result leaves a trivial second dimension, so you might want to convert it to a one-dimensional array, e.g.

In [22]: s.T.reshape(-1, 4).view('S4')[:,0]
Out[22]: 
array([b'SF  ', b'WF  ', b'GF  ', b'AF  ', b'RF  ', b'SS  ', b'NF  ',
       b'YC  ', b'IC  ', b'ES  ', b'LP  ', b'JP  ', b'SP  ', b'WP  ',
       b'PP  ', b'DF  ', b'RW  ', b'RC  ', b'WH  ', b'MH  ', b'BM  ',
       b'RA  ', b'WA  ', b'PB  ', b'GC  ', b'AS  ', b'CW  ', b'WO  ',
       b'WJ  ', b'LL  ', b'WB  ', b'KP  ', b'PY  ', b'DG  ', b'HT  ',
       b'CH  ', b'WI  ', b'    ', b'OT  '], 
      dtype='|S4')

Perfect. The transpose is the key I was missing. Shortly after posting I figured out how to do it with the stride_tricks module, but this is much better IMHO.

tharen · Accepted Answer · 2016-10-05 01:31:13Z

0

For completeness I'll include this alternative solution. Same results as @Warren Weckesser, but requires an additional import.

from numpy.lib import stride_tricks

spp = stride_tricks.as_strided(jsp, strides=(jsp.shape[1],1))

# View as S4 and strip whitespace
spp = np.char.strip(spp.view('S4'))

answered Oct 5, 2016 at 1:31

tharen

1,32112 silver badges22 bronze badges

Collectives™ on Stack Overflow

Handle Fortran Character Arrays from Python with F2PY

2 Answers 2

1 Comment

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

Comments

Your Answer

Sign up or log in

Post as a guest

Related