Sending a fixed length array from java to python through a socket

Question

I have a python server who's listening to a java client.

I want to send an array from java to python and respond by sending an array back.

I am listening from python like this:

INPUT_SPEC = {'rows': 5, 'columns': 2, 'dtype': np.dtype('float32')}
OUTPUT_SPEC = {'rows': 1, 'columns': 1, 'dtype': np.dtype('float32')}

while True:
    # Wait for a connection
    print >>sys.stderr, 'waiting for a connection'
    connection, client_address = sock.accept()
    nbytes = INPUT_SPEC['rows'] * INPUT_SPEC['columns'] * INPUT_SPEC['dtype'].itemsize
    try:
        print >>sys.stderr, 'connection from', client_address
data = connection.recv(nbytes)



 a = np.fromstring(data, dtype = INPUT_SPEC['dtype']).reshape((INPUT_SPEC['rows'], INPUT_SPEC['columns']))
    result = np.array(mysum(a), dtype = OUTPUT_SPEC['dtype']).reshape((OUTPUT_SPEC['rows'], OUTPUT_SPEC['columns']))
    print >>sys.stderr, 'sending data back to the client: {0}'.format(result)
    connection.sendall(result.tobytes())



except Exception as e:
    print(e)
    connection.close()
finally:
    # Clean up the connection
    connection.close()

And I am sending from java like this:

String hostName = "localhost";
        int portNumber = 10000;

        try (

                //open a socket
                Socket clientSocket = new Socket(hostName, portNumber);


                BufferedReader in =  new BufferedReader( new InputStreamReader(clientSocket.getInputStream()));
                PrintWriter out =  new PrintWriter(clientSocket.getOutputStream(), true);

        ) {

            System.out.println("Connected");

            Double[][] test2 =   new Double[5][2];
            test2[1][1]= 0.1;
            test2[1][0]= 0.2;
            test2[2][1]= 0.2;
            test2[2][0]= 0.2;
            test2[3][1]= 0.1;
            test2[3][0]= 0.2;
            test2[4][1]= 0.2;
            test2[4][0]= 0.2;
            test2[0][1]= 0.2;
            test2[0][0]= 0.2;



            out.println(test2);

            String response;
            while ((response = in.readLine()) != null)
            {
                System.out.println( response );
            }


        }

The error message I get is:

string size must be a multiple of element size

So I must not be defining my sent data well. Any suggestions?

mhawke · Accepted Answer · 2014-12-03 14:30:44Z

1

Your java code is sending this over the socket:

[[Ljava.lang.Double;@3be74fe9

which is basically what you get if you print an array variable. You could try converting test2 to a string using java.util.Arrays.deepToString(test2) which will produce a string representation of test2:

[[0.2, 0.2], [0.2, 0.1], [0.2, 0.2], [0.2, 0.1], [0.2, 0.2]]

and then send that to Python, however, you then have to parse it into a numpy array and it's not going to adapt well to potential changes in client software (e.g. another language).

You should consider a more general way of serialising the data using something like JSON. This will avoid pitfalls such as data type sizes, endianness, formatting, ordering, etc. It will also make it trivial to deserialise in Python and you wont need to nail up the array dimensions in Python, e.g.

import json
import numpy as np
data = connection.recv(nbytes)    # N.B. you may need multiple reads to get all of the data.
a = np.array(json.loads(data))

Coincidentally, in this case, the output of deepToString(test2) is compatible with JSON, but that might not always be so. Better to go for JSON proper using a Java JSON library such as http://www.json.org/java/index.html. Your Java code would be:

import org.json.*;
.
.
.
Double[][] test2 = new Double[5][2];
test2[1][1]= 0.1;
test2[1][0]= 0.2;
test2[2][1]= 0.2;
test2[2][0]= 0.2;
test2[3][1]= 0.1;
test2[3][0]= 0.2;
test2[4][1]= 0.2;
test2[4][0]= 0.2;
test2[0][1]= 0.2;
test2[0][0]= 0.2;

out.println(new JSONArray(test2).toString());

Python will receive the JSON string [[0.2,0.2],[0.2,0.1],[0.2,0.2],[0.2,0.1],[0.2,0.2]] which can then be parsed via np.array(json.loads(data)) to:

array([[ 0.2,  0.2],
       [ 0.2,  0.1],
       [ 0.2,  0.2],
       [ 0.2,  0.1],
       [ 0.2,  0.2]])

answered Dec 3, 2014 at 14:30

mhawke

87.5k10 gold badges122 silver badges142 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

dorien Over a year ago

Thanks so much for the explanation and suggestion. I have tried this and it outputs me "Expecting object: line 1 column 40 (char 39)". Is this because it does not read the entire json?

dorien Over a year ago

Oh indeed, because print(data) gives: [[0.2,0.2],[0.2,0.1],[0.2,0.2],[0.2,0.1]. Any suggestions on how to read more? Until a null byte or are there different ways?

dorien Over a year ago

Ok, almost there, I found it via while True: new_data = connection.recv(16) if new_data: data += new_data else: data += new_data[:] print >>sys.stderr, 'no more data from', client_address break data[:-1]---only this is not recognised as a json object. But prints as a string already!

dorien Over a year ago

Worked when removing an enter data= data.strip('\n'); Thanks a million!

Collectives™ on Stack Overflow

Sending a fixed length array from java to python through a socket

1 Answer 1

4 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

4 Comments

Your Answer

Sign up or log in

Post as a guest

Related