-1

I need to be able to call Python functions from Java, and get the return of those functions, which should be primitives only.
I also don't need to access Java objects in Python or other stuff like that.
But, most of all, I need speed. I need to be able to load the script at the start of my application and then call some functions at different time-intervals, which can sometimes get really tiny (30-40 times per second).
That's the main-reason why I don't think that Jython will work for me: It has all kinds of stuff that I don't need, and is incredibly slow.

So, those are the constraints/requirements I have:

  • Needs to be fast
  • I need to call python functions and get their result
  • The script should be loaded only once

Do you guys know about another way to simply call Python functions in a pre-loaded script (So that the script does not have to be loaded again and again for each function call)?

(PS: I do not think that this is a duplicate as I searched quite a lot and all links I found did not answer my question (For example Calling Python in Java?) )

6
  • Make a ReST call to a python service? Commented Oct 22, 2015 at 12:54
  • @ElliottFrisch Uh, what exactly do you mean? Let the Python scripts run as some kinds of services? The question would still be how do I actually access the function from Java, etc. And if you mean an online service, that wont work at all. I need the fastest way possible and thats probably the slowest. Commented Oct 22, 2015 at 12:58
  • I take your point, thrift is so slow that facebook is built on it. I'm confident your solution will scale better than that. Commented Oct 22, 2015 at 13:08
  • @ElliottFrisch Didn't know that. Also I thought you meant a "real" "online" service, which would be way too slow (Anything that requires internet access won't work for me as the application will need to work on offline machines.) - But you mean something like emulating a network on the machine itself, right? Sorry if I sound too stupid Commented Oct 22, 2015 at 13:14
  • Local sockets are pretty fast. A local cluster can handle scaling. The real question is what you want to optimize for (throughput or concurrency). Either way, using an n-tier design would be my strategy. Commented Oct 22, 2015 at 13:16

2 Answers 2

1

You could write some JNI code to embed the CPython interpreter.

You could write a Python program that will run as a daemon (or web service or whatever) and connect to it via some sort of networking. I am intentionally vague here since it is such a broad question with any number of possibilities.

I don't know that either of the above will actually be faster than calling in to Jython. Note that you can load the Jython interpreter once and (re)use it for each function call.

What if you rewrote your Python function in Java? You can even load it dynamically, if that is the reason you are trying to combine multiple languages. Alternatively, Java8 comes with a (new) JavaScript engine. You could use JavaScript instead of Python.

Sign up to request clarification or add additional context in comments.

7 Comments

Thanks for the answer. I guess I will have to implement each of these methods and do some benchmarking... Rewriting the function in Java isn't an option, as I am not the author of those functions, and the people programming them are no programmers: I figured Python as the way to go because it is easy to learn and has a lot of modules. (My application basically is some huge backbone for a few scripts)
@GandalfSchmidt then why don't you write the backbone in Python? Why add an extra layer of complexity by trying to jam two languages together?
@jonrsharpe Well, Python is more of a scripting language and there is some complexity involved in the application that I really don't want to write in Python. I also think that Python is too slow to run the whole application (The Java application is only a prototype since it is faster to write than in C++ - But once I got it working I will port it to C++)
@GandalfSchmidt you're just a glutton for punishment, huh? If you want a language that's fast to write a prototype in, why on earth choose Java?! Or just go straight to C++, which is easier to interoperate with Python than Java is.
@jonrsharpe Uh, well, personally I find Java easier to write a prototype in than C++, and I used to use Java a lot back then so I thought it would be a good choice. Also I know (and used before) some library in Java that handles a huge part of work for me which I would be obligated to write myself in C++, thats why I wanted it to be written in java first. But I guess you're right, I should directly use C++..
|
0

There are a bunch of Python/Java bridges which allow you to do this:

  • JEP embeds CPython in Java through JNI allowing Java to call down into Python. It's purely in-process.

  • jpy is another in-process implementation. One of its key features is support for fast pass-by-value operations with arrays by use of pointer hand-off.

  • jpype is another in-process implementation. Since it also uses internal C-based handoff it's highly performant.

  • PyJNIus is another implementation which accesses Java classes using JNI.

I am a maintainer of another one: PJRmi. For this case, the PJRmi version of the code might look something like this:

import com.deshaw.pjrmi.PythonMinion;
import com.deshaw.pjrmi.PythonMinionProvider;
import java.util.Arrays;

public class PythonFunctionRunner
{
    public static void main(String... args)
        throws Exception
    {
        // Create the Python instance
        final PythonMinion python = PythonMinionProvider.spawn();

        // Pretend that we read in this Python code from a file on disk
        final String code =
            "import numpy\n" +
            "def my_sum(array):\n" +
            "    return numpy.sum(array)\n"
            ;

        // Push it into the interpreter
        python.exec(code);

        // Create an argument to give it
        final double[] array = new double[10];
        for (int i=0; i < array.length; i++) {
            array[i] = i;
        }

        // Call the method on the array, seeing how fast it goes
        final int count = 10000;
        final long start = System.nanoTime();
        double result = 0;
        for (int i=0; i < count; i++) {
            result = python.invoke("my_sum", double.class, array);
        }
        final long end = System.nanoTime();
        final long micros = (end - start) / 1000;
        System.out.println("my_sum(" + Arrays.toString(array) + ") = " + result);
        System.out.println("Handled " + (count * 1_000_000 / micros) + " calls per second");
    }
}

which would yield something like:

$ java PythonFunctionRunner
my_sum([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0]) = 45.0
Handled 109 calls per second

In my limited experience I have found the other bridges to be similar in nature. Which one is best for you likely depends on exactly what you want.

1 Comment

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.