Timing analysis of Clojure code

Question

Hello Clojure experts,

I am trying to do some timing tests in Clojure 1.3 and I thought I'd ask a question based on an existing piece of code that solves a differential equation that is adapted from this blog post.

Code follows:

;; the differential equation is                                                                                            
;; dy/dt = f(y,t) = t - y                                                                                                  

(defn f [t y] (- t y))

;; solve using Euler's method                                                                                              
(defn solveEuler [t0 y0 h iter]
  (if (> iter 0)
    (let [t1 (+ t0 h)
          y1 (+ y0 (* h (f t0 y0)))]
      (recur t1 y1 h (dec iter)))
    [t0 y0 h iter]))

(defn multipleSolveEuler []
  (let [steps '(1 10 100 1000 10000 100000)
        results (map #(second (solveEuler 0.0 0.0 (/ 1.0 %) %)) steps)
        errors  (map #(- (Math/exp -1) %) results)]
    (partition 3 (interleave steps results errors))))

(def *cpuspeed* 2.0)

(defmacro cyclesperit [expr its]
  `(let [start# (. System (nanoTime))
         ret# ( ~@expr (/ 1.0 ~its) ~its )
         finish# (. System (nanoTime))]
     (int (/ (* *cpuspeed* (- finish# start#)) ~its))))

(defn solveEuler-2 [t0 y0 h its]
  (let [zero (int 0)]
    (loop [t0 (double t0), y0 (double y0), h (double h), its (int its)]
      (if (> its zero)
        (let [t1 (+ t0 h)
              y1 (+ y0 (* h (- t0 y0)))]
          (recur t1 y1 h (dec its)))
        [t0 y0 h its]))))

So when I say

(time solveItEuler-2 0.0 1.0 (/ 1.0 1000000000) 1000000000))

I get a time 6004.184 msecs on a 6 month old Macbook pro. I issue the command again, and I get around the same time. But when I run it 3 other times I get times in the range of 3500 msec. I have noticed this before for other code snippets, and was wondering why this is so. I suppose I would expect a roughly similar execution time across successive runs.

Is it my lack of understanding how "time" works, something I am missing, or some kind of caching happening under the hood?

Thanks.

i think that you should simplify your question and provide a much simpler code snippet. — viebel
– viebel, Commented Mar 10, 2012 at 17:31
Although my question wasn't really about a particular code snippet, but about the results of doing a timing analysis on the JVM, the example above has a "big enough" runtime to see dramatic differences - which is why I chose it. — endbegin
– endbegin, Commented Mar 11, 2012 at 19:53

Alex Taggart · Accepted Answer · 2012-03-09 20:19:02Z

7

It's not time that's relevant, but rather the JVM optimizing the code at runtime. What you observe is typical; execution time will drop and then stabilize after about 3 invocations.

answered Mar 9, 2012 at 20:19

Alex Taggart

7,83530 silver badges31 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

endbegin Over a year ago

I see. Therefore, in order to do a fairly accurate timing analysis, does one have to run a piece of code multiple times before saying something like "this code takes X secs to run"?

Alex Taggart Over a year ago

Precisely. This goes for all languages running on the JVM. Just keep running it until you get stable values.

viebel Over a year ago

could you please provide some material about this behavior of the JVM?

Gene T Over a year ago

@Yehonathan One ref. anoth. You want to search on "JVM warmup", "hotspot inlining", -XX:MaxInlineSize"

Brian · Accepted Answer · 2012-03-11 04:59:58Z

7

I recommend using the Criterium library for benchmarking. It is designed to deal with the pitfalls of benchmarking code that runs on the JVM.

answered Mar 11, 2012 at 4:59

Brian

1,82212 silver badges9 bronze badges

Comments

Gene T · Accepted Answer · 2012-03-12 10:48:00Z

2

(This is response to Viebel request for more info, which became too many chars for a single comment, This is not answer to orig. Q).

There are quite a few degrees of freedom: JVM GC and heap settings, startup time, number of warmup runs, data structures used, size of inlined methods.

http://www.infoq.com/articles/java-threading-optimizations-p2

http://groups.google.com/group/clojure/browse_thread/thread/776943086de213f9#

http://stas-blogspot.blogspot.com/2011/07/most-complete-list-of-xx-options-for.html

http://www.azulsystems.com/blog/cliff/2009-09-06-java-vs-c-performanceagain

edited Mar 12, 2012 at 10:48

answered Mar 12, 2012 at 10:27

Gene T

5,1761 gold badge27 silver badges25 bronze badges

Collectives™ on Stack Overflow

Timing analysis of Clojure code

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related