clojure core.async thread blocks

Question

I have the following code:

(defn -db-producer-factory [order-ids-chan next-chan]
  (thread
    (prn "db starting...")
    (while true
      (do
        (prn "db starting2...")
        ;;
        ;; issue spot!
        ;; it stays blocked here-- the orderid doesnt come off the chan
        ;;
        (let [order-id (<!! order-ids-chan)]
          (prn "db->" order-id)
          (condp = order-id
            :finished (>!! next-chan :finished)
            :>> (supress-w-nextexc
                  (->>
                    ; get denorm'd order
                    (-> (r/-get-order :live order-id)
                        denorm/order->denormalized)
                    ; put in a map to avoid nils
                    (hash-map :data)
                    (>!! next-chan)))))))))

(defn -stats-producer-factory [stats-db-chan next-chan]
  (thread
    (while true
      (do
        (let [msg (<!! stats-db-chan)
              data (:data msg)]
          (when data
            (do
              (prn "stats-> " (-> data :order :order-id))
              (supress-w-nextexc
                (q/stats-order-insert (-> data :order)))
              (supress-w-nextexc
                (q/stats-item-insert (-> data :items)))))
          (>!! next-chan msg))))))

(defn -do-orderids [orderids]
  (let [finished-chan (chan)
        order-ids-chan (chan)
        stats-db-chan (chan)
        db-producer (-db-producer-factory order-ids-chan stats-db-chan)
        stats-producer (-stats-producer-factory stats-db-chan finished-chan)]

    (prn "pre-pub")
    ;; pub ids and finished message
    (map #(>!! order-ids-chan %) (conj orderids :finished))
    ;; wait for finish
    (prn "finished? " (<!! finished-chan))
    ;; allow time for finishing
    ;(Thread/sleep 3000)
    ;; close all chans
    (map close! [finished-chan order-ids-chan stats-db-chan db-producer stats-producer])
    ))

The process is initiated via a call to -do-orderids like (-do-orderids [123]).

The output of the execution is:

"db starting..."
"pre-pub"
"db starting2..."

But then it blocks. Why doesn't it pass the orderid at the "issue spot"?

I haven't reviewed the whole thing, but it could be due to using (chan) which creates a zero-length queue. This means a put will block until a simultaneous read occurs. You may wish to make the queue non-zero length, perhaps (chan 9) for example. You could also use put! to have the put occur in a different thread (non-blocking). Also, you may with to use a go-loop rather than (while true ...) in a separate thread — Alan Thompson
– Alan Thompson, Commented Apr 29, 2016 at 21:46
So far i've noticed that if i change (map #(>!! order-ids-chan %) (conj orderids :finished)) to just (>!! order-ids-chan 123) it works... wondering why the map is part of the issue — Micah
– Micah, Commented Apr 29, 2016 at 21:52
Good catch. Map is lazy and won't actually perform the calculation until the result is requested. Try changing map to mapv (equiv to (vec (map ...))` — Alan Thompson
– Alan Thompson, Commented Apr 29, 2016 at 21:54
Why does (->> (conj orderids :finished) (map #(>!! order-ids-chan %))) also work? Does ->> force the result out of laziness? Also, thanks for introducing me to mapv, very happy to stop doing (doall (map)). @AlanThompson — Micah
– Micah, Commented Apr 29, 2016 at 22:05
using ->> should not make any difference, very surprised if you see a change from that alone. — Alan Thompson
– Alan Thompson, Commented Apr 29, 2016 at 22:26

glts · Accepted Answer · 2016-04-30 09:44:52Z

3

Your program blocks because the db-producer, who’s blocked waiting for an order ID, never actually receives an order ID on order-ids-chan.

That is because map is lazy. So, in this invocation

(map #(>!! order-ids-chan %) (conj orderids :finished))

the mapping function is never called and no order ID is ever put onto the channel.

Clojure rule of thumb:

Never use map for side-effects! Use run! instead.

I think substituting run! for map in that line (and on the last line where you call close! on the channel) should fix it.

Unrelated: all the do forms you used are redundant, you can safely remove them and reduce the level of nesting.

answered Apr 30, 2016 at 9:44

glts

23k12 gold badges80 silver badges94 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

clojure core.async thread blocks

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related