18

I have a complex Clojure data structure that I would like to serialize - basically the entire current game state for an online game I am developing so that I can implement save game files.

My requirements are:

  • Some form of human-readable text format (I'd probably prefer s-expressions, JSON and XML in that order but open to others)
  • Support all the usual Clojure data structures, keywords and primitives
  • Ability to provide custom serialization / deserialization functions for custom java classes, defrecords etc. (this is important because I need to do something like Java's readResolve in several cases)
  • Good performance is a nice-to-have

Any good recommendations?

Improve this question
3
  • Without knowing much about Clojure, is there a reason why this isn't achievable using the standard Java serialisation mechanisms invoked from Clojure? Commented Jul 21, 2010 at 16:30
  • @Gian - yes that's certainly possible, but I'm trying to learn the "Clojure way" of doing things :-) Commented Jul 21, 2010 at 16:54
  • IMO the Clojure way is to use Java's facilities where they provide good solutions to the problems they address. :-) Serializable may be a good solution for short-term storage / transfer of data structures. Having said that, I guess that for this use case a format better suited to long-term storage is needed and this might be provided by print-dup. (Serializable might run into problems if, say, the structure of the classes implementing the core Clojure data structures changes; print-dup likely won't.) Commented Jul 21, 2010 at 17:03

4 Answers 4

Reset to default
11

If you wanted to serialize things to S-expressions, you could use print-dup:

(binding [*print-dup* true] (println [1 2 3]))
; prints [1 2 3]

(defrecord Foo [x])
; => user.Foo
(binding [*print-dup* true] (println (Foo. :foo)))
; prints #=(user.Foo/create {:x :foo})

Note that printing a structure which holds, say, ten references to a single vector followed by reading it back gives you a datastructure with ten separate (not identical?), though equivalent in terms of structure (=) vectors.

To use this in cases where there is no default implementation provided, implement the multimethod clojure.core/print-dup.

Also, a lot of things in Clojure 1.2 are java.io.Serializable:

(every? (partial instance? java.io.Serializable)
        [{1 2} #{"asdf"} :foo 'foo (fn [] :foo)])
; => true

(defrecord Foo [])
(instance? java.io.Serializable (Foo.))
; => true

Note that you should avoid serializing runtime-created fns -- they are instances of one-off classes with weird names and you won't be able to deserialize them after restarting your JVM anyway. With AOT compilation, fns do get their own fixed classnames.

Update: As mentioned in a comment on the question, Serializable is best suited to short-term storage / transfer of data, whereas print-dup should be more robust as a long-term storage solution (working across many versions of the application, Clojure etc.). The reason is that print-dup doesn't in any way depend on the structure of the classes being serialized (so a vector print-dup'd today will still be readable when the vector implementation switches from Java to Clojure's deftype).

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I think that print-dup might be the superior solution for this use case, see my comment on the question... If the save games may become large, the printouts can always be compressed.
7

edn-format has now been released as a standard for data transfer using Clojure's data structures.

It is a pretty good fit for serialising Clojure data structures / values - and is supported across multiple languages so can also be used as a data interchange format.

Improve this answer

2 Comments

this works well for core Clojure data structures, but say you had a more exotic data structure in there like a PriorityMap (which looks just like a PersistentMap but behaves differently), then you'd need to use reader macros or something right ?
They have though of this in edn-format: you can "tag" values to indicate that they are of a special type and write your own handlers to construct the appropriate class. It's pretty celever!
6

If everything is a Clojure data structure, then it's already serialized (b/c of code<->data). Just dump the data structures onto disk. To restore, load them back and (eval).

Improve this answer

3 Comments

Bind *out* to a file stream and use (pr) (and then use (load-file) to read it back). See groups.google.com/group/clojure/browse_thread/thread/… for a working example.
Isn't evaluating them dangerous? Someone might sneak some code in there. They should be read but not evaluated I believe.
Well that's a concern with any serialization. If not evaluated, then you have to add a whole read/parse layer. If this is a situation where an untrusted user has access to the file, then I think I'd prefer to extend this a bit and put in an encryption layer or a hash.
3

for JSON you can use standard clojure-contrib.json. Although, as I remember, all Clojure objects should be serializable...

Improve this answer

3 Comments

Tried this, it doesn't like my Java classes: it throws e.g. java.lang.Exception: Don't know how to write JSON of class mikera.persistent.SparseMap. Is there any way to give it custom serialisation functions for your own classes?
You could have your class implement the clojure.contrib.json.Write_JSON interface (or the protocol clojure.contrib.json/Write-JSON, if the class is a Clojure record / type). Not sure how you'd go about reading them back, though; I think for non-standard data structures you'd want to use something like YAML. A quick Google search finds clj-yaml github.com/lancepantz/clj-yaml as a possible solution, though I don't really know anything about the project.
Ah, just noticed that clj-yaml's README says it only supports deserialization for now...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.