I need to create JSON objects from clojure maps that store things like clojure vars. The base implementation throws this kind of error when it sees them:
java.lang.Exception: Don't know how to write JSON of class clojure.lang.Var
Can anybody point me to sample code on how to extend the capabilities of the JSON writer?
Thanks.
Well, I figured out the answer. There's another SO question that answers it partially: How to map clojure code to and from JSON?
But here's the code that worked for me:
(defn- write-json-clojure-lang-var [x #^PrintWriter out]
(.print out (json-str (str x))))
(extend clojure.lang.Var clojure.contrib.json/Write-JSON
{:write-json write-json-clojure-lang-var})
Note that all I wanted to do is just render a string version of which Var I'm referring to. You could, of course, do many other things...
An update to the answer from zippy for those of us using the newer clojure.data.json. This is code that will work with the updated/new library:
(defn- write-json-clojure-lang-var [x #^PrintWriter out]
(.print out (json-str (str x))))
(extend clojure.lang.Var clojure.data.json/JSONWriter
{:-write write-json-clojure-lang-var})
Related
I am using Common Lisp, SBCL, Emacs, and Slime.
In SLIME's REPL, I have a variable holding the file location:
CL-USER> json-file
#P"/home/pedro/lisp/json-example.json"
The content of the file is:
{"keyOne": "valueOne"}
I would like to read the data inside the file and return a string:
"{"keyOne": "valueOne"}"
How can I do it? Is it possible to do it with cl-json famous library?
The documentation was terse/hard. It is not good on providing examples. I couldn't find how to do it.
From what I read from the documentation, the API is built around streams. The function json:decode-json is taking a stream in parameter and return an association list which is very convenient to use.
To extract a value from the key, you can use the function (assoc :key-1 assoc-list). It will return a cons with (key . value). To get the value, you need to use the cdr function.
(defparameter json-string "{\"key-1\": \"value-1\"}")
(with-input-from-string (json-stream json-string)
(let ((lisp-data (json:decode-json json-stream)))
(cdr (assoc :key-1 lisp-data))))
Obviously, if you have the data in a file you could directly use the stream:
(with-open-file (json-stream "myfile.json" :direction :input)
(let ((lisp-data (json:decode-json json-stream)))
(cdr (assoc :key-1 lisp-data))))
Edit due to the comment of OP
The content of the file is:
{"keyOne": "valueOne"}
I would like to read the data inside the file and return a string:
"{"keyOne": "valueOne"}"
This question seems completely unrelated to the JSON library, but anyhow, if one need to open a file and put its content into a string, he can use a function from uiop.
* (uiop:read-file-string "test.txt")
"{\"keyOne\": \"valueOne\"}"
uiop is a library shipped with ASDF, so it's probably available to most Common Lisp's distributions. It's a kind of de-facto standard library and have a lot of interesting functions.
I have done the test on my computer, it's seems to work. I can certify that no data was harmed during that test.
I want to jsonify the results of a query performed against a Postgres table containing a column of type text[], but the problem is that clojure.data.json.write-str doesn't seem to know how to handle PG arrays:
Exception Don't know how to write JSON of class org.postgresql.jdbc4.Jdbc4Array clojure.data.json/write-generic
Do I have to supply a custom handler, or is there a simpler way?
Just to wrap up, here's what works for me, putting together Jared's answer with the latest clojure.java.jdbc API changes (0.3.0-alpha5 at the time of writing) which deprecate some patterns that are commonly used (e.g. with-query-results).
First define additional handlers for every unsupported types you care about, for instance:
(extend-type org.postgresql.jdbc4.Jdbc4Array
json/JSONWriter
(-write [o out]
(json/-write (.getArray o) out)))
(extend-type java.sql.Timestamp
json/JSONWriter
(-write [date out]
(json/-write (str date) out)))
Then use with-open to perform the JSON encoding while the connection is still open (which is important at least for org.postgresql.jdbc4.Jdbc4Array):
(with-open [conn (jdbc/get-connection db)]
(json/write-str
(jdbc/query {:connection conn}
["SELECT * FROM ..."]))))
I guess this will change as the clojure.java.jdbc API evolves, but for the moment, it works.
It is better to extend IResultSetReadColumn because you are not
required to stay in the scope of with-open
Something along the following is also more concise:
(extend-protocol jdbc/IResultSetReadColumn
org.postgresql.jdbc4.Jdbc4Array
(result-set-read-column [pgobj metadata idx]
(.getArray pgobj)))
Make sure the namespace where this is placed-in is required!
cheshire is good about handling various data types and being extensible (see add-encoder)
It looks like the issue is org.postgresql.jdbc4.Jdbc4Array does not implement java.util.Collection. Try calling getArray before you serialize it.
Edit:
If it is a nested structure, then it might be best to implement a handler. clojure.data.json uses the JSONWriter protocol. You can try something like the following:
;; Note: Pseudo Code
(extend-type org.postgresql.jdbc4.Jdbc4Array
clojure.data.json/JSONWriter
(-write [o out] (clojure.data.json/-write (.getArray o) out)))
Suppose I have a record that is "function-like", at least in the sense that it represents an operation that could be applied to some arguments.
I can make it work as a function by implementing clojure.lang.IFn, something like:
(defrecord Func [f x]
clojure.lang.IFn
(invoke [this arg]
(f x arg))
(applyTo [this args]
(apply f x args)))
((->Func + 7) 1)
=> 8
(yes I know that I've just reimplemented an inferior version of partial.... it's just an example :-) )
Is making a record implement clojure.lang.IFn a good practice or not?
Any pitfalls to this approach?
I'm surprised it doesn't already. Records are supposed to be "a complete implementation of a persistent map". So to answer your question, I'd expect it to be a function of its keys, as a map is; anything else would be quite surprising.
I can not give a direct Yes/No answer, but I can share my experience.
I defined a record implemented clojure.lang.IFn. Implementing IFn was to let me test it through REPL environment easily.
That record was intended to be a Job class, which was going to be processed by a worker. Therefore, I also implemented another interface java.lang.Runnable and a run function.
When I really put the code into integration test, it threw exception. Why?
The worker logic was something like this:
It checked if the Job class is a Callable instance, if so, invoke the call function.
It checked if the Job class is a Runnable instance, if so, invoke the run function.
However, clojure.lang.IFn has already extended Callable and Runnable, so the exception raised because I forgot to implement the call function.
can i somehow find all functions/macros that take a specific type of parameter ?
for example, what function accepts a Namespace object as parameter ?
(this is because i can create a namespace and store it in a var, but i don't know where i could use that var; what function might i pass that var to ?)
here is some code:
user=> (def working-namespace (create-ns 'my-namespace))
#'user/working-namespace
;i created a namspace and want to use it later
user=> (class working-namespace)
clojure.lang.Namespace
; out of curiosity i found out that "working-namespace" is a Namespace object
user=> (ns working-namespace)
nil
working-namespace=>
; but how do i switch to it ? this didn't do what i wanted...
user=> (refer working-namespace)
java.lang.ClassCastException: clojure.lang.Namespace cannot be cast to clojure.lang.Symbol (NO_SOURCE_FILE:0)
; this did not work either as my expectations
user=> (the-ns working-namespace)
#<Namespace my-namespace>
user=> (class (the-ns working-namespace))
clojure.lang.Namespace
; great, this gave me the same thing, a Namespace
hence the question: how do i use it dynamically (that's why i had put my namespace into a var) ? how do i get something useful for me from a var that points to a namespace ?
i can try look around for functions that make use of a Namespace object or that convert it to something else. i did and only found "intern". searching by hand not seems not that promising
what if i have this problem a million time ? is there an automated way to get me what i'm looking for without having to ask around each time ?
In Clojure 1.2 and previous function arguments dont have types. every function argument is an object. So the question really becomes "how do i find functions that will cast the object I pass them into this type. so searching for type hints will find some of them, though it wont get you everything. I wish it where more possible to answer this in general.
starting with 1.3 (current dev branch 9/2010) function paramerters and return types can have a defined type and will be passed/returned as that type instead of being cast to object and then cast on the other side. This drops one of the zeros from the exacution time of numerical functions with the important limitation that it only works for :static functions and only with direct calls (ie: not through map/reduce/filter/etc.) There is not a lot published on this change yet though it has the important breaking change that integers are no longer boxed by default and integer (actually Long) overflow throws an exception. you can read more here
(defn ^:static fib ^long [^long n]
(if (<= n 1)
1
(+ (fib (dec n)) (fib (- n 2)))))
so after 1.3 is released and widely adopted you will see code with more commonly defined types because they will offer a big speed benefit and then you will be able to find more functions by argument type though still not all of them.
At the same lecture where I learned about function argument types, Rich mentioned plans in the distant Clojure future (after 'Clojure in Clojure') about better support for exposing the compiler internals to tools such as IDEs. So there is hope that someday you will get a real answer to this question.
Dynamic languages make this slightly more difficult in practice and a lot harder in theory.
You already got a good answer from Arthur, so I'll only answer the "how do i get something useful for me from a var that points to a namespace ?". From (doc ns), note that it says unevaluated:
user=> (doc ns)
-------------------------
clojure.core/ns
([name docstring? attr-map? references*])
Macro
Sets *ns* to the namespace named by name (unevaluated), creating it
Now there's something you could do with in-ns if you want (the whole namespace object -> string -> symbol conversion is probably stupid, but enough to illustrate my point):
user=> (in-ns (symbol (str working-namespace)))
#<Namespace my-namespace>
my-namespace=>
I don't think you can do that without a lot of hackery in a dynamic language. If you want to now what function that take namespaces look at the documentation of the namespace stuff.
For example cleaning namespaces or reload them.
You wrote:
user=> (ns working-namespace)
nil
working-namespace=>
; but how do i switch to it ? this didn't do what i wanted...
But you did switch to the working-namespace namespace (that's why the prompt changed), so I'm not clear as to what "you wanted".
As I noted earlier, you need to present the ultimate problem are you trying to solve. It's entirely likely that messing with namespace objects won't be the solution.
I have a crazy idea, which involves putting some clojure code into CouchDB and writing views that query it. I don't want to store the clojure code as plain text, because then I would have to worry about parsing it in the views. Formatting and comments don't need to be preserved, but the code should be able to go in and out of the database without changing in structure. Keywords, symbols, and strings should all stay in their native type. Additionally, I want the code to look elegant and be efficient.
I'm thinking of representing things as follows:
Symbols as strings that start with '
Keywords as strings that start with :
Strings unmodified, except when they start with ' or :, in which case they're escaped with a backslash.
(parens) as an array
[brackets] as an array with "_[]" as the first element
maps ({}) as an object
sets (#{}) as an object with the values set to 1 and "_#{}" included.
Critiques, experiences, and ideas are appreciated.
Edit: Here's what happens if I try reading and writing JSON code using json functions from clojure.contrib:
user> code
((ns bz.json.app (:use (ring.middleware file))) (defn hello [req] {:status 200, :headers {"Content-Type" "text/plain"}, :body "Hello World!"}) (def app (wrap-file hello "public")))
user> (read-json (json-str code))
[["ns" "bz.json.app" ["use" ["ring.middleware" "file"]]] ["defn" "hello" ["req"] {"body" "Hello World!", "headers" {"Content-Type" "text/plain"}, "status" 200}] ["def" "app" ["wrap-file" "hello" "public"]]]
There's a fair bit that needs to be done for line 4 of the above to be exactly like line 2. It appears that it's a library project, unless there's a function somewhere that does it that I don't know about.
With such a library, here's what calling it might look like:
user> (= (json-to-code (read-json (json-str (code-to-json code)))) code)
true
As mikera suggested, clojure.contrib.json/write-json will convert not only primitive types, but Clojure's ISeqs and Java's Maps, Collections and Arrays, too. This should cover most of your code (seen as data), but in the event you want to write anything fancier, it's easy to extend the JSON writer, by mimicking out Stuart Sierra's source code (see here):
(defn- write-json-fancy-type [x #^PrintWriter out]
(write-json-string (str x) out)) ;; or something useful here!
(extend your-namespace.FancyType clojure.contrib.json/Write-JSON
{:write-json write-json-fancy-type})
This is assuming you don't need to store computed bytecode, or captured closures. This would be an entirely different game, significantly harder. But since most Clojure code (like most Lisp's) can be seen as a tree/forest of S-Expressions, you should be OK.
Parsing out JSON back to the data can be done with clojure.contrib.json/read-json (take a short time to look at the options on its definition, you may want to use them). After that, eval may be your best friend.
If you want to use JSON as a representation, I'd strongly suggest using clojure.contrib.json, which already does the job of converting Clojure data structures to JSON pretty seamlessly.
No point reinventing the wheel :-)
I've used it pretty successfully in my current Clojure project. If it doesn't do everything you want, then you can always contribute a patch to improve it!
I think your idea is sound, but I'd simplify the handling of collections by using tagged arrays (["list", …], ["vector", …]) instead. Apart from that, I wouldn't change the implementation strategy.
I like your idea and to code in Clojure, so I took a stab at implementing your code-to-json (with the above suggestion incorporated) at https://gist.github.com/3219854.
This is the output it generates:
(code-to-json example-code)
; => ["list" ["list" "'ns" "'bz.json.app" ["list" ":use" ["list" "'ring.middleware" "'file"]]] ["list" "'defn" "'hello" ["vector" "'req"] {":status" 200, ":headers" {"Content-Type" "text/plain"}, ":body" "Hello World!"}] ["list" "'def" "'app" ["list" "'wrap-file" "'hello" "public"]]]
json-to-code is left as an exercise for the reader. ;)
clojure.contrib.json has been superseded by clojure.data.json:
(require '[clojure.data.json :as json])
(json/write-str {:a 1 :b 2})
;;=> "{\"a\":1,\"b\":2}"
(json/read-str "{\"a\":1,\"b\":2}")
;;=> {"a" 1, "b" 2}
You might also like to use cheshire which has a nice API and support for various extensions such as custom encoding and SMILE (binary JSON):
(:require [cheshire.core :as json])
(json/encode {:a 1 :b 2})
;;=> "{\"a\":1,\"b\":2}"
(json/decode "{\"a\":1,\"b\":2}")
;;=> {"a" 1, "b" 2}
For completeness sake there is also clj-json which uses Jackson underneath to parse the JSON.
With version 0.1.2 of clojure.data.json the whole thing could look like this I guess:
(require ['clojure.data.json :as 'json])
(defn- json-write-date [s ^java.io.PrintWriter out escape-unicode?]
(.write out (str "\""
(.format (java.text.SimpleDateFormat. "yyyyMMddHHmmssZ") s) "\"")))
(extend java.util.Date clojure.data.json/Write-JSON {:write-json json-write-date})
(json/json-str { :example (java.util.Date.)})
"{\"example\":\"20120318182612+0100\"}"`