I want to jsonify the results of a query performed against a Postgres table containing a column of type text[], but the problem is that clojure.data.json.write-str doesn't seem to know how to handle PG arrays:
Exception Don't know how to write JSON of class org.postgresql.jdbc4.Jdbc4Array clojure.data.json/write-generic
Do I have to supply a custom handler, or is there a simpler way?
Just to wrap up, here's what works for me, putting together Jared's answer with the latest clojure.java.jdbc API changes (0.3.0-alpha5 at the time of writing) which deprecate some patterns that are commonly used (e.g. with-query-results).
First define additional handlers for every unsupported types you care about, for instance:
(extend-type org.postgresql.jdbc4.Jdbc4Array
json/JSONWriter
(-write [o out]
(json/-write (.getArray o) out)))
(extend-type java.sql.Timestamp
json/JSONWriter
(-write [date out]
(json/-write (str date) out)))
Then use with-open to perform the JSON encoding while the connection is still open (which is important at least for org.postgresql.jdbc4.Jdbc4Array):
(with-open [conn (jdbc/get-connection db)]
(json/write-str
(jdbc/query {:connection conn}
["SELECT * FROM ..."]))))
I guess this will change as the clojure.java.jdbc API evolves, but for the moment, it works.
It is better to extend IResultSetReadColumn because you are not
required to stay in the scope of with-open
Something along the following is also more concise:
(extend-protocol jdbc/IResultSetReadColumn
org.postgresql.jdbc4.Jdbc4Array
(result-set-read-column [pgobj metadata idx]
(.getArray pgobj)))
Make sure the namespace where this is placed-in is required!
cheshire is good about handling various data types and being extensible (see add-encoder)
It looks like the issue is org.postgresql.jdbc4.Jdbc4Array does not implement java.util.Collection. Try calling getArray before you serialize it.
Edit:
If it is a nested structure, then it might be best to implement a handler. clojure.data.json uses the JSONWriter protocol. You can try something like the following:
;; Note: Pseudo Code
(extend-type org.postgresql.jdbc4.Jdbc4Array
clojure.data.json/JSONWriter
(-write [o out] (clojure.data.json/-write (.getArray o) out)))
Related
My problem requires applying custom logical functions to a structure. Those functions are stored in a database as a string. I have data like this:
(def fruits {:apple {:color "red" :ripe? true}
:strawberry {:color "red" :ripe? false}})
And I have this cond check:
"(some (fn [fruit] (-> fruit val :ripe? false?)) fruits)"
Unfortunatelly I can't get this right even though I tried various approaches:
1)
(cljs/eval-str (cljs/empty-state)
"(some (fn [fruit] (-> fruit val :ripe? false?)) my.main/fruits)"
""
{:eval cljs/js-eval}
identity)
This works yet it yields errors:
WARNING: No such namespace: my.main, could not locate my/main.cljs, my/main.cljc, or Closure namespace "" at line 1
WARNING: Use of undeclared Var my.main/fruits at line 1
Also this approach obviously wouldn't work in advanced compilation.
2) I tried to leverage approach that works in Clojure:
((eval
(read-string
"(fn [fruits]
(some (fn [fruit] (-> fruit val :ripe? false?)) fruits))"))
fruits)
I can't see why this wouldn't work in advanced compilation. Unfortunatelly it simply returns nil every single time.
Is it just me who fails to come up with a solution or is CLJS just not capable of doing that yet?
I suspect your going to have a vary hard time achieving your requirement using
this approach. The big problem is likely going to be due to the way
clojurescirpt needs to be compiled into javascript (using Google closure). You
can probably get it to work doing some clever stuff with externals and using low
level javascript interop and the closure library, but I suspect it will be hard
work.
A couple of alternative approaches which may be worth considering
Store the data in the database in edn format. Using edn, you may be able to
read it into a var and then execute it
Change direction - do you really need to store complete functions or could you
instead define a type of DSL which obtains parameters from the database which
will provide the necessary level of dynamic execution.
Could you have some sort of pre-processing solution i.e. write the function in
clojurescript, but use closure functionality to compile that to javascript and
insert that into the database instead of the raw clojurescript. This would
make the initial storing of the data more complex, but may simplify calling
the dynamic functions at runtime. You could even include some code checks or
validation to reduce the likelihood of code taken from the database doing the
wrong thing.
There are so many risks associated with using totally dynamic code it is almost
never a good solution. Aside from the numerous security issues you have with
this approach, you also have to deal with elegantly handling problems arising
from buggy definitions being inserted into the database (i.e. a buggy function
definition which crashes your app or corrupts data. If you just have to have the
ability to dynamically execute unknown code, then at least edn provides some
additional protection you don't get with eval-str - but really, just don't do
it.
After hours of experiments and struggling with evaluating functions from strings I decided to write DSL.
In database I store string with a map containing these parameters:
:where? - vector containing path to a desired answer.
:what? - answer(s) I'm looking for.
:strict? (optional) - A boolean. If true then answers need to be exactly the same as :what? rule (order doesn’t matter).
Then I just evaluate that simple cljs file. It works both on advanced and none optimization mode.
(defn standard-cond-met? [{:keys [what? where? strict?]
:or {strict? false}}]
(let [answer (get-in answers (conj where? :values))]
(if strict?
(= (sort what?) (sort answer))
(clojure.set/subset?
(set what?)
(set answer)))))
I have a web service written in Clojure. It has a simple GET method implemented, which returns back a JSON object representing the router's current position and different time counters.
My code has a bunch of atoms to keep track of time. Each atom represents different activities a machine might be doing at a given time. For example: calibrating, idle, stuck, or working:
(def idle-time (atom 0))
(def working-time (atom 0))
(def stuck-time (atom 0))
(def calibration-time (atom 0))
Towards the end I have a loop that updates the position and time counters every 15 seconds:
(defn update-machine-info []
(let [machine-info (parse-data-files)]
(update-time-counters machine-info)
(reset! new-state (merge machine-info
{:idleCounter #idle-time
:workingCounter #working-time
:stuckCounter #stuck-time
:calibrationCounter #calibration-time}))))
(loop []
(future
(Thread/sleep 15000)
(update-machine-info)
(recur)))
Currently this code runs into race condition, meaning the position and time counters are not updating. However, the Web Service still responses back a proper JSON response, albeit with old values.
Web Service is using Cheshire to generate map into JSON, here my GET implementation:
(defroutes app-routes
(GET "/" [] (resource :available-media-types ["application/json"]
:handle-ok (generate-string (get-machine-information))))
(route/not-found "Not Found"))
Should I be using refs instead of atoms? Am I using future correctly? Is (Thread/sleep 15000) causing the issue, because atoms are async?
Please let me know if you see an obvious bug in my code.
I don't think you can reliably recur inside a future to a loop that's outside the future (not completely sure), but why not try something like this instead?
(future
(loop []
(Thread/sleep 15000)
(update-machine-info)
(recur)))
That way loop/recur stays within the same thread.
Other than that, it's possible that if update-machine-counters throws an exception the loop will stop, and you'll never see the exception because the future is never dereferenced. An agent ( http://clojure.org/agents ) might be better suited for this, since you can register an error handler.
I think what is happening is that the process where you call your futures is terminating before your futures actually execute. For what your doing, futures are probably the wrong type of construct. I also don't think your loop future recor sequence is doing what you think.
There is a lot of guesswork here as it isn't clear exactly where you are actually defining and calling your code. I think you probably want to use something like agents, which you need to setup in the root process and then send a message to them in your handler before you return your response.
Suppose I have a record that is "function-like", at least in the sense that it represents an operation that could be applied to some arguments.
I can make it work as a function by implementing clojure.lang.IFn, something like:
(defrecord Func [f x]
clojure.lang.IFn
(invoke [this arg]
(f x arg))
(applyTo [this args]
(apply f x args)))
((->Func + 7) 1)
=> 8
(yes I know that I've just reimplemented an inferior version of partial.... it's just an example :-) )
Is making a record implement clojure.lang.IFn a good practice or not?
Any pitfalls to this approach?
I'm surprised it doesn't already. Records are supposed to be "a complete implementation of a persistent map". So to answer your question, I'd expect it to be a function of its keys, as a map is; anything else would be quite surprising.
I can not give a direct Yes/No answer, but I can share my experience.
I defined a record implemented clojure.lang.IFn. Implementing IFn was to let me test it through REPL environment easily.
That record was intended to be a Job class, which was going to be processed by a worker. Therefore, I also implemented another interface java.lang.Runnable and a run function.
When I really put the code into integration test, it threw exception. Why?
The worker logic was something like this:
It checked if the Job class is a Callable instance, if so, invoke the call function.
It checked if the Job class is a Runnable instance, if so, invoke the run function.
However, clojure.lang.IFn has already extended Callable and Runnable, so the exception raised because I forgot to implement the call function.
I need to create JSON objects from clojure maps that store things like clojure vars. The base implementation throws this kind of error when it sees them:
java.lang.Exception: Don't know how to write JSON of class clojure.lang.Var
Can anybody point me to sample code on how to extend the capabilities of the JSON writer?
Thanks.
Well, I figured out the answer. There's another SO question that answers it partially: How to map clojure code to and from JSON?
But here's the code that worked for me:
(defn- write-json-clojure-lang-var [x #^PrintWriter out]
(.print out (json-str (str x))))
(extend clojure.lang.Var clojure.contrib.json/Write-JSON
{:write-json write-json-clojure-lang-var})
Note that all I wanted to do is just render a string version of which Var I'm referring to. You could, of course, do many other things...
An update to the answer from zippy for those of us using the newer clojure.data.json. This is code that will work with the updated/new library:
(defn- write-json-clojure-lang-var [x #^PrintWriter out]
(.print out (json-str (str x))))
(extend clojure.lang.Var clojure.data.json/JSONWriter
{:-write write-json-clojure-lang-var})
can i somehow find all functions/macros that take a specific type of parameter ?
for example, what function accepts a Namespace object as parameter ?
(this is because i can create a namespace and store it in a var, but i don't know where i could use that var; what function might i pass that var to ?)
here is some code:
user=> (def working-namespace (create-ns 'my-namespace))
#'user/working-namespace
;i created a namspace and want to use it later
user=> (class working-namespace)
clojure.lang.Namespace
; out of curiosity i found out that "working-namespace" is a Namespace object
user=> (ns working-namespace)
nil
working-namespace=>
; but how do i switch to it ? this didn't do what i wanted...
user=> (refer working-namespace)
java.lang.ClassCastException: clojure.lang.Namespace cannot be cast to clojure.lang.Symbol (NO_SOURCE_FILE:0)
; this did not work either as my expectations
user=> (the-ns working-namespace)
#<Namespace my-namespace>
user=> (class (the-ns working-namespace))
clojure.lang.Namespace
; great, this gave me the same thing, a Namespace
hence the question: how do i use it dynamically (that's why i had put my namespace into a var) ? how do i get something useful for me from a var that points to a namespace ?
i can try look around for functions that make use of a Namespace object or that convert it to something else. i did and only found "intern". searching by hand not seems not that promising
what if i have this problem a million time ? is there an automated way to get me what i'm looking for without having to ask around each time ?
In Clojure 1.2 and previous function arguments dont have types. every function argument is an object. So the question really becomes "how do i find functions that will cast the object I pass them into this type. so searching for type hints will find some of them, though it wont get you everything. I wish it where more possible to answer this in general.
starting with 1.3 (current dev branch 9/2010) function paramerters and return types can have a defined type and will be passed/returned as that type instead of being cast to object and then cast on the other side. This drops one of the zeros from the exacution time of numerical functions with the important limitation that it only works for :static functions and only with direct calls (ie: not through map/reduce/filter/etc.) There is not a lot published on this change yet though it has the important breaking change that integers are no longer boxed by default and integer (actually Long) overflow throws an exception. you can read more here
(defn ^:static fib ^long [^long n]
(if (<= n 1)
1
(+ (fib (dec n)) (fib (- n 2)))))
so after 1.3 is released and widely adopted you will see code with more commonly defined types because they will offer a big speed benefit and then you will be able to find more functions by argument type though still not all of them.
At the same lecture where I learned about function argument types, Rich mentioned plans in the distant Clojure future (after 'Clojure in Clojure') about better support for exposing the compiler internals to tools such as IDEs. So there is hope that someday you will get a real answer to this question.
Dynamic languages make this slightly more difficult in practice and a lot harder in theory.
You already got a good answer from Arthur, so I'll only answer the "how do i get something useful for me from a var that points to a namespace ?". From (doc ns), note that it says unevaluated:
user=> (doc ns)
-------------------------
clojure.core/ns
([name docstring? attr-map? references*])
Macro
Sets *ns* to the namespace named by name (unevaluated), creating it
Now there's something you could do with in-ns if you want (the whole namespace object -> string -> symbol conversion is probably stupid, but enough to illustrate my point):
user=> (in-ns (symbol (str working-namespace)))
#<Namespace my-namespace>
my-namespace=>
I don't think you can do that without a lot of hackery in a dynamic language. If you want to now what function that take namespaces look at the documentation of the namespace stuff.
For example cleaning namespaces or reload them.
You wrote:
user=> (ns working-namespace)
nil
working-namespace=>
; but how do i switch to it ? this didn't do what i wanted...
But you did switch to the working-namespace namespace (that's why the prompt changed), so I'm not clear as to what "you wanted".
As I noted earlier, you need to present the ultimate problem are you trying to solve. It's entirely likely that messing with namespace objects won't be the solution.