How to convert json-string into CLOS object using cl-json library? - json

If there is a class and a json:
(defclass foo ()
((bar :initarg :bar)))
(defvar *input* "\{ \"bar\" : 3 }")
How to convert *input* into an instance of foo using cl-json library?
I guess it should be something like:
(with-decoder-simple-clos-semantics
(let ((*prototype-name* 'foo))
(decode-json-from-string *input*)))
But it produces:
Invalid SB-MOP:SLOT-DEFINITION initialization: the
initialization argument :NAME was constant: :BAR.
[Condition of type SB-PCL::SLOTD-INITIALIZATION-ERROR]
What am I doing wrong?

The cause of the error is that cl-json:*json-symbols-package* is bound to the KEYWORD package: when JSON keys are turned into symbols, they become keywords which apparently are not valid as slot names.
Fluid objects
The following works:
(let ((json:*json-symbols-package* (find-package :cl-user)))
(json:with-decoder-simple-clos-semantics
(json:decode-json-from-string "{ \"bar\" : 3 }")))
(note: you only need backslashes before double-quote characters)
You obtain a FLUID-OBJECT.
Prototype key in JSON data
Now, you can also define your own class:
(in-package :cl-user)
(defclass foo ()
((bar :initarg :bar)))
And then, the JSON needs to have a "prototype" key:
(let ((json:*json-symbols-package* (find-package :cl-user)))
(json:with-decoder-simple-clos-semantics
(json:decode-json-from-string
"{ \"bar\" : 3 ,
\"prototype\" : { \"lispClass\" : \"foo\",
\"lispPackage\" : \"cl-user\" }}")))
The above returns an instance of FOO.
You can use a different key than "prototype" by rebinding *prototype-name*.
Force a default prototype (hack)
Without changing the existing library code, you can hack around it to change the behavior of the decoding step. The code is organized around special variables that are used as callbacks at various point of the parsing, so it is a matter of wrapping the expected function with your own:
(defun wrap-for-class (class &optional (fn json::*end-of-object-handler*))
(let ((prototype (make-instance 'json::prototype :lisp-class class)))
(lambda ()
;; dynamically rebind *prototype* right around calling fn
(let ((json::*prototype* prototype))
(funcall fn)))))
The above creates a prototype object for the given class (symbol), capture the current binding of *end-of-object-handler*, and returns a closure that, when called, bind *prototype* to the closed-over prototype instance.
Then, you call it as follows:
(let ((json:*json-symbols-package* *package*))
(json:with-decoder-simple-clos-semantics
(let ((json::*end-of-object-handler* (wrap-for-class 'foo)))
(json:decode-json-from-string
"{ \"bar\" : 3 }"))))
And you have an instance of FOO.
Recursion
Note that if you define foo as follows:
(defclass foo ()
((bar :initarg :bar :accessor bar)
(foo :initarg :foo :accessor foo)))
Then the hack also reads nested JSON objects as FOO:
(let ((json:*json-symbols-package* *package*))
(json:with-decoder-simple-clos-semantics
(let ((json::*end-of-object-handler* (wrap-for-class 'foo)))
(json:decode-json-from-string
"{ \"bar\" : 3, \"foo\" : { \"bar\" : 10} }"))))
=> #<FOO {1007A70E23}>
> (describe *)
#<FOO {1007A70E23}>
[standard-object]
Slots with :INSTANCE allocation:
BAR = 3
FOO = #<FOO {1007A70D53}>
> (describe (foo **))
#<FOO {1007A70D53}>
[standard-object]
Slots with :INSTANCE allocation:
BAR = 10
FOO = #<unbound slot>

Related

How to represent non-standard java objects when encoding to JSON in Clojure?

I have a standard clojure map of things. The keys are keywords, and the values are arbitrary values. They can be nil, numbers, strings, or any other kind of JVM object/class.
I need to know how to encode this map into JSON, so that "normal" values map to the usual JSON values (e.g. keywords -> strings, integers -> JSON numbers, etc.), while values of any other class map to the string representations of those values, like this:
{
:a 1
:b :myword
:c "hey"
:d <this is an "unprintable" java File object>
}
gets encoded thus:
{ "a": 1, "b": "myword", "c": "hey", "d": "#object[java.io.File 0x6944e53e foo]" }
I want to do this because my program is a CLI parsing library and I'm working with the caller of the library to build this map up, so I don't exactly know what types of data will be in it. However, I would like to print it to the screen all the same to aid the caller in debugging. I have tried to naively give this map to cheshire, but when I do, cheshire keeps choking with this error:
Exception in thread "main" com.fasterxml.jackson.core.JsonGenerationException: Cannot JSON encode object of class: class java.io.File: foo
Bonus: I am trying to keep my dependency counts down and I have already vetted cheshire as my JSON library, but full marks if you can find a way to do the above without it.
With cheshire you can add an encoder for java.lang.Object
user> (require ['cheshire.core :as 'cheshire])
nil
user> (require ['cheshire.generate :as 'generate])
nil
user> (generate/add-encoder Object (fn [obj jsonGenerator] (.writeString jsonGenerator (str obj))))
nil
user> (def json (cheshire/generate-string {:a 1 :b nil :c "hello" :d (java.io.File. "/tmp")}))
#'user/json
user> (println json)
{"a":1,"b":null,"c":"hello","d":"/tmp"}
you can also override the print-method for some objects you're interested in:
(defmethod print-method java.io.File [^java.io.File f ^java.io.Writer w]
(print-simple (str "\"File:" (.getCanonicalPath f) "\"") w))
the method gets called by the print subsystem every time it needs to print out the object of this type:
user> {:a 10 :b (java.io.File. ".")}
;;=> {:a 10,
;; :b "File:/home/xxxx/dev/projects/clj"}
Cheshire includes Custom Encoders that you can create and register to serialize arbitrary classes.
OTOH if you want to read the JSON back and reproduce the same types back in Java, you'll need to add some metadata too. A common pattern is to encode the type as some field like __type or *class*, like this, so that the deserializer can find the right types:
{
__type: "org.foo.User"
name: "Jane Foo"
...
}
Unless I'm missing something, there is no need for JSON here. Just use prn:
(let [file (java.io.File. "/tmp/foo.txt")]
(prn {:a 1 :b "foo" :f file})
=> {:a 1,
:b "foo",
:f #object[java.io.File 0x5d617472 "/tmp/foo.txt"]}
Perfectly readable.
As Denis said, you'll need more work if you want to read back the data, but that is impossible for things like a File object anyway.
You can get the result as a string (suitable for println etc) using a related function pretty-str if you prefer:
(ns tst.demo.core
(:use tupelo.test)
(:require
[tupelo.core :as t] ))
(dotest
(let [file (java.io.File. "/tmp/foo.txt")]
(println (t/pretty-str {:a 1 :b "foo" :f file}))
))
=> {:a 1,
:b "foo",
:f #object[java.io.File 0x17d96ed9 "/tmp/foo.txt"]}
Update
Here is a technique I frequently use when I need to coerce data into another form, esp. for debugging or unit tests:
(ns tst.demo.core
(:use tupelo.core tupelo.test)
(:require
[clojure.walk :as walk]))
(defn walk-coerce-jsonable
[edn-data]
(let [coerce-jsonable (fn [item]
(cond
; coerce collections to simplest form
(sequential? item) (vec item)
(map? item) (into {} item)
(set? item) (into #{} item)
; coerce leaf values to String if can't JSON them
:else (try
(edn->json item)
item ; return item if no exception
(catch Exception ex
(pr-str item))))) ; if exception, return string version of item
result (walk/postwalk coerce-jsonable edn-data)]
result))
(dotest
(let [file (java.io.File. "/tmp/foo.txt")
m {:a 1 :b "foo" :f file}]
(println :converted (edn->json (walk-coerce-jsonable m)))
))
with result
-------------------------------
Clojure 1.10.1 Java 14
-------------------------------
Testing tst.demo.core
:converted {"a":1,"b":"foo","f":"#object[java.io.File 0x40429f12 \"/tmp/foo.txt\"]"}

Reflect on Clojure Function Name

How do I use reflection in Clojure to extract the function name of a symbol that evaluates to a function?
(require '[clojure.reflect :as r])
(defn foo [x] x)
(r/reflect foo)
=> {#clojure.reflect.Field ... 'gobbledygook}
Something about (map :name (:members (r/reflect foo)))?
No, please do not use str because it behaves differently. In my example, here is what I've got:
user=> (defn foo [x] x)
#'user/foo
user=> (str foo)
"user$foo#7cb20059"
Instead, use combination of meta and var calls:
(-> foo var meta)
This code returns a map of variable's metadata. The key :name holds the function name as a string:
(-> foo var meta :name)
foo
If you also want to have an ns's name, do the following:
user=> (-> foo var meta :ns ns-name)
user
It gets the namespace object and returns its name with the ns-name function as a string.
Those methods do not depend on Clojure or REPL version and thus are more maintainable.
Looks like I can use str:
(str foo)
=> "#'my-project/foo"

Clojure: create function whose name is given as a parameter at run-time

I'm trying to create a Clojure function, that returns another function with a custom name. My attempts so far:
(defn function-with-custom-name [name] (fn name [] 42))
(function-with-custom-name "hello")
# --> #object[lang.main$function_with_custom_name$name__4660 0xa6afefa "lang.main$function_with_custom_name$name__4660#a6afefa"]
# syntactically ok, but its name is 'name' and not 'hello'
(defn function-with-custom-name [name] (fn (symbol name) [] 42))
# --> CompilerException java.lang.IllegalArgumentException: Parameter declaration symbol should be a vector, compiling:(/tmp/form-init3365336074265515078.clj:1:40)
(defn function-with-custom-name [name] (fn '(symbol name) [] 42))
# --> CompilerException java.lang.IllegalArgumentException: Parameter declaration quote should be a vector, compiling:(/tmp/form-init3365336074265515078.clj:1:40)
I understand that fn is a macro, and therefore proper quoting is probably important for the parameter, but as per above, I could not get it right, but I'm 99% sure there is a way, since (looking at the source of fn), the only criteria is that the first parameter should be recognized as a symbol.
Any hints on how to do this?
EDIT: Use-case, as asked in the comment: I'm writing a simple language interpreter in Clojure, which (among other things) lets you create functions. The functions from my language are currently represented by anonymous Clojure functions. However, it would make debugging the interpreter much easier, if the Clojure functions also did have a name.
EDIT2: The first edit made me think, and I came to the conclusion that I cannot use macro-based solutions for this problem, since I need to create the functions run-time (and, as far as I remember, macros can only work at compile-time). --> Changed the question title for clarity. Still, please don't delete the macro-based answers, since they give helpful insight.
You can use defmacro.
(defmacro function-with-custom-name [name]
`(fn ~(symbol name) ([] 42)))
you can also do it in runtime, without using macros, using namespace functions instead. It can give you the way to register functions from some input for example (i can't really find any good reason for this though, but maybe it's just me)
user> (defn runtime-defn [f-name f]
(intern (ns-name *ns*) (symbol f-name) f))
#'user/runtime-defn
user> (runtime-defn "my-fun" #(* 100 %))
#'user/my-fun
user> (my-fun 123)
;;=> 12300
user> (runtime-defn (read) #(* 200 %))
#input "danger!!!"
#'user/danger!!!
user> (danger!!! 1)
;;=> 200
Update:
For the simple version, you can use defmacro. For a more complicated version (such as used by the potemkin library) you need to mess around with creating a var and "intern-ing" it into the clojure namespace. This is accomplished via clojure.core/intern:
(ns tst.demo.core
(:use demo.core tupelo.test )
(:require [tupelo.core :as t] ))
(t/refer-tupelo)
(defmacro make-fn-macro-unquote [name]
(spyxx name)
`(defn ~name [] 42))
(defn make-fn-func-quote [name2]
(spyxx name2)
(intern 'tst.demo.core (symbol name2) (fn [] 43)))
(dotest
(make-fn-macro-unquote fred)
(spyxx fred)
(is= 42 (spyx (fred)))
(let [wilma-var (make-fn-func-quote "wilma")]
(spyxx wilma-var)
(is= 43 (spyx (wilma-var)))))
Look at the output:
name => clojure.lang.Symbol->fred
fred => tst.demo.core$fn__38817$fred__38818->#object[tst.demo.core$fn__38817$fred__38818 0x5f832a1 "tst.demo.core$fn__38817$fred__38818#5f832a1"]
(fred) => 42
name2 => java.lang.String->"wilma"
wilma-var => clojure.lang.Var->#'tst.demo.core/wilma
(wilma-var) => 43
Note that fred is a clojure function, while wilma-var is a clojure var. Please see this post on the relationship between a symbol like foo and a var and a function.
Note also that the macro version takes an unquoted symbol fred as input, while the function version takes a plain string in double-quotes as input.
So fred is a symbol pointing to a function, while wilma-var is a symbol pointing to a var (which then points to a function). In either case, Clojure allows us to type (fred) or (wilma-var) to make a function call, and we get either 42 or 43 as a result.

Is there a way to make a function var dynamic after definition?

Post updated to make it relevant to the course of events (responses and eliminating clutter).
Thank you very much for your time and help !
In some previous version of Clojure every var could be bound with a "binding" form.
Nowadays you get "Can't dynamically bind non-dynamic var" if not defined as dynamic.
In some contexts making a function var dynamic after definition could be useful (stubbing/mocking in testing).
Don't try:
(def ^{:dynamic true} log-call #'log-call)
It will eventually cause StackOverflowError since you are defining a function that calls itself (thank you for your explanation).
The updated question:
The approach suggested does not seem to work.
Forms called from the binding form don't get the binding defined.
Could you please help to figure out what I'm missing??
Here is the updated code:
(def all-expenses [{:amount 33.0 :date "today"}
{:amount 44.0 :date "yesterday"}])
(defn fetch-all-expenses [])
(defn fetch-expenses-greater-than [threshold]
(let [all (fetch-all-expenses)]
;calling from a nested form does not see the dynamically bound definition!
(println "2) from fetch-expenses-greater-than:" (fetch-all-expenses))
all))
(defn wrap [f]
(fn [& args] (apply f args)))
(def ^{:dynamic true} fetch-all-expenses (wrap fetch-all-expenses))
(binding [fetch-all-expenses (constantly all-expenses)]
(let [filtered (fetch-expenses-greater-than 15.0)]
(println "1) from inside binding:" (fetch-all-expenses))));calling from binding form OK!
The result of executing the in the repl is:
2) from fetch-expenses-greater-than: nil
1) from inside binding: [{:date today, :amount 33.0} {:date yesterday, :amount 44.0}]
nil
If I change the definition of fetch-all-expenses to
(defn ^:dynamic fetch-all-expenses [])
The result is as expected:
2) from fetch-expenses-greater-than: [{:date today, :amount 33.0} {:date yesterday, :amount 44.0}]
1) from inside binding: [{:date today, :amount 33.0} {:date yesterday, :amount 44.0}]
nil
It is possible to make a Var dynamic after it's been defined, but this will have no effect on code compiled before this change (it will still use the root binding). Use with-redefs to install custom replacements for functions during tests and the like.
The reason for this is that whether a Var is marked dynamic or not determines the way code using this Var is compiled. If it is not dynamic, such code will just get the root binding directly, saving some work; if it is dynamic, it will go through the somewhat more complex process of checking whether there is a thread-local binding in place for it.
So, there is no way to cause already compiled code use a custom function installed with binding after marking the Var holding the function dynamic. However, those calls still go through the Var, they just happen to go directly to the root binding, so you can use custom replacement functions for testing and the like if you install them as root bindings for the appropriate Vars. with-redefs encapsulates all the necessary logic for doing this cleanly.
Let's see how this works at the REPL:
;; define a non-dynamic Var:
(def foo 0)
;; this will throw an exception complaining about the attempt
;; to bind a non-dynamic Var:
(binding [foo 1]
foo)
;; let's define a function using foo;
;; we'll use it further down:
(defn bar []
foo)
;; now let's mark the Var dynamic:
(.setDynamic #'foo)
;; this will now evaluate to 1:
(binding [foo 1]
foo)
;; however, this will still return 0:
(binding [foo 1]
(bar))
(def ^{:dynamic true} log-call #'log-call) This statement says: "Create a var log-call and bind it to the var log-call. So when you try to use log-call var it will keep refering itself forever and hence StackOverflow exception.
You can try something like this:
(defn wrap [f]
(fn [& args] (apply f args)))
(def ^{:dynamic true} log-call (wrap log-call))
(def ^{:dynamic true} fetch-all-expenses (wrap fetch-all-expenses))
Thank you very much for your answer #MichaƂMarczyk. That explains it.
With code using the var before making it dynamic:
(def foo 0)
(defn bar []
foo)
(.setDynamic #'foo)
(binding [foo 1]
;; prints foo 1 . bar 0
(println "foo" foo ". bar" (bar)))
With code using the var after making it dynamic:
(def foo 0)
(.setDynamic #'foo)
(defn bar []
foo)
(binding [foo 1]
;; prints foo 1 . bar 1
(println "foo" foo ". bar" (bar)))
And yes!....with with-redefs instead of binding everything works as expected. That's exactly what I was needing.

Clojure: creating new instance from String class name

In Clojure, given a class name as a string, I need to create a new instance of the class. In other words, how would I implement new-instance-from-class-name in
(def my-class-name "org.myorg.pkg.Foo")
; calls constructor of org.myorg.pkg.Foo with arguments 1, 2 and 3
(new-instance-from-class-name my-class-name 1 2 3)
I am looking for a solution more elegant than
calling the Java newInstance method on a constructor from the class
using eval, load-string, ...
In practice, I will be using it on classes created using defrecord. So if there is any special syntax for that scenario, I would be quite interested.
There are two good ways to do this. Which is best depends on the specific circumstance.
The first is reflection:
(clojure.lang.Reflector/invokeConstructor
(resolve (symbol "Integer"))
(to-array ["16"]))
That's like calling (new Integer "16") ...include any other ctor arguments you need in the to-array vector. This is easy, but slower at runtime than using new with sufficient type hints.
The second option is as fast as possible, but a bit more complicated, and uses eval:
(defn make-factory [classname & types]
(let [args (map #(with-meta (symbol (str "x" %2)) {:tag %1}) types (range))]
(eval `(fn [~#args] (new ~(symbol classname) ~#args)))))
(def int-factory (make-factory "Integer" 'String))
(int-factory "42")
The key point is to eval code that defines an anonymous function, as make-factory does. This is slow -- slower than the reflection example above, so only do it as infrequently as possible such as once per class. But having done that you have a regular Clojure function that you can store somewhere, in a var like int-factory in this example, or in a hash-map or vector depending on how you'll be using it. Regardless, this factory function will run at full compiled speed, can be inlined by HotSpot, etc. and will always run much faster than the reflection example.
When you're specifically dealing with classes generated by deftype or defrecord, you can skip the type list since those classes always have exactly two ctors each with different arities. This allows something like:
(defn record-factory [recordname]
(let [recordclass ^Class (resolve (symbol recordname))
max-arg-count (apply max (map #(count (.getParameterTypes %))
(.getConstructors recordclass)))
args (map #(symbol (str "x" %)) (range (- max-arg-count 2)))]
(eval `(fn [~#args] (new ~(symbol recordname) ~#args)))))
(defrecord ExampleRecord [a b c])
(def example-record-factory (record-factory "ExampleRecord"))
(example-record-factory "F." "Scott" 'Fitzgerald)
Since 'new' is a special form, I'm not sure there you can do this without a macro. Here is a way to do it using a macro:
user=> (defmacro str-new [s & args] `(new ~(symbol s) ~#args))
#'user/str-new
user=> (str-new "String" "LOL")
"LOL"
Check out Michal's comment on the limitations of this macro.
Here is a technique for extending defrecord to automatically create well-named constructor functions to construct record instances (either new or based on an existing record).
http://david-mcneil.com/post/765563763/enhanced-clojure-records
In Clojure 1.3, defrecord will automatically defn a factory function using the record name with "->" prepended. Similarly, a variant that takes a map will be the record name prepended with "map->".
user=> (defrecord MyRec [a b])
user.MyRec
user=> (->MyRec 1 "one")
#user.MyRec{:a 1, :b "one"}
user=> (map->MyRec {:a 2})
#user.MyRec{:a 2, :b nil}
A macro like this should work to create an instance from the string name of the record type:
(defmacro newbie [recname & args] `(~(symbol (str "->" recname)) ~#args))