I am using Common Lisp, SBCL, Emacs, and Slime.
In SLIME's REPL, I have a variable holding the file location:
CL-USER> json-file
#P"/home/pedro/lisp/json-example.json"
The content of the file is:
{"keyOne": "valueOne"}
I would like to read the data inside the file and return a string:
"{"keyOne": "valueOne"}"
How can I do it? Is it possible to do it with cl-json famous library?
The documentation was terse/hard. It is not good on providing examples. I couldn't find how to do it.
From what I read from the documentation, the API is built around streams. The function json:decode-json is taking a stream in parameter and return an association list which is very convenient to use.
To extract a value from the key, you can use the function (assoc :key-1 assoc-list). It will return a cons with (key . value). To get the value, you need to use the cdr function.
(defparameter json-string "{\"key-1\": \"value-1\"}")
(with-input-from-string (json-stream json-string)
(let ((lisp-data (json:decode-json json-stream)))
(cdr (assoc :key-1 lisp-data))))
Obviously, if you have the data in a file you could directly use the stream:
(with-open-file (json-stream "myfile.json" :direction :input)
(let ((lisp-data (json:decode-json json-stream)))
(cdr (assoc :key-1 lisp-data))))
Edit due to the comment of OP
The content of the file is:
{"keyOne": "valueOne"}
I would like to read the data inside the file and return a string:
"{"keyOne": "valueOne"}"
This question seems completely unrelated to the JSON library, but anyhow, if one need to open a file and put its content into a string, he can use a function from uiop.
* (uiop:read-file-string "test.txt")
"{\"keyOne\": \"valueOne\"}"
uiop is a library shipped with ASDF, so it's probably available to most Common Lisp's distributions. It's a kind of de-facto standard library and have a lot of interesting functions.
I have done the test on my computer, it's seems to work. I can certify that no data was harmed during that test.
Related
I'm attempting to use a string saved in a variable to call a function like so:
(defn update-product-list [] "Test")
(defn handle-state-change [action]
((resolve (symbol action))))
(handle-state-change "update-product-list")
However, this gives me the following error: Assert failed: Argument to resolve must be a quoted symbol
I've also tried changing the above line to:
((resolve (quote (symbol action))))
But this still gives an error. I also tried changing it just to:
((resolve 'action))
But this gives a different error I don't quite understand: js/action is shadowed by a local. I don't want to override the function just call it. Not sure where I'm going wrong. I've looked at a few examples, but can't see to pin it down.
ClojureScript supports :advanced optimization, in which Google Closure Compiler will rename, inline, or eliminate (unused) functions in order to implement minification. In short, the name of the function you want to look up will, in general, simply no longer exist under :advanced.
Because of this, ClojureScript's resolve is a compile-time facility (a macro requiring a literal quoted symbol).
If you are using :simple or self-hosted ClojureScript, more options are available to you because the support needed persists into runtime. For example Planck has a planck.core/resolve that behave's like Clojure's resolve. A similar approach is possible in Lumo, and similar facilities can be fashioned if using :simple.
In general though, given :advanced, if you need to map strings to a set of functions, you need to somehow arrange to have a static mapping constructed at compile time to support this (the set of functions must be known a priori, at compile time).
If you have a namespace (the name of which is statically known at compile time) which defines functions that need to be dynamically called via strings, you could consider making use of ns-publics:
cljs.user=> (ns foo.core)
foo.core=> (defn square [x] (* x x))
#'foo.core/square
foo.core=> (in-ns 'cljs.user)
nil
cljs.user=> (when-some [fn-var ((ns-publics 'foo.core) (symbol "square"))]
(fn-var 3))
9
This will work under :advanced. The mapping constructed by ns-publics is static; built at compile-time. If you have multiple namespaces that need such treatment, you could merge several calls to ns-publics to build a larger map.
The advantage of this approach is that the code involved is pretty short and requires little maintenance. The disadvantage is that it dumps all of the public vars of the namespace (foo.core in this example) into your generated code (and the generated code for vars is somewhat verbose). Another disadvantage is that you need to statically know the namespace(s) involved at compile time.
If you need to further minimize generated code size, you could just build / maintain a simple static map from string to function value as in
(def fns {"square" foo.core/square})
and use it appropriately, keeping it up to date as your codebase evolves.
Another option would be to mark the functions that you need to access using ^:export meta, and then to call those functions using JavaScript interop. For example if you define the function this way
(defn ^:export square [x] (* x x))
then you can use strings / interop to lookup the function and call it at runtime. Here's an example:
((goog.object/getValueByKeys js/window #js ["foo" "core" "square"]) 3)
The use of ^:export and :advanced is covered here. If you know that you are using :simple or less, then you can simply use JavaScript interop to call the functions of interest, without needn't to use ^:export.
Note that there is no general solution that would let you look up a function by name at runtime under :advanced without somehow putting some aspect of that function into your code at compile time. (In fact, if a function is not referenced in a way that Google Closure Compiler can statically, see, the function implementation will be completely eliminated as dead code.) In the above, ns-publics grabs all the vars for a namespace at compile time, rolling your own lookup map sets up static code to refer to the function value, and using ^:export statically arranges to make the name of the function persist into runtime.
You need to use it like this:
((resolve 'inc) 5)) => 6
or, deconstructed a bit:
(let [the-fn (resolve 'inc)]
(the-fn 7))
=> 8
If you have the function name as a string, use the symbol function to convert from string => symbol (from clojuredocs.org):
user=> ((-> "first" symbol resolve) [1 2 3])
1
And, never forget the Clojure CheatSheet!
My problem requires applying custom logical functions to a structure. Those functions are stored in a database as a string. I have data like this:
(def fruits {:apple {:color "red" :ripe? true}
:strawberry {:color "red" :ripe? false}})
And I have this cond check:
"(some (fn [fruit] (-> fruit val :ripe? false?)) fruits)"
Unfortunatelly I can't get this right even though I tried various approaches:
1)
(cljs/eval-str (cljs/empty-state)
"(some (fn [fruit] (-> fruit val :ripe? false?)) my.main/fruits)"
""
{:eval cljs/js-eval}
identity)
This works yet it yields errors:
WARNING: No such namespace: my.main, could not locate my/main.cljs, my/main.cljc, or Closure namespace "" at line 1
WARNING: Use of undeclared Var my.main/fruits at line 1
Also this approach obviously wouldn't work in advanced compilation.
2) I tried to leverage approach that works in Clojure:
((eval
(read-string
"(fn [fruits]
(some (fn [fruit] (-> fruit val :ripe? false?)) fruits))"))
fruits)
I can't see why this wouldn't work in advanced compilation. Unfortunatelly it simply returns nil every single time.
Is it just me who fails to come up with a solution or is CLJS just not capable of doing that yet?
I suspect your going to have a vary hard time achieving your requirement using
this approach. The big problem is likely going to be due to the way
clojurescirpt needs to be compiled into javascript (using Google closure). You
can probably get it to work doing some clever stuff with externals and using low
level javascript interop and the closure library, but I suspect it will be hard
work.
A couple of alternative approaches which may be worth considering
Store the data in the database in edn format. Using edn, you may be able to
read it into a var and then execute it
Change direction - do you really need to store complete functions or could you
instead define a type of DSL which obtains parameters from the database which
will provide the necessary level of dynamic execution.
Could you have some sort of pre-processing solution i.e. write the function in
clojurescript, but use closure functionality to compile that to javascript and
insert that into the database instead of the raw clojurescript. This would
make the initial storing of the data more complex, but may simplify calling
the dynamic functions at runtime. You could even include some code checks or
validation to reduce the likelihood of code taken from the database doing the
wrong thing.
There are so many risks associated with using totally dynamic code it is almost
never a good solution. Aside from the numerous security issues you have with
this approach, you also have to deal with elegantly handling problems arising
from buggy definitions being inserted into the database (i.e. a buggy function
definition which crashes your app or corrupts data. If you just have to have the
ability to dynamically execute unknown code, then at least edn provides some
additional protection you don't get with eval-str - but really, just don't do
it.
After hours of experiments and struggling with evaluating functions from strings I decided to write DSL.
In database I store string with a map containing these parameters:
:where? - vector containing path to a desired answer.
:what? - answer(s) I'm looking for.
:strict? (optional) - A boolean. If true then answers need to be exactly the same as :what? rule (order doesn’t matter).
Then I just evaluate that simple cljs file. It works both on advanced and none optimization mode.
(defn standard-cond-met? [{:keys [what? where? strict?]
:or {strict? false}}]
(let [answer (get-in answers (conj where? :values))]
(if strict?
(= (sort what?) (sort answer))
(clojure.set/subset?
(set what?)
(set answer)))))
Is there a way to serialize functions at runtime in Clojure? I'd like to be able to send stateless (but not pure) functions over the wire in a serialized format (probably edn, but I'm open to anything).
For example...
If I run prn-str on a function, I don't get what I expected/wanted.
user=> (def fn1 (fn [x] (* x 2)))
#'user/fn1
user=> (def data {:test 1 :key "value"})
#'user/data
user=> (defn fn2 [x] (* x 2))
#'user/fn2
user=> (prn-str fn1)
"#object[user$fn1 0x28b9c6e2 \"user$fn1#28b9c6e2\"]\n"
user=> (prn-str data)
"{:test 1, :key \"value\"}\n"
user=> (prn-str fn2)
"#object[user$fn2 0x206c48f5 \"user$fn2#206c48f5\"]\n"
user=>
I would have wanted/expected something like this:
user=> (prn-str fn2)
"(fn [x] (* x 2))\n"
or, maybe,
user=> (prn-str fn2)
"(defn fn2 [x] (* x 2))\n"
You would have to use quote or ' to prevent evaluation and eval to force evaluation:
(def fn1 '(fn [x] (* x 2)))
(prn-str fn1) ;;=> "(fn [x] (* x 2))\n"
((eval fn1) 1) ;;=> 2
Flambo, a Clojure wrapper for Spark, uses the serializable-fn library to serialize functions (which Spark requires). Sparkling, another wrapper for Spark, uses native Clojure functions through this Java abstract class that implements the Java interface Serializable.
You have basically two choices:
pass source code (s-expressions stored as clojure data)
pass jar files and load them on the other side.
for the first option you save the source at the time the function is compiles (almost always when it is defined) and then pass the same source expression to the other computer and let it compile the same thing. so first you might make a vector of expressions:
(domain-functions '[(defn foo [x] x)
(defn bar [y] (inc y)]
then you can store this into a database and each client can pass it to read and then they will all have the same functions.
The second option depends on the fact that each time you define a function it produces a class file in the /target directory and then loads it. You can then syncronize this directory and load them on the other side. This approach is of course completely crazy, though people do crazy stuff around here. I recommend the first approach
And as a personal note:
I'm doing this now with datomic, and I have adopted the practice of putting the git-hash into the function name using a macro so I know absolutly for certain that when I call a function, I'm getting the same function I see in the editor. This brings peace of mind when running many instances that all pull from the same DB.
At some point it ceases to be Clojure, so the expectation that we can arbitrarily round trip from source to machine instructions and back is a little bit off.
We should be able to serialize a function to a byte array and send that across the wire though. I suspect you'd need to grab the function's java.lang.Class object and then pass that through a java.lang.instrument.ClassFileTransformer to get the bytes. Once you have those you can pass them through to the friendly java.lang.ClassLoader on the remote jvm.
You could use clojure.repl/source.
(with-out-str (source filter))
to get a string, or
(read-string (with-out-str (source filter)))
to get a clojure list.
There really isn't a good way, and for good reason simply shipping a function to another computer or storing it in a DB can cause lots of problems, not the least of which is that that function may require other functions that aren't on the other end.
A much better idea is to stick with data. Instead of writing the function, write the name of the function as a event, and then that even can be translated later by whatever is reading your data. Stick with data, that's the idiomatic way.
Please see #7755661 first. I am using ECL and basically want to execute some code, trap any kind of condition that may occur and then continue execution, without prompting or entering the debugger. This is easy to achieve with the following handler-case macro:
(handler-case
(load "code.lisp") ; this may raise a condition
(error (condition)
(print condition))) ; this prints sth like #<a UNBOUND-VARIABLE>
My only problem is that I cannot find a generic way to print a more meaningful error for the user. Indeed my application is an HTTP server and the output goes to a web page. code.lisp is written by the user and it can raise any kind of condition, I do now want to list them all in my code. I would just like to print the same error message I see on the REPL when I do not use handler-case, but in the HTML page, e.g. for an "unbound variable" error, a string like "The variable VAR is unbound".
By inspecting a condition object of type UNBOUND-VARIABLE I see it has two slots: SI:REPORT-FUNCTION, which is a compiled function and SI:NAME, set to the name of the variable in this case. I guess SI:REPORT-FUNCTION could be what I need to invoke but how can I call it? If I try:
(handler-case foo (error (condition) (SI::REPORT-FUNCTION condition)))
it tells me that SI:REPORT-FUNCTION is undefined. SI or SYS in ECL is a package for functions and variables internal to the implementation, but I don't worry if my code is not portable, as long as it works.
BTW in other kinds of condition objects there are also other apparently useful slots for my purpose, named SI:FORMAT-CONTROL and SI:FORMAT-ARGUMENT, but I cannot access any of them from my code too.
I was looking for somethink alike to the getMessage() method of Java exception objects in Lisp, but none of my sources ever mentions something like that.
Moreover, is there any hope to be able to get the line number in code.lisp where the error occurred too? Without that it would be difficult for the user to locate the problem in his code.lisp source file. I would really want to provide this information and stopping at the first error is acceptable for me.
In Common Lisp when print escaping is disabled, the error message is printed.
CL-USER > (handler-case
a
(error (condition)
(write condition :escape nil)))
The variable A is unbound.
#<UNBOUND-VARIABLE 4020059743>
Note that PRINT binds *print-escape* to T.
Using PRINC works - it binds *print-escape* to NIL.
CL-USER > (handler-case
a
(error (condition)
(princ condition)))
The variable A is unbound.
#<UNBOUND-VARIABLE 4020175C0B>
This is described in CLHS 9.1.3 Printing Conditions.
Also note, when you have an object, which has a slot and the value of this slot is a function, then you need to get the slot value using the function SLOT-VALUE and then use FUNCALL or APPLY and call the function with the correct arguments.
If you have a condition of type simple-condition then it has a format-control and a format-argument information. This is described with an example how to use it for FORMAT in CLHS Function SIMPLE-CONDITION-FORMAT-CONTROL, SIMPLE-CONDITION-FORMAT-ARGUMENTS
My answer below is based on one I already gave at the ECL mailing list. Actually I would claim that this is not an embedding problem, but a Lisp one. You want to get some information at the file position of the form which caused the error. This is not attached to a condition because conditions happen independently of whether the form evaluated was interpreted, compiled or part of a function that is already installed in the Lisp image. In other words, it is up to you to know the position of the file which is being read and do some wrapping that adds the information.
The following is nonstandard and prone to change: ECL helps you by defining a variable ext::source-location when LOAD is used on a source file. This variable contains a CONS that should NEVER be changed or stored by the user, but you can get the file as (CAR EXT:*SOURCE-LOCATION*) and the file position as (CDR EXT:*SOURCE-LOCATION*). The plan is then to embed your LOAD form inside a HANDLER-BIND
(defparameter *error-message* nil)
(defparameter *error-tag* (cons))
(defun capture-error (condition)
(setf *error*
(format nil "At character ~S in file ~S an error was found:~%~A"
(cdr ext:*source-location*)
(car ext:*source-location*)
condition)))
(throw *error-tag* *error-message*))
(defun safely-load (file)
(handler-bind ((serious-condition #'capture-error))
(catch *error-tag*
(load file)
nil)))
(SAFELY-LOAD "myfile.lisp") will return either NIL or the formatted error.
In any case I strongly believe that relying on LOAD for this is doomed to fail. You should create your own version of LOAD, starting from this
(defun my-load (userfile)
(with-open-file (stream userfile :direction :input :external-format ....whateverformat...)
(loop for form = (read stream nil nil nil)
while form
do (eval-form-with-error-catching form))))
where EVAL-FORM-.... implements something like the code above. This function can be made more sophisticated and you may keep track of file positions, line numbers, etc. Your code will also be more portable this way.
So please, read the ANSI Spec and learn the language. The fact that you did not know how to print readably a condition and instead tried to play with ECL internals shows that you might face further problems in the future, trying to go with non-portable solutions (hidden slot names, report functions, etc) instead of first trying the standard way.
I am using mochiweb and I don't know how to use its json encoder to deal with complex data structure. What's the differences between mochijson and mochijson2? Is there any good example? I always get the following errors:
46> T6={struct,[{hello,"asdf"},{from,"1"},{to,{a,"aa"}}]}.
{struct,[{hello,"asdf"},{from,"1"},{to,{a,"aa"}}]}
47> mochijson2:encode(T6).
** exception exit: {json_encode,{bad_term,{a,"aa"}}}
in function mochijson2:json_encode/2
in call from mochijson2:'-json_encode_proplist/2-fun-0-'/3
in call from lists:foldl/3
in call from mochijson2:json_encode_proplist/2
39> T3={struct,[{hello,"asdf"},{[{from,"1"},{to,"2"}]}]}.
{struct,[{hello,"asdf"},{[{from,"1"},{to,"2"}]}]}
40> mochijson:encode(T3).
** exception error: no function clause matching mochijson:'-json_encode_proplist/2-fun-0-'({[{from,"1"},{to,"2"}]},
[44,"\"asdf\"",58,"\"hello\"",123],
{encoder,unicode,null})
in function lists:foldl/3
in call from mochijson:json_encode_proplist/2
mochijson2 works with strings as binaries, lists as arrays, and only decodes UTF-8. mochijson decodes and encodes unicode code points.
To create a deep struct I did the following:
2> L = {struct, [{key2, [192]}]}.
{struct,[{key2,"?"}]}
3>
3> L2 = {struct, [{key, L}]}.
{struct,[{key,{struct,[{key2,"?"}]}}]}
4>
4> mochijson:encode(L2).
[123,"\"key\"",58,
[123,"\"key2\"",58,[34,"\\u00c0",34],125],
125]
So if you recursively create your data structure using lists then you'll be fine. I'm not sure why deep structs aren't supported, but they don't seem to be, at least not the way you're trying to create them. Maybe someone else knows a more clever way to do this.
You can also check out this thread: mochijson2 examples!
or
Video Tutorial To Start Developing Web Applications on Erlang
T6={struct, [{hello,"asdf"},{from,"1"},{to, {a,"aa"}} ]}.
should instead be:
T6={struct, [{hello,"asdf"},{from,"1"},{to, {struct, [{a,"aa"}]}} ]}.
The nested structures need to have the same "struct" style as the top-level object.