I am using mochiweb and I don't know how to use its json encoder to deal with complex data structure. What's the differences between mochijson and mochijson2? Is there any good example? I always get the following errors:
46> T6={struct,[{hello,"asdf"},{from,"1"},{to,{a,"aa"}}]}.
{struct,[{hello,"asdf"},{from,"1"},{to,{a,"aa"}}]}
47> mochijson2:encode(T6).
** exception exit: {json_encode,{bad_term,{a,"aa"}}}
in function mochijson2:json_encode/2
in call from mochijson2:'-json_encode_proplist/2-fun-0-'/3
in call from lists:foldl/3
in call from mochijson2:json_encode_proplist/2
39> T3={struct,[{hello,"asdf"},{[{from,"1"},{to,"2"}]}]}.
{struct,[{hello,"asdf"},{[{from,"1"},{to,"2"}]}]}
40> mochijson:encode(T3).
** exception error: no function clause matching mochijson:'-json_encode_proplist/2-fun-0-'({[{from,"1"},{to,"2"}]},
[44,"\"asdf\"",58,"\"hello\"",123],
{encoder,unicode,null})
in function lists:foldl/3
in call from mochijson:json_encode_proplist/2
mochijson2 works with strings as binaries, lists as arrays, and only decodes UTF-8. mochijson decodes and encodes unicode code points.
To create a deep struct I did the following:
2> L = {struct, [{key2, [192]}]}.
{struct,[{key2,"?"}]}
3>
3> L2 = {struct, [{key, L}]}.
{struct,[{key,{struct,[{key2,"?"}]}}]}
4>
4> mochijson:encode(L2).
[123,"\"key\"",58,
[123,"\"key2\"",58,[34,"\\u00c0",34],125],
125]
So if you recursively create your data structure using lists then you'll be fine. I'm not sure why deep structs aren't supported, but they don't seem to be, at least not the way you're trying to create them. Maybe someone else knows a more clever way to do this.
You can also check out this thread: mochijson2 examples!
or
Video Tutorial To Start Developing Web Applications on Erlang
T6={struct, [{hello,"asdf"},{from,"1"},{to, {a,"aa"}} ]}.
should instead be:
T6={struct, [{hello,"asdf"},{from,"1"},{to, {struct, [{a,"aa"}]}} ]}.
The nested structures need to have the same "struct" style as the top-level object.
Related
I am using Common Lisp, SBCL, Emacs, and Slime.
In SLIME's REPL, I have a variable holding the file location:
CL-USER> json-file
#P"/home/pedro/lisp/json-example.json"
The content of the file is:
{"keyOne": "valueOne"}
I would like to read the data inside the file and return a string:
"{"keyOne": "valueOne"}"
How can I do it? Is it possible to do it with cl-json famous library?
The documentation was terse/hard. It is not good on providing examples. I couldn't find how to do it.
From what I read from the documentation, the API is built around streams. The function json:decode-json is taking a stream in parameter and return an association list which is very convenient to use.
To extract a value from the key, you can use the function (assoc :key-1 assoc-list). It will return a cons with (key . value). To get the value, you need to use the cdr function.
(defparameter json-string "{\"key-1\": \"value-1\"}")
(with-input-from-string (json-stream json-string)
(let ((lisp-data (json:decode-json json-stream)))
(cdr (assoc :key-1 lisp-data))))
Obviously, if you have the data in a file you could directly use the stream:
(with-open-file (json-stream "myfile.json" :direction :input)
(let ((lisp-data (json:decode-json json-stream)))
(cdr (assoc :key-1 lisp-data))))
Edit due to the comment of OP
The content of the file is:
{"keyOne": "valueOne"}
I would like to read the data inside the file and return a string:
"{"keyOne": "valueOne"}"
This question seems completely unrelated to the JSON library, but anyhow, if one need to open a file and put its content into a string, he can use a function from uiop.
* (uiop:read-file-string "test.txt")
"{\"keyOne\": \"valueOne\"}"
uiop is a library shipped with ASDF, so it's probably available to most Common Lisp's distributions. It's a kind of de-facto standard library and have a lot of interesting functions.
I have done the test on my computer, it's seems to work. I can certify that no data was harmed during that test.
I'm redoing the backend of a very basic framework that connects to a completely customizable frontend. It was originally in PHP but for the refactor have been plodding away in F#. Although it seems like PHP might be the more suited language. But people keep telling me you can do everything in F# and I like the syntax and need to learn and this seemingly simple project has me stumped when it comes to JSON. This is a further fleshed out version of my question yesterday, but it got alot more complex than I thought.
Here goes.
The frontend is basically a collection of HTML files, which are simply loaded in PHP and preg_replace() is used to replace things like [var: varName] or [var: array|key] or the troublesome one: [lang: hello]. That needs to be replaced by a variable defined in a translation dictionary, which is stored as JSON which is also editable by a non-programmer.
I can't change the frontend or the JSON files, and both are designed to be edited by non-programmers so it is very likely that there will be errors, calls to language variables that don't exist etc.
So we might have 2 json files, english.json and french.json
english.json contains:
{
"hello":"Hello",
"bye":"Goodbye"
}
french.json:
{
"hello": "Bonjour",
"duck": "Canard"
//Plus users can add whatever else they want here and expect to be able to use it in a template
}
There is a template that contains
<b>[lang: hello]</b>
<span>Favourite Animal: [lang:duck]</span>
In this case, if the language is set to "english" and english.json is being loaded, that should read:
<b>Hello</b>
<span>Favourite Animal: </span>
Or in French:
<b>Bonjour</b>
<span>Favourite Animal: Canard</span>
We can assume that the json format key: value is always string:string but ideally I'd like to handle string: 'T as well but that might be beyond the scope of this question.
So I need to convert a JSON file (called by dynamic name, which gave F# Data a bit of an issue I couldn't solve last night as it only allowed a static filename as a sample, and since these two files have potential to be different from sample and provided, the type provider doesn't work) to a dictionary or some other collection.
Now inside the template parsing function I need to replace [lang: hello] with something like
let key = "duck"
(*Magic function to convert JSON to usable collection*)
let languageString = convertedJSONCollection.[key] (*And obviously check if containsKey first*)
Which means I need to call the key dynamically, and I couldn't figure out how to do that with the type that FSharp.Data provided.
I have played around with some Thoth as well to some promising results that ended up going nowhere. I avoided JSON.NET because I thought it was paid, but just realised I am mistaken there so might be an avenue to explore
For comparison, the PHP function looks something like this:
function loadLanguage($lang='english){
$json = file_get_contents("$lang.json");
return json_decode($json, true);
}
$key = 'duck';
$langVars = loadLanguage();
$duck = $langVars[$key] || "";
Is there a clean way to do this in F#/.NET? JSON seems really painful to work with in comparison to PHP/Javascript and I'm starting to lose my mind. Am I going to have to write my own parser (which means probably going back to PHP)?
Cheers to all you F# geniuses who know the answer :p
open Thoth.Json.Net
let deserialiseDictionary (s: string) =
s
|> Decode.unsafeFromString (Decode.keyValuePairs Decode.string)
|> Map.ofList
let printDictionary json =
json
|> deserialiseDictionary
|> fun m -> printfn "%s" m.["hello"] // Hello
For the question about 'T the question becomes, what can 'T be? For json it very limited, it can be a number of things, string, json-object, number, bool or json array. What should happen if it is bool or a number?
This is my first question and I have tried my best to find an answer - I have looked everywhere for an answer but haven't managed to find anything concrete to answer this in both the oj docs and ruby json docs and here.
Oj is a gem that serves to improve serialization/deserialization speeds and can be found at: https://github.com/ohler55/oj
I noticed this difference when I tried to dump and parse a hash with a NaN contained in it, twice, and compared the two, i.e.
# Create json Dump
dump = JSON.dump ({x: Float::NAN})
# Create first JSON load
json_load = JSON.parse(dump, allow_nan: true)
# Create second JSON load
json_load_2 = JSON.parse(dump, allow_nan: true)
# Create first OJ load
oj_load = Oj.load(dump, :mode => :compat)
# Create second OJload
oj_load_2 = Oj.load(dump, :mode => :compat)
json_load == json_load_2 # Returns true
oj_load == oj_load_2 # Returns false
I always thought NaN could not be compared to NaN so this confused me for a while until I realised that json_load and json_load_2 have the same object ID and oj_load and oj_load_2 do not.
Can anyone point me in the direction of where this memory allocation/object ID allocation occurs or how I can control that behaviour with OJ?
Thanks and sorry if this answer is floating somewhere on the internet where I could not find it.
Additional info:
I am running Ruby 1.9.3.
Here's the output from my tests re object IDs:
puts Float::NAN.object_id; puts JSON.parse(%q({"x":NaN}), allow_nan: true)["x"].object_id; puts JSON.parse(%q({"x":NaN}), allow_nan: true)["x"].object_id
70129392082680
70129387898880
70129387898880
puts Float::NAN.object_id; puts Oj.load(%q({"x":NaN}), allow_nan: true)["x"].object_id; puts Oj.load(%q({"x":NaN}), allow_nan: true)["x"].object_id
70255410134280
70255410063100
70255410062620
Perhaps I am doing something wrong?
I believe that is a deep implementation detail. Oj does this:
if (ni->nan) {
rnum = rb_float_new(0.0/0.0);
}
I can't find a Ruby equivalent for that, Float.new doesn't appear to exist, but it does create a new Float object every time (from an actual C's NaN it constructs on-site), hence different object_ids.
Whereas Ruby's JSON module uses (also in C) its own JSON::NaN Float object everywhere:
CNaN = rb_const_get(mJSON, rb_intern("NaN"));
That explains why you get different NaNs' object_ids with Oj and same with Ruby's JSON.
No matter what object_ids the resulting hashes have, the problem is with NaNs. If they have the same object_ids, the enclosing hashes are considered equal. If not, they are not.
According to the docs, Hash#== uses Object#== for values that only outputs true if and only if the argument is the same object (same object_id). This contradicts NaN's property of not being equal to itself.
Spectacular. Inheritance gone haywire.
One could, probably, modify Oj's C code (and even make a pull request with it) to use a constant like Ruby's JSON module does. It's a subtle change, but it's in the spirit of being compat, I guess.
I want to jsonify the results of a query performed against a Postgres table containing a column of type text[], but the problem is that clojure.data.json.write-str doesn't seem to know how to handle PG arrays:
Exception Don't know how to write JSON of class org.postgresql.jdbc4.Jdbc4Array clojure.data.json/write-generic
Do I have to supply a custom handler, or is there a simpler way?
Just to wrap up, here's what works for me, putting together Jared's answer with the latest clojure.java.jdbc API changes (0.3.0-alpha5 at the time of writing) which deprecate some patterns that are commonly used (e.g. with-query-results).
First define additional handlers for every unsupported types you care about, for instance:
(extend-type org.postgresql.jdbc4.Jdbc4Array
json/JSONWriter
(-write [o out]
(json/-write (.getArray o) out)))
(extend-type java.sql.Timestamp
json/JSONWriter
(-write [date out]
(json/-write (str date) out)))
Then use with-open to perform the JSON encoding while the connection is still open (which is important at least for org.postgresql.jdbc4.Jdbc4Array):
(with-open [conn (jdbc/get-connection db)]
(json/write-str
(jdbc/query {:connection conn}
["SELECT * FROM ..."]))))
I guess this will change as the clojure.java.jdbc API evolves, but for the moment, it works.
It is better to extend IResultSetReadColumn because you are not
required to stay in the scope of with-open
Something along the following is also more concise:
(extend-protocol jdbc/IResultSetReadColumn
org.postgresql.jdbc4.Jdbc4Array
(result-set-read-column [pgobj metadata idx]
(.getArray pgobj)))
Make sure the namespace where this is placed-in is required!
cheshire is good about handling various data types and being extensible (see add-encoder)
It looks like the issue is org.postgresql.jdbc4.Jdbc4Array does not implement java.util.Collection. Try calling getArray before you serialize it.
Edit:
If it is a nested structure, then it might be best to implement a handler. clojure.data.json uses the JSONWriter protocol. You can try something like the following:
;; Note: Pseudo Code
(extend-type org.postgresql.jdbc4.Jdbc4Array
clojure.data.json/JSONWriter
(-write [o out] (clojure.data.json/-write (.getArray o) out)))
I've been working on a very small program to grab details about Half Life 2 servers (using the protocol-srcds library). The workflow is pretty straightforward; it takes a list of servers from a file, queries each of them, and writes the output out to another file (which is read by a PHP script for display, as I'm tied to vBulletin). Would be nice if it was done in SQL or something, but seeing as I'm still just learning, that's a step too far for now!
Anyway, my question relates to serialization, namely, serializing to JSON. For now, I've written a scrappy helper function jsonify, such that:
jsonify (Just (SRCDS.GameServerInfo serverVersion
serverName
serverMap
serverMod
serverModDesc
serverAppId
serverPlayers
serverMaxPlayers
serverBots
serverType
serverOS
serverPassword
serverSecure
serverGameVersioning)) =
toJSObject [ ("serverName", serverName)
, ("serverMap", serverMap)
, ("serverPlayers", show serverPlayers)
, ("serverMaxPlayers", show serverMaxPlayers) ]
(I'm using the Text.JSON package). This is obviously not ideal. At this stage, however, I don't understand using instances to define serializers for records, and my attempts to do so met a wall of frustration in the type system.
Could someone please walk me through the "correct" way of doing this? How would I go about defining an instance that serializes the record? What functions should I use in the instance (showJSON?).
Thanks in advance for any help.
You might want to consider using Data.Aeson instead which might be regarded as the successor to Text.JSON.
With aeson you define separate instances for serialize/deserializing (with Text.JSON you have to define both directions even if you need only one, otherwise the compile will annoy you -- unless you silence the warning somehow), and it provides a few operators making defining instances a bit more compact, e.g. the example from #hammar's answer can be written a little bit less noisy as shown below with the aeson API:
instance ToJSON SRCDS.GameServerInfo where
toJSON (SRCDS.GameServerInfo {..}) = object
[ "serverName" .= serverName
, "serverMap" .= serverMap
, "serverPlayers" .= serverPlayers
, "serverMaxPlayers" .= serverMaxPlayers
]
One simple thing you can do is use record wildcards to cut down on the pattern code.
As for your type system problems, it's hard to give help without seeing error messages and what you've tried so far, however I suspect one thing that might be confusing is that the result of toJSObject will have to be wrapped in a JSObject data constructor as the return type of showJSON is supposed to be a JSValue. Similarly, the values of your object should also be of type JSValue. The easiest way to do this is to use their JSON instance and call showJSON to convert the values.
instance JSON SRCDS.GameServerInfo where
showJSON (SRCDS.GameServerInfo {..}) =
JSObject $ toJSObject [ ("serverName", showJSON serverName)
, ("serverMap", showJSON serverMap)
, ("serverPlayers", showJSON serverPlayers)
, ("serverMaxPlayers", showJSON serverMaxPlayers) ]