I am sending the below POST data to an XQuery module.
When I try to read it, I have tried everything, fn:doc, from-json, etc...., I cannot find a solution. Depending of what I try to do, I either get coercion error or data is not a string.
Can anyone help me put the Artist value to an $artist variable, etc...?
Here is the post request:
curl --anyauth --user admin:admin -H "Content-Type: application/json; charset=UTF-8" -H "Accept: application/json" -X POST -g "http://localhost:8030/LATEST/resources/jsontest/" -d "{"Artist" :"bowie","Weeks" : {"gte" :"1960-01-01","lte" :"2020-12-31"},"Genres":"","Released":"" }"*
and here is my code reading the POST request:
declare function test:post(
$context as map:map,
$params as map:map,
$input as document-node()*
) as document-node()*
{
xdmp:log($input)
};
and the result is this:
{"Artist":"bowie", "Weeks":{"gte":"1960-01-01", "lte":"2020-12-31"}, "Genres":"", "Released":""}
which looks like a json document (to my untrained eye), but i can't figure out how to "read/assign" the value for Artist (in this case "bowie") and put it in a variable.
I tried fn:doc, because I thought it should be a document-node
I tried let $artist := $input/Artist/data(), but get a coercion error
I tried many other option (from-json, etc..)
but I always wind up with some sort of error
how can I get access to the values in the JSON? I've been running around in circles through the MarkLogic documentation and different searches and nothing seems to work.
The subtle differences between a document-node(), a JSON object, a JSON document stored in the database as an object-node(), and a string of JSON can be confusing. Especially when if you log it and see the stringified representation, you just see the JSON and it is difficult to tell what it really was.
When in doubt, xdmp:describe() and xdmp:type() can be helpful for that. They can help tell you what sort of "thing" you really have.
When the method interface says you will get a document-node(), that is a generic container for stuff. It will have a child node(), but that node could be lots of things. It could be an XML document and the children could be comments, processing instructions, element, or it could be a binary(), text(), or a json:object. In this case, you know that you are going to have a JSON object.
You can use XPath to select that child node:
$json := $input/node()
With a JSON Object, you can access the Artist property by using map:get()
let $artist := map:get($json, "Artist")
The difference between a JSON object and an object-node() (JSON document stored in the database within MarkLogic) is that a regular JSON Object is just a data structure, but the object-node() lives in the database and has been parsed and saved as database nodes tied to indexes, so there are extra features and functionality. After inserting a JSON object into the database, if you later retrieve it with fn:doc(), then you it will be a JSON object-node() in the database, and you can do some neat things such as use XPath against it.
xdmp:from-json-string() can be used to parse a JSON string. It will construct a JSON object (which is basically a specialized map).
You can convert that object-node() into a JSON object using xdmp:from-json().
It is also possible to serialize the json:object map as an XML structure using json:transform-from-json() and then transform that XML structure back into a json:object using json:transform-to-json().
Related
I am using the Yason library in common-lisp, I want to parse a json string but would like the parser to keep one a its node unparsed.
Typically with an example like that:
{
"metadata1" : "mydata1",
"metadata2" : "mydata2",
"payload" : {...my long payload object},
"otherNodesToParse" : {...}
}
How can I set the yason parser to parse my json but skip the payload node and keep it as a string in the json format.
Use: let's say I just want the envelope data (everything that's not the payload), and to forward the payload as-is (as json string) to another system.
If I parse the whole json (so including payload) and then re-encode the payload to json, it is inefficient. The payload size could also be pretty big.
How do you know where the end of the payload object is in the stream? You do so by parsing the stream: if you don't parse the stream you simply can't know where the end of the object is: that's the nature of JSON's syntax (as it is the nature of CL's default syntax). For instance the only way you can know the difference between where to continue after
{x:1}
and after
{x:1.2}
is by parsing the two things.
So you must necessarily parse the whole thing.
So the answer to your question is: you can't do this.
You could (but not, I think, with YASON) decide that you did not want to build an object as a result of the parse. And perhaps, if the stream you are parsing corresponds to something with random access like a string or a file, you could note the start and end positions in the stream to later extract a string from it corresponding to the unparsed data (or you could perhaps build it up as you go).
It looks as if some or all of this might be possible with CL-JSON, but you'd have to work at it.
Unless the objects you are reading are vast the benefit of this seems questionable-to-none. If you really do want to do something like this efficiently you need a serialisation scheme which tells you how long things are.
The majority of my development experience has been from dynamically typed languages like PHP and Javascript. I've been practicing with Golang for about a month now by re-creating some of my old PHP/Javascript REST APIs in Golang. I feel like I'm not doing things the Golang way most of the time. Or more generally, I'm not use to working with strongly typed languages. I feel like I'm making excessive use of map[string]interface{} and slices of them to box up data as it comes in from http requests or when it gets shipped out as json http output. So what I'd like to know is if what I'm about to describe goes against the philosophy of golang development? Or if I'm breaking the principles of developing with strongly typed languages?
Right now, about 90% of the program flow for REST Apis I've rewritten with Golang can be described by these 5 steps.
STEP 1 - Receive Data
I receive http form data from http.Request.ParseForm() as formvals := map[string][]string. Sometimes I will store serialized JSON objects that need to be unmarshaled like jsonUserInfo := json.Unmarshal(formvals["user_information"][0]) /* gives some complex json object */.
STEP 2 - Validate Data
I do validation on formvals to make sure all the data values are what I expect before using it in SQL queries. I treat everyting as a string, then use Regex to determine if the string format and business logic is valid (eg. IsEmail, IsNumeric, IsFloat, IsCASLCompliant, IsEligibleForVoting,IsLibraryCardExpired etc...). I've written my own Regex and custom functions for these types of validations
STEP 3 - Bind Data to SQL Queries
I use golang's database/sql.DB to take my formvals and bind them to my Query and Exec functions like this Query("SELECT * FROM tblUser WHERE user_id = ?, user_birthday > ? ",formvals["user_id"][0], jsonUserInfo["birthday"]). I never care about the data types I'm supplying as arguments to be bound, so they're all probably strings. I trust the validation in the step immediately above has determined they are acceptable for SQL use.
STEP 4 - Bind SQL results to []map[string]interface{}{}
I Scan() the results of my queries into a sqlResult := []map[string]interface{}{} because I don't care if the value types are null, strings, float, ints or whatever. So the schema of an sqlResult might look like:
sqlResult =>
[0] {
"user_id":"1"
"user_name":"Bob Smith"
"age":"45"
"weight":"34.22"
},
[1] {
"user_id":"2"
"user_name":"Jane Do"
"age":nil
"weight":"22.22"
}
I wrote my own eager load function so that I can bind more information like so EagerLoad("tblAddress", "JOIN ON tblAddress.user_id",&sqlResult) which then populates sqlResult with more information of the type []map[string]interface{}{} such that it looks like this:
sqlResult =>
[0] {
"user_id":"1"
"user_name":"Bob Smith"
"age":"45"
"weight":"34.22"
"addresses"=>
[0] {
"type":"home"
"address1":"56 Front Street West"
"postal":"L3L3L3"
"lat":"34.3422242"
"lng":"34.5523422"
}
[1] {
"type":"work"
"address1":"5 Kennedy Avenue"
"postal":"L3L3L3"
"lat":"34.3422242"
"lng":"34.5523422"
}
},
[1] {
"user_id":"2"
"user_name":"Jane Do"
"age":nil
"weight":"22.22"
"addresses"=>
[0] {
"type":"home"
"address1":"56 Front Street West"
"postal":"L3L3L3"
"lat":"34.3422242"
"lng":"34.5523422"
}
}
STEP 5 - JSON Marshal and send HTTP Response
then I do a http.ResponseWriter.Write(json.Marshal(sqlResult)) and output data for my REST API
Recently, I've been revisiting articles with code samples that use structs in places I would have used map[string]interface{}. For example, I wanted to refactor Step 2 with a more standard approach that other golang developers would use. So I found this https://godoc.org/gopkg.in/go-playground/validator.v9, except all it's examples are with structs . I also noticed that most blogs that talk about database/sql scan their SQL results into typed variables or structs with typed properties, as opposed to my Step 4 which just puts everything into map[string]interface{}
Hence, i started writing this question. I feel the map[string]interface{} is so useful because majority of the time,I don't really care what the data is and it gives me to the freedom in Step 4 to construct any data schema on the fly before I dump it as JSON http response. I do all this with as little code verbosity as possible. But this means my code is not as ready to leverage Go's validation tools, and it doesn't seem to comply with the golang community's way of doing things.
So my question is, what do other golang developers do with regards to Step 2 and Step 4? Especially in Step 4...do Golang developers really encourage specifying the schema of the data through structs and strongly typed properties? Do they also specify structs with strongly typed properties along with every eager loading call they make? Doesn't that seem like so much more code verbosity?
It really depends on the requirements just like you have said you don't require to process the json it comes from the request or from the sql results. Then you can easily unmarshal into interface{}. And marshal the json coming from sql results.
For Step 2
Golang has library which works on validation of structs used to unmarshal json with tags for the fields inside.
https://github.com/go-playground/validator
type Test struct {
Field `validate:"max=10,min=1"`
}
// max will be checked then min
you can also go to godoc for validation library. It is very good implementation of validation for json values using struct tags.
For STEP 4
Most of the times, We use structs if we know the format and data of our JSON. Because it provides us more control over the data types and other functionality. For example if you wants to empty a JSON feild if you don't require it in your JSON. You should use struct with _ json tag.
Now you have said that you don't care if the result coming from sql is empty or not. But if you do it again comes to using struct. You can scan the result into struct with sql.NullTypes. With that also you can provide json tag for omitempty if you wants to omit the json object when marshaling the data when sending a response.
Struct values encode as JSON objects. Each exported struct field
becomes a member of the object, using the field name as the object
key, unless the field is omitted for one of the reasons given below.
The encoding of each struct field can be customized by the format
string stored under the "json" key in the struct field's tag. The
format string gives the name of the field, possibly followed by a
comma-separated list of options. The name may be empty in order to
specify options without overriding the default field name.
The "omitempty" option specifies that the field should be omitted from
the encoding if the field has an empty value, defined as false, 0, a
nil pointer, a nil interface value, and any empty array, slice, map,
or string.
As a special case, if the field tag is "-", the field is always
omitted. Note that a field with name "-" can still be generated using
the tag "-,".
Example of json tags
// Field appears in JSON as key "myName".
Field int `json:"myName"`
// Field appears in JSON as key "myName" and
// the field is omitted from the object if its value is empty,
// as defined above.
Field int `json:"myName,omitempty"`
// Field appears in JSON as key "Field" (the default), but
// the field is skipped if empty.
// Note the leading comma.
Field int `json:",omitempty"`
// Field is ignored by this package.
Field int `json:"-"`
// Field appears in JSON as key "-".
Field int `json:"-,"`
As you can analyze from above information given in Golang spec for json marshal. Struct provide so much control over json. That's why Golang developer most probably use structs.
Now on using map[string]interface{} you should use it when you don't the structure of your json coming from the server or the types of fields. Most Golang developers stick to structs wherever they can.
I have a need to represent JSON object in the feature file. I could use a json file for this for the code to pick up. But this would mean that i cant pass values from the feature file.
Scenario: Test
Given a condition is met
Then the following json response is sent
| json |
| {"dddd":"dddd","ggggg":"ggggg"}|
Above works for a normal json. However if there are nested objects etc then writing the json in a single line like above would make the feature very difficult to read and difficult to fix.
Please let me know.
You can use a string to do that, it makes the json much more legible.
Then the following json response is sent
"""
{
'dddd': 'dddd',
'ggggg': 'ggggg',
'somethingelse': {
'thing': 'thingvalue',
'thing2': 'thing2value'
}
}
"""
In the code, you can use it directly:
Then(/^the following json response is sent$/) do |message|
expect(rest_stub.body).to eq(message)
end
or something like that.
So I run golang appengine with go endpoints package ...
I use structs to marshal and un marshal my json incoming requests and out going responses ..
type BusinessWorker struct {
Wid string `json:"wid" datastore:"Worker_id" endpoints:"req,desc=Worker id. string value"`
Phone string `json:"phone" datastore:"Phone" endpoints:"req,desc=Worker phone number. string value"`
}
So as you can see after I validate the data this obj is saved or loaded to/from the datastore ..
My question is ..
there are many cases I dont want to respond with all my data that is saved in the datastore .. is there some sort of attribute that I can give to the param that i dont wanna include in my response only in my incoming requests ?
It seems so elementary .. and I cant find it .. ?
Maybe you would like to try one or a combination of the following approaches:
Tag of "-" so that the field is ignored. e.g. json:"-"
omitempty can be included in your 'json:' and will cause the field not to be included in the resulting json. So you could set the fields you want to hide to nil, prior to serializing to json. e.g. json:"myName,omitempty"
copier - there are some projects like: jinzhu's copier that would allow you to copy your entity to a simplified structure, or you could roll your own. (a combination of JSON un-marshalling and marshalling can produce similar results).
For more details about the JSON package see Golang Json marshal docs
I read about JSON from internet but still i have not got the grasp of it. I am reading this article
http://webcloud.se/log/AJAX-in-Django-with-jQuery/
I could not understood the first part where the function is using JSON
def xhr_test(request, format):
if request.is_ajax():
if format == 'xml':
mimetype = 'application/xml'
if format == 'json':
mimetype = 'application/javascript'
data = serializers.serialize(format, ExampleModel.objects.all())
return HttpResponse(data,mimetype)
# If you want to prevent non XHR calls
else:
return HttpResponse(status=400)
My Main Problems are
From where the function is getting format variable
Does format is json mean that data given to function is json or data which will be recived is json
Can anyone give me simple example that what will be the ouput of this function
data = serializers.serialize(format, ExampleModel.objects.all())
How will I use that data when i get that response in jquery function
If i don't use JSON in above function then how will the input and response back will chnage
Thanks
From where the function is getting format variable
In practice, there are lots of ways this format could be populated. HTTP provides an Accept: header that requests can use to indicate the preferred Content-Type for the response. On the client, you might use xhr.setRequestHeader('accept', 'application/json') to tell the server that you want your response in json format. In practice, though, very few frameworks actually do this. This being django, arguments to view functions are usually set in the urlconf, you might craft a urlconf like this:
urlpatterns = patterns('',
# ...
(r'^xhr_test.(?<format>.*)$', 'path.to.xhr_test'),
)
2 . Does format is json mean that data given to function is json or data which will be recived is json
This particular view doesn't do anything at all with the request body, and is certainly providing a response body in the supplied format
4 . How will I use that data when i get that response in jquery function
Depending on how complicated your request needs to be, you can use jQuery.getJSON, which will pass your callback with regular JavaScript objects that result from parsing the JSON. If you need to do a bit more work to get the request right, you can use jQuery.parseJSON to process the json data, and that will return the same JavaScript objects.
From the urlconf, just like it says in the article right below it.
The query set will be serialized as JSON.
It will be the query set represented as either XML or JSON. python manage.py shell is your friend.
You will decode it, then iterate over it.
You'll need to find some other format to serialize it in instead.