Using only the values of JSON data that is unmarshalled - json

I'm looking to take a list of ids and add them to a slice. However if I don't custom unmarshall the data, I cannot get to the values of the ids and it is just an array of JSON objects. And if I do custom unmarshall the data I want, I am told that I cannot append that slice-like array of data because it has a custom type from the structs I used to unmarshall.
I would prefer to only make the request once to get both the "data" and "meta" information from a rest API, I then unmarshall that data into the following:
{Data:[{Ids:39647} {Ids:39648} {Ids:39649} {Ids:39650} {Ids:39651} {Ids:39652} {Ids:39653} {Ids:39654} {Ids:39655} {Ids:39656} {Ids:39657} {Ids:39658} {Ids:39659} {Ids:39660} {Ids:39661} {Ids:39662} {Ids:39663} {Ids:39664} {Ids:39665} {Ids:39666} {Ids:39667} {Ids:39668} {Ids:39669} {Ids:39670} {Ids:39671} {Ids:39672} {Ids:39673}] Meta:{Metadata:{CurrentPage:3 TotalPages:656}}}
I would like to get the Ids in a slice like so:
[39647 39649 39650 ...]
Preferably without having to marshall and then unmarshall again, but beggars can't be choosers. This would then need to be of slice type []int so that I can append it to another slice and use all standard library slice things with it.
Edit: It was requested I add the basic JSON Structure to the question.
"data": [
{
"id": 38926
},
{
"id": 38927
},
//... and so on.
],
"meta": {
"pagination": {
"total": 163795,
"current_page": 3,
"total_pages": 81898
}
}
}

If you just want to get a straight slice of int-"id",without unmarshalling the whole JSON, you could use a JSON parser library to do that, ex-https://github.com/tidwall/gjson.
package main
import "github.com/tidwall/gjson"
json := `{
"data":[
{
"id":38926
},
{
"id":38927
}
],
"meta":{
"pagination":{
"total":163795,
"current_page":3,
"total_pages":81898
}
}
}`
func main() {
value := gjson.Get(json, "data.#.id")
println(value.Array()) // etc
}

Related

How to get the results of a TDE in MarkLogic in RDF/JSON or Turtle format?

With any kind of Template Driven Extraction (TDE) in MarkLogic, how can I convert the results I get from the tde:node-data-extract function into RDF/JSON format? The JSON format returned by this method is not compliant with RDF/JSON, so I can't use it directly to insert triples into another database. In this case, I don't want to insert the triples into the same database that I'm applying the template against, I just want to use the template to create triples from XML data.
Here's an example of the JSON output that I get from the tde:node-data-extract function:
{
"document/pt/document/39627370": [{
"triple": {
"subject": "http://www.example.com/document/id/39627370",
"predicate": "http://www.example.com/typeOf",
"object": {
"datatype": "http://www.w3.org/2001/XMLSchema#string",
"value": "http://www.example.com/document"
}
}
},
{
"triple": {
"subject": "http://www.example.com/publisher/Oxford_University_Press",
"predicate": "http://www.example.com/typeOf",
"object": {
"datatype": "http://www.w3.org/2001/XMLSchema#string",
"value": "http://www.example.com/publisher"
}
}
}
}
}
Convert each "triple" property into a triple object using sem.triple(). Then serialize the array of sem.triple objects using sem.rdfSerialize().
https://docs.marklogic.com/sem.triple
https://docs.marklogic.com/sem.rdfSerialize
With the help from John and Mads, I found a slight variation that works really well assuming you're in the query console. $docs is any sequence of documents and $template is the TDE template.
let $jsontriples := tde:node-data-extract($docs, $template)
for $key in map:keys($jsontriples)
let $entry := map:get($jsontriples, $key)
return $entry["triple"]
This will return the triples automatically serialized into Turtle format in the query console Result tab, which you can switch to JSON or Text. I assume the answer from John is the most correct in a situation where the serialization is not automatically performed (e.g. when not using the query console).

Is there a way to convert a JSON string into a Mongo extended JSON using Go?

I have a sample JSON body that contains some strings. I want some of the strings to be converted into Mongo Extended JSON. For example, the JSON body passed in is the following:
{
"GuidBinary": "734cba69-4851-4869-8d0e-e870d6fb3065",
"DateTime": "12/12/2012",
"RegularString": "abcd"
}
And I want to convert this into
{
"GuidBinary": {
"$binary": {
"base64": "<payload>",
"subType": 0x03
}
},
"DateTime": {"$date": "<ISO-8601 Date/Time Format>"},
"RegularString": "abcd"
}
Is there a way to do this in Go, either through a package like mongo-go-driver or another method?
Yes, it's possible. It's part of the official mongo-go driver. Generating this extended JSON is published as the bson.MarshalExtJSON() function.
Example using it:
m := map[string]interface{}{
"GuidBinary": []byte{1, 2, 3, 4, 5},
"DateTime": time.Now(),
"RegularString": "abcd",
}
out, err := bson.MarshalExtJSON(m, false, false)
fmt.Println(string(out), err)
This will output (try it on the Go Playground):
{"DateTime":{"$date":"2009-11-10T23:00:00Z"},"RegularString":"abcd",
"GuidBinary":{"$binary":{"base64":"AQIDBAU=","subType":"00"}}} <nil>
So what you need to do is unmarshal your original JSON (using the encoding/json package), and you need to do some post processing on it: GuidBinary is a regular string in the input JSON, but it represents a UUID. You need to parse it into a UUID value (there are probably lots of libraries for this).
You also need to convert (parse) the DateTime which is also given as a JSON text in the input, but it represents a date. Use time.Parse() for this. And now you can call bson.MarshalExtJSON() to generate the extended JSON.

Comparing Json data. Python 3

I have the following Json file and I need to compare data to see how many times each value repeat itself. The problem is, I have no idea about handling Json. I don't want the answer to my exercise, I want to know how to access the data. Json:
{
"tickets": [
{
"ticket_id": 0,
"timestamp": "2016/05/26 04:47:02",
"file_hash": "c9d4e03c5632416f",
"src_ip": "6.19.128.119",
"dst_ip": "145.231.76.44"
},
{
"ticket_id": 1,
"timestamp": "2017/05/28 16:14:22",
"file_hash": "ce8a056490a3fd3c",
"src_ip": "100.139.125.30",
"dst_ip": "145.231.76.44"
},
{
"ticket_id": 2,
"timestamp": "2015/08/23 03:27:10",
"file_hash": "d17f572496f48a11",
"src_ip": "67.153.41.75",
"dst_ip": "239.168.56.243"
},
{
"ticket_id": 3,
"timestamp": "2016/02/26 14:01:33",
"file_hash": "3b28f2abc966a386",
"src_ip": "6.19.128.119",
"dst_ip": "137.164.166.84"
},
]
}
If this is a string representation of the object, first you need to set a variable and parse the string to have object you can work with.
jsonString = "{...your json string...}"
Then parse the string,
import json
jsonObject = json.loads(jsonString)
To access the data within it's like any other js object. Example :
jsonObject.tickets[0].timestamp
would return "2016/05/26 04:47:02"
tickets is the key within the jsonObject, 0 is the index of the first object in the list of tickets.
You can use the built-in "json" library to parse your file into an object:
import json
f = open('myfile.json','r')
tickets = json.loads(f.read())
This will return a "tickets" object. How you "compare" (or what exactly you compare) is up to you.

Using struct with API call in Go

I'm new to Go and working hard to follow its style and I'm not sure how to proceed.
I want to push a JSON object to a Geckoboard leaderboard, which I believe requires the following format based on the API doc and the one for leaderboards specifically:
{
"api_key": "222f66ab58130a8ece8ccd7be57f12e2",
"data": {
"item": [
{ "label": "Bob", "value": 4, "previous_value": 6 },
{ "label": "Alice", "value": 3, "previous_value": 4 }
]
}
}
My instinct is to build a struct for the API call itself and another called Contestants, which will be nested under item. In order to use json.Marshall(Contestant1), the naming convention of my variables would not meet fmt's expectations:
// Contestant structure to nest into the API call
type Contestant struct {
label string
value int8
previous_rank int8
}
This feels incorrect. How should I configure my Contestant objects and be able to marshall them into JSON without breaking convention?
To output a proper JSON object from a structure, you have to export the fields of this structure. To do it, just capitalize the first letter of the field.
Then you can add some kind of annotations, to tell your program how to name your JSON fields :
type Contestant struct {
Label string `json:"label"`
Value int8 `json:"value"`
PreviousRank int8 `json:"previous_rank"`
}

How to add "data" and "paging" section on JSON marshalling

I know i can customize the JSON response registering JSON marshallers to Domain entities, even i can create profiles with names for different responses.
This is done filling an array that later will be marshalled like:
JSON.registerObjectMarshaller(myDomain) {
def returnArray = [:]
returnArray['id'] = it.id
returnArray['name'] = it.name
returnArray['price'] = it.price
return returnArray
}
What i want is to alter the way it gets marshalled to have two sections like
{
"paging": {
"total": 100
},
"data": [
{
"id": 1,
"description": "description 1",
}
},
...
]
}
I assume i have to implemetn a custom JSON Marshaller but i don't know how to use it for a specific response instead of wide application.
EDIT: I assume i'll need a custom RENDERER apart from the marshaller. Is this one that i don't know how to use for specific response.
What about a simple:
def json = new JSON([ paging: [ total: myArray.totalCount ], data: myArray ])
Your domain objects will be converted with the marshaller you have set up while your paging data will simply be transformed into JSON.