below my json :
"temperature": {
"level": 8,
"format": function (value) {
return value + ' (C°)';
},
"minimum": null,
...
...
}
My goal is to write key format, I have try this but without much conviction...
package require rl_json
namespace import rl_json::json
set rj "{}"
set s1 {function (value) {
return value + ' (C°)';
}
}
json set rj temperature [json object [list level {number 8} format [json template {{"~L:s1"}}] minimum {null}]]
Error parsing JSON value: Expecting : after object key at offset 8
What you are trying to produce is not (partial) JSON, as defined, but rather general Javascript. As such, you cannot reasonably expect JSON-specific tooling (such as rl_json) to work with it. It is usually better to keep your JSON and your Javascript separated, with the data in JSON (including any dynamic generation) and your Javascript as a static artefact (or generated from something like Typescript). Mixing code and dynamic generation requires great care to get right; it is usually better to try to write things so you don't need to be that careful!
If you must do this defintely-not-advised thing, use something like subst or format to inject the variable parts (which you can generate with rl_json) in the overall string. No code for that from me: my advice is don't do that.
Related
I need to pass a json string as a value to one parameter of a POST request body. My request body looks like this:
"parameter1":"abc",
"parameter2":"def",
"parameter3": "{\"id\":\"\",\"key1\":\"test123\",\"prod1\":{\"id\":\"\",\"key3\":\"test123\",\"key4\":\"12334\",\"key5\":\"3\",\"key6\":\"234334\"},\"prod2\":{\"id\":\"\",\"key7\":\"test234\",\"key8\":1,\"key9\":true}}\"",
"parameter4":false,
"parameter5":"ghi"
}
For parameter3 I need to be pass a string value in json format. The json file is located in my local system and is a huge file, so it would make sense if I can pass it as a jmeter variable. I tried as below:
{
"parameter1":"abc",
"parameter2":"def",
"parameter3": "${jsonObj}",
"parameter4":false,
"parameter5":"ghi"
}
after adding a JSR223 preprocessor with the code below:
import org.apache.jmeter.util.JMeterUtils;
String fileContents = new File("path to json//myJson.json").getText('UTF-8');
vars.put("fileContents",fileContents);
var deltaJson = vars.get("fileContents");
var jsonObj = JSON.parse(deltaJson);
vars.put("jsonObj", JSON.stringify(jsonObj));
But I get below error:
exceptions":{"exceptionType":"System.JSONException","exceptionMessage":"Unexpected character ('$' (code 36)): expected a valid value (number, String, array, object, 'true', 'false' or 'null') at input location [1,2]"}
Can anyone help me in resolving this issue?
There is an easier way of doing this, JMeter comes with __FileToString() function so you can achieve the same much faster and without having to do any scripting
Something like:
{
"parameter1": "abc",
"parameter2": "def",
"parameter3": ${__FileToString(path to json//myJson.json,UTF-8,jsonObj)},
"parameter4": false,
"parameter5": "ghi"
}
Also be aware of the following facts:
the recommended language for using in JSR223 Test Elements is Groovy as it provides the maximum performance
you seem to be using JSON object which cannot be used outside of the browser context therefore your code fails to generate proper JSON hence your request fails as you're passing ${jsonObj} as it is, the substitution doesn't happen, you can look to jmeter.log file yourself and see the exact reason of your script failure
I've been playing with Kotlinx.serialisation. I've been trying to find a quick way to use Kotlinx.serialisation to create a plain simple JSON (mostly to send it away), with minimum code clutter.
For a simple string such as:
{"Album": "Foxtrot", "Year": 1972}
I've been doing is something like:
val str:String = Json.stringify(mapOf(
"Album" to JsonPrimitive("Foxtrot"),
"Year" to JsonPrimitive(1972)))
Which is far from being nice. My elements are mostly primitive, so I wish I had something like:
val str:String = Json.stringify(mapOf(
"Album" to "Sergeant Pepper",
"Year" to 1967))
Furthermore, I'd be glad to have a solution with a nested JSON. Something like:
Json.stringify(JsonObject("Movies", JsonArray(
JsonObject("Name" to "Johnny English 3", "Rate" to 8),
JsonObject("Name" to "Grease", "Rate" to 1))))
That would produce:
{
"Movies": [
{
"Name":"Johnny English 3",
"Rate":8
},
{
"Name":"Grease",
"Rate":1
}
]
}
(not necessarily prettified, even better not)
Is there anything like that?
Note: It's important to use a serialiser, and not a direct string such as
"""{"Name":$name, "Val": $year}"""
because it's unsafe to concat strings. Any illegal char might disintegrate the JSON! I don't want to deal with escaping illegal chars :-(
Thanks
Does this set of extension methods give you what you want?
#ImplicitReflectionSerializer
fun Map<*, *>.toJson() = Json.stringify(toJsonObject())
#ImplicitReflectionSerializer
fun Map<*, *>.toJsonObject(): JsonObject = JsonObject(map {
it.key.toString() to it.value.toJsonElement()
}.toMap())
#ImplicitReflectionSerializer
fun Any?.toJsonElement(): JsonElement = when (this) {
null -> JsonNull
is Number -> JsonPrimitive(this)
is String -> JsonPrimitive(this)
is Boolean -> JsonPrimitive(this)
is Map<*, *> -> this.toJsonObject()
is Iterable<*> -> JsonArray(this.map { it.toJsonElement() })
is Array<*> -> JsonArray(this.map { it.toJsonElement() })
else -> JsonPrimitive(this.toString()) // Or throw some "unsupported" exception?
}
This allows you to pass in a Map with various types of keys/values in it, and get back a JSON representation of it. In the map, each value can be a primitive (string, number or boolean), null, another map (representing a child node in the JSON), or an array or collection of any of the above.
You can call it as follows:
val json = mapOf(
"Album" to "Sergeant Pepper",
"Year" to 1967,
"TestNullValue" to null,
"Musicians" to mapOf(
"John" to arrayOf("Guitar", "Vocals"),
"Paul" to arrayOf("Bass", "Guitar", "Vocals"),
"George" to arrayOf("Guitar", "Sitar", "Vocals"),
"Ringo" to arrayOf("Drums")
)
).toJson()
This returns the following JSON, not prettified, as you wanted:
{"Album":"Sergeant Pepper","Year":1967,"TestNullValue":null,"Musicians":{"John":["Guitar","Vocals"],"Paul":["Bass","Guitar","Vocals"],"George":["Guitar","Sitar","Vocals"],"Ringo":["Drums"]}}
You probably also want to add handling for some other types, e.g. dates.
But can I just check that you want to manually build up JSON in code this way rather than creating data classes for all your JSON structures and serializing them that way? I think that is generally the more standard way of handling this kind of stuff. Though maybe your use case does not allow that.
It's also worth noting that the code has to use the ImplicitReflectionSerializer annotation, as it's using reflection to figure out which serializer to use for each bit. This is still experimental functionality which might change in future.
I'm having troubles with the JSON classes of the CPP REST SDK. I can't figure out when to use json::value, json::object and json::array. Especially the latter two seem very alike. Also the usage of json::array is rather unintuitive to me. Finally I want to write the JSON to a file or at least to stdcout, so I can check it is correct.
It was way easier for me to use json-spirit, but since I want to make REST requests later on I thought I'd save me the string/wstring madness and use the json classes of the CPP REST SDK.
What I want to achieve is a JSON file like this:
{
"foo-list" : [
{
"bar" : "value1",
"bob" : "value2"
}
]
}
This is the code I tried:
json::value arr;
int i{0};
for(auto& thing : things)
{
json::value obj;
obj[L"bar"] = json::value::string(thing.first);
obj[L"bob"] = json::value::string(thing.second);
arr[i++] = obj;
}
json::value result;
result[L"foo-list"] = arr;
Do I really need this extra counter variable i? Seems rather inelegant. Would using json::array/json::object make things nicer? And how do I write my JSON to a file?
This could help you:
json::value output;
output[L"foo-list"][L"bar"] = json::value::string(utility::conversions::to_utf16string("value1"));
output[L"foo-list"][L"bob"] = json::value::string(utility::conversions::to_utf16string("value2"));
output[L"foo-list"][L"bobList"][0] = json::value::string(utility::conversions::to_utf16string("bobValue1"));
output[L"foo-list"][L"bobList"][1] = json::value::string(utility::conversions::to_utf16string("bobValue1"));
output[L"foo-list"][L"bobList"][2] = json::value::string(utility::conversions::to_utf16string("bobValue1"));
If you want to create list, like bobList, you really need to use some iterator variable.
Otherwise you will get only bunch of separate variables.
For output to console use
cout << output.serialize().c_str();
And finally, this will lead to
{
"foo-list":{
"bar":"value1",
"bob":"value2",
"bobList":[
"bobValue1",
"bobValue1",
"bobValue1"
]
}
}
-- To answer your first question, in arrays, JSON values are stored at ordered indices. Thus, arrays can be traversed efficiently compared to objects as objects are more like hashmaps where the input key goes through hashing mechanism every time and a match within a hashtable is found to reach the value of that key. Thus, arrays are efficient especially when we are trying to traverse through a large JSON.
-- To answer your second question. As you mentioned you need to create something like this.
{
"foo-list" : [
{
"bar" : "value1",
"bob" : "value2"
}
]
}
-- If we were to have json::object with these two json values {"bar":"value1"} and {"bob":"value2"} as the value of the key foo-list,(if we were to have curly braces instead of the square above) it can be implemented as
result[U("foo-list")][U("bar")] = "value1";
result[U("foo-list")][U("bob")] = "value2";
-- But here json::object with these two JSON values {"bar":"value1"} and {"bob":"value2"} is at index 0 of a json::array; and this array is the value of the key foo-list. Thus you need the index variable to implement something like
result[U("foo-list")][0][U("bar")] = "value1";
result[U("foo-list")][0][U("bob")] = "value2";
-- To answer your third question, as correctly pointed out by #Zdeno you can use serialize to convert json::value to string and dump it to the file
I am writing a simple perl script to read JSON from a file and insert into MongoDB. But I am facing issues with json decoding.
All non-string values in my original json are getting converted to object type after decode_json.
Input JSON(only part of it since it's original is huge) -
{
"_id": 2006010100000801089,
"show_image" : false,
"event" : "publish",
"publish_date" :1136091600,
"data_version" : 1
}
JSON that gets inserted to MongoDB -
{
"_id": NumberLong("2006010100000801089"),
"show_image" : BinData(0,"MA=="),
"event" : "publish",
"publish_date" :NumberLong(1136091600),
"data_version" : NumberLong(1)
}
I am providing the custom _id for the documents, which I want to get converted to NumberLong type. That is working as expected as you can see from the JSON above. But notice how other non-string values for show_image, publish_date and data_version got converted to it's object representation.
Is there any way I can retain the original type for these values?
Perl code snipper that does the insert -
use MongoDB;
use MongoDB::OID;
use JSON;
use JSON::XS
while(my $record = <$source_file>) {
my $record_decoded = decode_json($record);
$db_collection->insert($record_decoded);
}
Perl version used v5.18.2.
I looked up JSON::XS docs but couldn't find a way to do this. Any help is appreciated. Thanks in advance!
I am very new to perl. Sorry if this is a trivial question.
I am providing the custom _id for the documents, which I want to get converted to NumberLong type. That is working as expected as you can see from the JSON above. But notice how other non-string values for show_image, publish_date and data_version got converted to it's object representation.
From your example all of the data types are actually matching aside from the boolean value for show_image which is currently being converted to binary data.
It is expected that numeric types are displayed as NumberLong or NumberInt when queried from the mongo shell. The mongo shell uses JavaScript, which only has a single numeric type of Number (64-bit floating point). Shell helpers like NumberLong() and NumberInt() are used to represent values in MongoDB's BSON data types that do not have a native JavaScript equivalent.
Referring to my sample JSON, I want value of show_image to be inserted as false instead of BinData(0,"MA==") and publish_date to be inserted as 1136091600 instead of NumberLong(1136091600)
While it's OK to insert publish_date as a Unixtime if that suits your use case, you may find it more useful to use MongoDB's Date type instead. There are convenience methods for querying dates including Date Aggregation Operators. FYI, date fields will be displayed in the mongo shell with an ISODate() wrapper.
The boolean value for show_image definitely needs an assist, though.
If you use Data::Dumper to inspect the result from decode_json(), you will see that the show_image field is a blessed object:
'show_image' => bless( do{\(my $o = 0)}, 'JSON::PP::Boolean' )
In order to get the expected boolean value in MongoDB, the recommended approach in the MongoDB module docs is to use the boolean module (see: MongoDB::DataTypes).
I couldn't find an obvious built-in option for JSON or JSON::XS to support serialising booleans to something other than the JSON emulated boolean class, but one solution would be to use the Data::Clean::Base module which is part of the Data::Clean::JSON distribution.
Sample snippet (excluding the MongoDB set up):
use Data::Clean::Base;
use boolean;
my $cleanser = Data::Clean::Base->new(
'JSON::XS::Boolean' => ['call_func', 'boolean::boolean'],
'JSON::PP::Boolean' => ['call_func', 'boolean::boolean']
);
while (my $record = <$source_file>) {
my $record_decoded = decode_json($record);
$cleanser->clean_in_place($record_decoded);
$db_collection->insert($record_decoded);
}
Sample record as saved in MongoDB 3.0.2:
{
"_id": NumberLong("2006010100000801089"),
"event": "publish",
"data_version": NumberLong("1"),
"show_image": false,
"publish_date": NumberLong("1136091600")
}
JSON data contains only (double-precision) numbers, strings, and the special values true, false, and null. They can be arranged in arrays or "objects" (hashes).
The MongoDB engine is converting these basic types into something more complex, but the original values are available in the hash referred to by $record_decoded, like so
$record_decoded->{_id}
$record_decoded->{show_image}
$record_decoded->{event}
$record_decoded->{publish_date}
$record_decoded->{data_version}
Is that what you wanted?
The object serialization documentation (particularly allow_tags) in JSON::XS may do something like what you want. Note, though, that this is not a standard JSON feature and will only work with JSON::XS.
I am using Jackson to process JSON that comes in chunks in Hadoop. That means, they are big files that are cut up in blocks (in my problem it's 128M but it doesn't really matter).
For efficiency reasons, I need it to be streaming (not possible to build the whole tree in memory).
I am using a mixture of JsonParser and ObjectMapper to read from my input.
At the moment, I am using a custom InputFormat that is not splittable, so I can read my whole JSON.
The structure of the (valid) JSON is something like:
[ { "Rep":
{
"date":"2013-07-26 00:00:00",
"TBook":
[
{
"TBookC":"ABCD",
"Records":
[
{"TSSName":"AAA",
...
},
{"TSSName":"AAB",
...
},
{"TSSName":"ZZZ",
...
}
] } ] } } ]
The records I want to read in my RecordReader are the elements inside the "Records" element. The "..." means that there is more info there, which conforms my record.
If I have an only split, there is no problem at all.
I use a JsonParser for fine grain (headers and move to "Records" token) and then I use ObjectMapper and JsonParser to read records as Objects. For details:
configure(JsonParser.Feature.AUTO_CLOSE_SOURCE, false);
MappingJsonFactory factory = new MappingJsonFactory();
mapper = new ObjectMapper(factory);
mapper.configure(Feature.FAIL_ON_UNKNOWN_PROPERTIES,false);
mapper.configure(SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS,false);
parser = factory.createJsonParser(iStream);
mapper.readValue(parser, JsonNode.class);
Now, let's imagine I have a file with two inputsplits (i.e. there are a lot of elements in "Records").
The valid JSON starts on the first split, and I read and keep the headers (which I need for each record, in this case the "date" field).
The split would cut anywhere in the Records array. So let's assume I get a second split like this:
...
},
{"TSSName":"ZZZ",
...
},
{"TSSName":"ZZZ2",
...
}
] } ] } } ]
I can check before I start parsing, to move the InputStream (FSDataInputStream) to the beginning ("{" ) of the record with the next "TSSNAME" in it (and this is done OK). It's fine to discard the trailing "garbage" at the beginning. So we got this:
{"TSSName":"ZZZ",
...
},
{"TSSName":"ZZZ2",
...
},
...
] } ] } } ]
Then I handle it to the JsonParser/ObjectMapper pair seen above.
The first object "ZZZ" is read OK.
But for the next "ZZZ2", it breaks: the JSONParser complaints about malformed JSON. It is encountering a "," not being in an array. So it fails. And then I cannot keep on reading my records.
How could this problem be solved, so I can still be reading my records from the second (and nth) splits? How could I make the parser ignore these errors on the commas, or either let the parser know in advance it's reading contents of an array?
It seems it's OK just catching the exception: the parser goes on and it's able to keep on reading objects via the ObjectMapper.
I don't really like it - I would like an option where the parser could not throw Exceptions on nonstandard or even bad JSON. So I don't know if this fully answers the question, but I hope it helps.