How to replace & with & in Ruby - json
I am sending Json to a confluence page but receive a BadRequest error. I have tested extensively and been able to send Json using the same code. The only difference between the json that will not send is that it contains an ampersand &. The Json that will send has escaped the ampersand &.
jsonHolder = {
"id": "....",
"type": "page"......
"body": {
"storage": {
"value": "<table><tbody><tr><td>Row1</td><td>Row2</td><td>Row3</td><td>Empty Sour & Tar</td></tr></tbody></table>",
},
}
This is only a sample of a much larger set and if possable I would only escape & as when I escape the overall json, formatting becomes an issue.
The json is being put into Net::HTTP::Put.new(uri) body.
p jsonHolder = JSON.generate(jsonResult)
request.body = jsonHolder
I have been trying to find a similar examples but many seem to be working on strings, but when I .to_s, the json is parsed and not correct.
Related
How to read invalid JSON format amazon firehose
I've got this most horrible scenario in where i want to read the files that kinesis firehose creates on our S3.
Kinesis firehose creates files that don't have every json object on a new line, but simply a json object concatenated file.
{"param1":"value1","param2":numericvalue2,"param3":"nested {bracket}"}{"param1":"value1","param2":numericvalue2,"param3":"nested {bracket}"}{"param1":"value1","param2":numericvalue2,"param3":"nested {bracket}"}
Now is this a scenario not supported by normal JSON.parse and i have tried working with following regex: .scan(/({((\".?\":.?)*?)})/)
But the scan only works in scenario's without nested brackets it seems.
Does anybody know an working/better/more elegant way to solve this problem?
The one in the initial anwser is for unquoted jsons which happens some times. this one:
({((\\?\".*?\\?\")*?)})
Works for quoted jsons and unquoted jsons
Besides this improved it a bit, to keep it simpler.. as you can have integer and normal values.. anything within string literals will be ignored due too the double capturing group.
https://regex101.com/r/kPSc0i/1
Modify the input to be one large JSON array, then parse that:
input = File.read("input.json")
json = "[#{input.rstrip.gsub(/\}\s*\{/, '},{')}]"
data = JSON.parse(json)
You might want to combine the first two to save some memory:
json = "[#{File.read('input.json').rstrip.gsub(/\}\s*\{/, '},{')}]"
data = JSON.parse(json)
This assumes that } followed by some whitespace followed by { never occurs inside a key or value in your JSON encoded data.
As you concluded in your most recent comment, the put_records_batch in firehose requires you to manually put delimiters in your records to be easily parsed by the consumers. You can add a new line or some special character that is solely used for parsing, % for example, which should never be used in your payload.
Other option would be sending record by record. This would be only viable if your use case does not require high throughput. For that you may loop on every record and load as a stringified data blob. If done in Python, we would have a dictionary "records" having all our json objects.
import json
def send_to_firehose(records):
firehose_client = boto3.client('firehose')
for record in records:
data = json.dumps(record)
firehose_client.put_record(DeliveryStreamName=<your stream>,
Record={
'Data': data
}
)
Firehose by default buffers the data before sending it to your bucket and it should end up with something like this. This will be easy to parse and load in memory in your preferred data structure.
[
{
"metadata": {
"schema_id": "4096"
},
"payload": {
"zaza": 12,
"price": 20,
"message": "Testing sendnig the data in message attribute",
"source": "coming routing to firehose"
}
},
{
"metadata": {
"schema_id": "4096"
},
"payload": {
"zaza": 12,
"price": 20,
"message": "Testing sendnig the data in message attribute",
"source": "coming routing to firehose"
}
}
]
json escape characters with back and forward slash
A part of my json schema is coming off with backward and forward slash after serialization.
My question is, is this a valid encoding? I'm having issues with the api too for some reason, so trying to see where the problem is.
"_links": {
"altAssetUrl": {
"href": "\/publication\/d40a4e4c-d6a3-45ae-98b3-924b31d8712a\/altasset\/48baad57-81a5-4d32-a2a1-e52c5cbe964d\/"
},
"contentUrl": {
"href": "\/publication\/d40a4e4c-d6a3-45ae-98b3-924b31d8712a\/article\/test\/contents;contentVersion=1521071354969\/"
}
},
In another area I noticed special characters %2F etc.
"socialShareUrl": "https:\/\/example.com\/ssp?entityRef=%2Fpublication%2Fd40a4e4c-d6a3-45ae-98b3-924b31d8712a%2Farticle%2Ftest",
Please advise on what can I do to fix this escaping of slashes, I'm using .net language.
My question is, is this a valid encoding?
Yes.
var json = '"This has a slash\\/"';
console.log("Raw JSON: " + json);
var str = JSON.parse(json);
console.log("String result of parsing JSON: " + str);
In another area I noticed special characters %2F etc.
Perfectly normal URL encoding.
How to express "arbitrary JSON" in swagger-spec?
Let's say I have a REST service that can accept any arbitrary JSON in the request body. How do I model this using swagger-spec?
I thought about Model Objects, but I could only think to wrap the arbitrary JSON (as a string) within a container JSON object, like {"payload": "{ some JSON object serialized to a string }"}, which isn't really useful.
Or, is there some other way to express that an endpoint can receive arbitrary JSON in the request body?
Model the request's body payload as parameters with schema of just "type": "object". The swagger UI editor will then prompt the user with a large textarea containing {} which they can populate with a JSON object.
"/endpoint": {
"post": {
"parameters": [
{
"description": "Arbitrary JSON object payload",
"in": "body",
"name": "body",
"required": true,
"schema": {
"type": "object"
}
}
]
}
Swagger tries to be deterministic when it comes to APIs, so what you're asking is not directly supported. The only way I can think of to achieve what you want is to set the "consumes" property to "application/json" and add a "body" parameter of type string. This would in theory say that only JSON should be sent, but in effect, any string could be sent. Also, this may break some third party tools if they'd try to convert to string to a JSON object before sending it to the server.
Invalid JSON but validates on JSONLint
I have the following JSON which validates on JSONLint.com but when I pass it to JSON.parse() I get the error
SyntaxError: JSON.parse: unexpected character
...0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,...
This is apparently the last "correct": line
var theJSON = JSON.parse({
"data": [
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
"wrong": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
"correct": "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
}
]
});
You, like many, have confused JavaScript's literal syntax with JSON. This happens a lot as JSON uses a subset of JavaScript's literal syntax so it looks a lot alike. JSON, however, is always a string. It is a serialized data scheme for porting data structures between langs/platforms.
Also confusing is that a string of JSON which has been output by any platform can be copied and pasted right into JavaScript and used. Again, this is because of the shared syntax. Having pasted such output right into JavaScript, however, one is no longer using JSON--they are now writing JavaScript in literal syntax. That is, unless, you pasted it between quotes and properly escaped the resulting string. But there's no sense in doing so as then it needs to be parsed in order to end up with what you already had.
JSON.parse() is a method for unserializing data which had been serialized into JSON. It expects a string because, well, JSON is a string. You're passing an object (in literal syntax). It does not need parsing...it is already the thing you want.
Wrapping your object literal in single quotes would make the code work, but it would be pointless to do so as the parse would simply result in what you already have.
Your code would be better written if you replaced the variable named theJSON with one named theObject and made it look as such:
var theObject = {
data: [
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
},
{
wrong: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0",
correct: "0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0"
}
]
};
Whatever code wanted to use the parse result should be fine once you've done it.
Passing apostrophe as part of JSON string
I have a problem that my JSON service is not being called, due to bad format probably.
Still, I dont understand what is wrong it it. I read about it and found out that apostrophes should not be escaped. Also when I escape them, it doesnt work.
"{
"fields": [
{
"Text": "PaymentReminders",
"Value": "'yes'"
}
]
}"
And yes, I really need 'yes' to be under apostrophes.
I am expecting a String on server side, which I then deserialize. It works without apostrophes.
Thanks!
edit1:
This is the structure that accepts in on the server:
Public Class TemplateField
Public Property Value() As String = "val"
Public Property Text() As String = "tex"
End Class
Public Class FieldsList
Public Property fields() As TemplateField()
End Class
and it gets deserialzed like this:
Dim jsSerializer As New JavaScriptSerializer
Dim fieldsArray As EventInfoDetails.FieldsList
fieldsArray = jsSerializer.Deserialize(Of EventInfoDetails.FieldsList)(fields)
and all that works, unless it contains apostrophes. Like I cannot stick apostrophe inside a string.
JSON does not only not require to escape apostrophes, but in fact it does not allow doing so (contrary to JavaScript). So your
"Value": "'yes'"
Is perfectly valid JSON. This is, unless you were inserting this JSON as a String literal inside JavaScript code, in which case it would be JavaScript the one requiring you to escape your ' as \' (you'd need two escapes, the JSON one and the JavaScript one on top of it).
Anyway, there's something strange about your code:
"{
"fields": [
{
"Text": "PaymentReminders",
"Value": "'yes'"
}
]
}"
Why is your entire JSON structure surrounded by quotes (")? Is it a string literal of any kind inside other programming language? In such case, you might need to follow that language's escaping rules for those quote (") symbols. Both Java and VB, for example, would use \" there...