I have a server written in Rust, this server gets a request in JSON, the JSON the server is getting is a string and sometimes users write quotes inside the value. For example when making a new forum thread.
The only thing I really need to do is to escape the quotes inside the value.
So this:
"{"name":""test"", "username":"tomdrc1", "date_created":"07/12/2019", "category":"Developer", "content":"awdawdasdwd"}"
Needs to be turned into this:
"{"name":"\"test\"", "username":"tomdrc1", "date_created":"07/12/2019", "category":"Developer", "content":"awdawdasdwd"}"
I tried to replace:
let data = let data = "{"name":""test"", "username":"tomdrc1", "date_created":"07/12/2019", "category":"Developer", "content":"awdawdasdwd"}".to_string().replace("\"", "\\\"");
let res: serde_json::Value = serde_json::from_str(&data).unwrap();
But it results in the following error:
thread '' panicked at 'called Result::unwrap() on an Err value: Error("key must be a string", line: 1, column: 2)
I suspect because it transforms the string to the following:
let data = "{\"name\":\"\"test\"\", \"username\":\"tomdrc1\", \"date_created\":\"07/12/2019\", \"category\":\"Developer\", \"content\":\"awdawdasdwd\"}"
If I understand your question right, the issue is that you are receiving strings which should be JSON but are in fact malformed (perhaps generated by concatenating strings).
If you are unable to fix the source of those non-JSON strings the only solution I can think of involves a lot of heavy lifting with caveat:
Writing a custom "malformed-JSON" parser
Careful inspection/testing/analysis of how the broken client is broken
Using the brokenness information to fix the "malformed-JSON"
Using the fixed JSON to do normal request processing
I would recommend not to do that except for maybe a training excercise. Fixing the client will be done in minutes but implementing this perfectly on the server will take days or weeks. The next time this one problematic client has been changed you'll have to redo all the hard work.
The real answer:
Return "400 Bad Request" with some additional "malformed json" hint
Fix the client if you have access to it
Additional notes:
Avoid unwrapping in a server
Look for ways to propagate the Result::Err to caller and use it to trigger a "400 Bad Request" response
Check out error handling chapter in the Rust book for more
Related
I have a JSON file in Azure Blob storage that I need to parse and insert rows into SQL using the Logic App.
I am using the "Get Blob Content" and my first attempt was to then pass to "Parse JSON". It returns and error": InvalidTemplate. Unable to process template language expressions in action 'Parse_JSON' inputs at line '1' and column '2856'"
I found some discussion that indicated that the content needs to be converted to a string so I used "Compose" and edited the code as suggested to
"inputs": "#base64ToString(body('Get_blob_content').$content)"
This works but then the InvalidTemplate issue gets pushed to the Parse function and I get the InvalidTemplate error there. I have tried wrapping the output in JSON expression and a few other things but I just can't get it to parse.
If I take a sample or even the entire JSON and put it into the INPUT of the Parse function it works without issue but it will not accept the blob content as JSON.
The only thing I have been able to do successfully from blob content is to take it as a string and update a row in SQL to later use the OPENJSON in SQL...but I run into an issue there that is for another post.
I am at a loss of what to do.
You don't post much information about your logic app actions, so maybe you could refer to my flow design. I test with a json data with array.
The below is my flow picture. I'm not using compose action, and use decodeBase64(body('Get_blob_content')['$content']) as the Parse Json content.
And if select property from the json, you need set the array index. I set a variable to get a value 'body('Parse_JSON')1['name']'.
you could have a try with this, if still fail, please provide more information or some sample to let us have a test.
I have a stored proc that returns json. The proc works fine and when I call it in SSMS it returns a json object that I can click and see as valid json. When I call it through Visual Studio it does the same, and clicking into it and pressing return formats it nicely so it's readable (and not all on a single line). If I take the string and pass it back into the sql function IsJson, it also returns true, so I'm 100% confident that the json coming out of the proc is valid.
However, my frontend developer is unable to parse it, and it seems to boil down to something trying to convert it somewhere.
The json (as output from the proc) is:
{"APIResult":[{"ID":200,"Status_Message":"Success","Developer_Message":"Successful login for D56D10AC-3A74-42FC-8BB5-F783A6C0C556 33E4F907-F1E5-4F1B-8CA5-5521291E683A (AppID: (null)).","User_Message":"Successful login","User":[{"ID":"D56D10AC-3A74-42FC-8BB5-F783A6C0C556"}]}]}
Using Postman to hit the live API (or localhost) I get back the data I expect (as above) however, it wraps the entire thing in double quotes and escapes all the double quotes around each element, so I get:
"{\"APIResult\":[{\"ID\":200,\"Status_Message\":\"Success\",\"Developer_Message\":\"Successful login for D56D10AC-3A74-42FC-8BB5-F783A6C0C556 33E4F907-F1E5-4F1B-8CA5-5521291E683A (AppID: (null)).\",\"User_Message\":\"Successful login\",\"User\":[{\"ID\":\"D56D10AC-3A74-42FC-8BB5-F783A6C0C556\"}]}]}"
When I try to parse this in sql (using isjson) it returns false. However, parsing this same code though jsonlint.com interprets it as valid. Trying it through the parser my developer is using http://json.parser.online.fr throws it out as errors too.
At a push, I can tell my developer to strip the first and last characters and replace every \" with " but this seems a bit faffy.
Are there different interpretations of json? I've read that this is often caused by the calling app expecting a string that it needs to json-ify and because it's getting a json object it's json-ifying the json, but even if I force sql server to output the string as a string and not json, it still seems the same. SQL is definitely pushing it out correctly but whatever my developer is calling the api through returns the same format as postman, and doesn't like it. How can I ensure that what the calling code gets is what SQL is giving and not some weird interpretation?
#Tim Try to deserialize the output of stored procedure. Use JSonConvert.Derialize in C#
I am not able to extract quoted strings using JSON_EXTRACT function in MySQL 5.7.
Sample input:
{
"email": "d'souza#email.com",
"body": "I may have "random quotes '(single)/(double)" " in my source data"
}
Tried using,
SELECT
#valid_source := JSON_VALID(jsonString), NULL
IF(#valid_source, JSON_UNQUOTE(JSON_EXTRACT(jsonString, "$.email")), NULL)
I get an error stating: Invalid JSON text in argument 1 to function json_extract: "Missing a closing quotation mark in string." at position xxx
Any help will be appreciated, thanks!
Here is the fix that worked for me:
I used the operator "-->" instead of JSON_UNQUOTE(JSON_EXTRACT("jsonString")) and it did not throw me any error for any kind of quotes in my input string. Please note that the above Sample JSON was only one of the use cases that I was expecting in my input. I have around 4 million records, with all different combinations of characters and quotes since it contains the email bodies and using the operators instead of actual commands worked perfectly fine, it's weird since both are essentially the same but never the less I'm happy that I could resolve it using a small fix.
{
#valid_json := JSON_VALID(inputString),
IF(#valid_json, inputString ->> '$.email', NULL) AS EMAIL,
}
You are getting this error because the contents in the field, 'inputString' are invalid JSONs for some record. I think it's a mistake to blindly handle this error without persisting any error message for downstream / future users.
This approach may be acceptable (although I wouldn't accept this from a data engineer). A better solution would be to put a more informative error in the IF statement rather than a null, perhaps something like
CONCAT_WS(" ", 'INCORRECTLY FORMED JSON :', inputString), so that your overall function becomes
IF(valid_json(inputString), input_string ->> '$.email', CONCAT_WS(" ", 'INCORRECTLY FORMED JSON :', inputString)) as EMAIL.
That way any downstream users will be able to identify any underlying data quality issues that may be affecting the validity of conclusions. Along those lines, it may be worth having a completely separate field like
JSON_VALID(inputString) AS INPUT_JSON_VALID that you can use for easier downstream filtering.
When recaptcha makes the call to https://www.google.com/recaptcha/api2/userverify?k=
It comes back with this
)]}'
["uvresp",,,,2]
Granted with a valid k it comes back with a bit more. However the )]}' is clearly invalid json.
When I try to retrieve the response with grecaptcha.getResponse() I get an empty string.
Same result when using curl.
Any help would be appreciated.
Actually the value returned is not valid json but well parsed by the Google's API.
Is it a protection ? I don't know, but if you look at the javascript, you can find that:
var jm=function(a,b,c,d,e,g,h,l,r){this.xl=a;this.$c=c||"GET";this.Ka=d;this.Gg=e||null;this.Td=m(h)?h:1;this.ye=0;this.xh=this.Nh=!1;this.uh=b;this.Mh=g;this.md=l||"";this.Zb=!!r;this.Wf=null};f=jm.prototype;f.getUrl=function(){return this.xl};f.ug=function(){return this.$c};f.Ca=function(){return this.Ka};f.fi=function(){return this.Zb};f.bi=function(){return this.md};var nm=function(){G.call(this);this.nj=new hm(0,mm,1,10,5E3);H(this,this.nj);this.ad=0};x(nm,G);var mm=new Nh;nm.prototype.send=function(a){return new Lc(function(b,c){var d=String(this.ad++);this.nj.send(d,a.Uf.toString(),a.ug(),a.Ca(),mm,void 0,u(function(a,d){var h=d.target;if(Xk(h)){var l=a.ml;h.B?(h=h.B.responseText,0==h.indexOf(")]}'\n")&&(h=h.substring(5)),h=Hg(h)):h=void 0;b(new l(h))}else c(new om(a))},this,a))},this)};var om=function(a){y.call(this);this.request=a};x(om,y);
especially take a look at:
var l=a.ml;h.B?(h=h.B.responseText,0==h.indexOf(")]}'\n")&&(h=h.substring(5)),h=Hg(h)):h=void 0;`
The parser explicitly checks that the value begins by )]} and strips it.
I suggest you to just apply the same substring on the "json" string
I am getting text from MySQL via a laravel model and converting the content to JSON. Unfortunately the text contains new lines and carriage returns, is there any way to get the contents properly escaped?
Article::find(1)->toJSON();
Gives me an Ajax/JSON error in my view so I was wondering where the problem is. Am I either storing the content the wrong way or am I retrieving it the wrong way?
Thanks.
This is the JSON string I am getting for a test article:
{"id":22,"short_title":"Another test article","long_title":"Longer title for the test article mention in the short title","description":"This article describes a computer classroom where strange things happen and stuf","content":"This is a test article and I think this will generate the error I am looking for. <\/p>\r\n\r\n Maybe or maybe not. <\/p>\r\n","deleted_at":null,"created_at":"2014-04-25 09:10:45","updated_at":"2014-04-25 09:10:45","category_id":1,"zenra_link":"","published_on":null,"published":1,"source_url":"http:\/\/blog.livedoor.jp\/morisitakurumi\/archives\/51856280.html","source_title":"Some source article","source_date":null,"slug":"another-test-article","view_count":0}
The content portion is generated by a textarea that's running CKEditor and it gets saved to a MySQL MEDIUMTEXT field.
Next thing I do is that I inject this into my view to populate my backbone views like this:
<script>var initialData = JSON.parse('{{ $cJson }}');</script>
And that's where the console tells me: Uncaught SyntaxError: Unexpected Token. As suggested I tested the above string on jsonlint.com and it comes back as valid.
I figured out the problem:
<script>var initialData = JSON.parse('{{ $cJson }}');</script>
This is where the error happens. JSON.parse freaks out on the string I supply. If I remove JSON.parse it works like this:
<script>var initialData = {{ $cJson }};</script>
No more unexpected token errors. I was on the wrong track with the problem because I read on so many places that the carriage returns and new lines produce errors in the JSON format.