I have a problem with JSON. Before we discovered error, column 'data' in mysql database had a type VARCHAR(255), where serialized json stored. About 2 months it worked well, but when project started to grow, 255 chars - became insufficient. But we forget to change type to TEXT. Now we have a problem, that our serialized json is stripped to 255 chars and now is invalid. I dont care about lost data, but I need to make minimal parsable/valid json.
for ex:
data = '{"state_id":[null,20],"dispatcher_id":[null,6057525],"uir":[null,{"level":"2"'
I need to make it valid, like this
data = '{"state_id":[null,20],"dispatcher_id":[null,6057525],"uir":[null,{"level":"2"}]}'
add }]} at the end of json.
Is there any quick way to do it? Or I should write my own parser/fixer?
You can use jsonlist
it's a pure implementation in JavaScript for validating JSON.
According to the validation you'll know what is missing in the end and add the missing value.
you can also check a demo online
I've done this task, so I want to share my solution
Related
I'm using Screaming Frog as a way to extract data from a Json generated from an URL.
The Json generated is this form :
{"ville":[{"codePostal":"13009","ville":"VAUFREGE","popin":"ouverturePopin","zoneLivraison":"1300913982","url":""},{"codePostal":"13009","ville":"LES BAUMETTES","popin":"ouverturePopin","zoneLivraison":"1300913989","url":""},{"codePostal":"13009","ville":"MARSEILLE 9EME ARRON","popin":"ouverturePopin","zoneLivraison":"1300913209","url":""}]}
I'm using this regex in Custom > Extraction in Screaming Frog as a way to extract the values of "codePostal".
"codePostal":".*?"
Problem is it doesn't extract anything.
When I test my regex in regex101, it seems correct.
Do you have any clue about what is wrong ?
Thanks.
Regards.
Have you tried to save the output to understand what ScreamingFrog sees? It doesn't matter - not at the beginning - whether your RegEx works.
That said, don't forget that SF is a Java based tool hence it is the engine used by the reg ex, so make sure you test your regular expressions with the correct dialect.
You need to specify group extractors enclosed in parentheses. For instance in your example, you need to have ("codePostal":".*?") as extractor.
In addition if you simply want to extract the value, you could use the following instead.
"codePostal":"(.*?)"
It's not a problem with your Regular Expression. It seems to be that the problem is with the Content Type. ScreamingFrog isn't properly reading application/JSON content types for scraping. Hopefully they will fix this bug.
I'm using jsoncpp to store information in JSON format. I now have the need to store a json string inside of another json string... In other words, I need to store a sub_item inside of an item.
Using jsoncpp to generate the JSON string, I get this...
{"id":"1","name":"Advil","sub_item":"{\"id\":\"2\",\"name\":\"Liquid Gel Advil\"}\n"}
Which works perfectly fine during runtime. However, when my program saves this information into a MySQL database (on exit) and then loads it back up when I restart the program, it loads the same JSON string from the MySQL database, but it now looks like this...
{"id":"1","name":"Advil","sub_item":"{"id":"2","name":"Liquid Gel Advil"}"}
Which is an invalid JSON string. I'm not sure why this is happening can someone please tell me what the heck is going on...
My MySQL query string reads like so:
UPDATE json_string_test SET jsonstring='{"id":"1","name":"Advil","sub_item":"{\"id\":\"2\",\"name\":\"Liquid Gel Advil\"}\n"}';
Upon further research I found out FastWriter was depreciated, and StreamWriterBuilder was the recommended writer. However, it still produced the same problem as FastWrtier....
I managed to rig a fix by doing the following...
1) Before saving to the database, I replaced ALL substrings matching \" to \\" in ONLY the child JSON string (with id 2).
2) Upon loading the JSON string, I replaced ALL substrings matching \\" to \" in ONLY the child JSON string (with id 2).
I don't understand why the heck I have to do this, so if anyone has a better solution or an explanation... I'd love to hear it.
I think you need to transpose your last 2 characters (assuming your posted string is verbatim). You have
Advil\"}\n"}
but I think you need
Advil\"}\n}"
So I have been taking some XML and parsing it through xml2js package to get JSON. The parsing worked fine though I keep getting "$" for the list of attributes. For example:
<AddresseeInformation name="SOME COMPANY LTD"></AddresseeInformation>
Becomes
"AddresseeInformation":[{"$":{"name":"SOME COMPANY LTD"}}]
I dont want this as MongoJS is saying that key $ must not start with '$' when I try to upload it to the DB - so I am going to have to change every instance of "$" or figure out how to stop it from happening.
Here is the regex I have tried to change every instance:
JSONstring.replace('"$"'/g, '"init"');
.replace(/"$"/g, '"init"');
.replace(/'"$"'/g, '"init"');
None of that worked - so I am sending it out to you guys - also if anyone knows how to stop the XML attributes being parsed with a key of "$" I would love you forever.
This, under "Options", seems to suggest that creating the parser with new Parser({attrkey:"init"}) does what you want.
For the RegExp solution, the problem is that $ has a special meaning and you have to escape it: /"\$"/g. (Although technically it should probably be /"\$"\s*:/g in case a value happens to be "$". Or /(?<!\\)"\$"\s*:/g if you really want to be careful and exclude keys ending in "$. And RegExp probably isn't the best solution anyways.)
Got myself in a tricky situation. I'm using local storage to save values from a popup window, and then paste them into an input when focus returns to parent window.
But then something rather awkward takes place, when I try to store ';' separated values, is that I get only the 1st set, losing all the rest of the string. What makes it more bizarre is that after saving my value, I test by calling
alert('SELECTED : ' + localStorage.getItem('MyStr'));
the whole string is there... but on the script I retrieve this value, when i'm checking
alert(localStorage.getItem('MyStr'));
Only the 3rd set is there, i.e.: I store something like
abcdefg;123323;ffasfs;5445;iuiuifa;
but when I need to get it back, theres only
ffasfs
I could use some help then, I'm all new to this whole thing, and killing myself to get a website working.
Thanks in advance, sorry if my question looks stupid.
Store your values in localStorage as JSON strings. This may even help you build more complex objects for the future.
For now though... Just do:
localStorage.setItem("your key", JSON.stringify("abcdef;1234;whatever"));
This procedure will not only sanitize your input but also create oppertunity to store serialized objects in the future.
It's important to note that while JSON.stringify is pretty much supported everywhere... Not all browsers have it built in.
For those cases, check out json2.js.
Hope that helps.
I have some big json files that are slightly different in the types that the fields contain.
{ "a":"1" }
vs.
{ "a":1 }
When I unmarshal the second I get:
cannot unmarshal number into Go value of type string
However since these jsons are large I would like to have the actual field that is in error so I can fix them. The UnmarshalTypeError does not hold the Struct's field type.
Does anybody know of a way to get to field name? (not debugging I have a lot of different fields that err)
[EDIT]
I know how to solve the type conversion. What I need is a method to see what fields I need to apply that conversion to.
The short answer is that you can't.
However, to fix your problem, there is multiple solutions:
Dive into the json.Unmarshal source code to change its working and add the information you need: copy the function to a local package, do your edits, and use this function
Use a thrid-party tool to help you, for example a JSON validator compatible with JSON Schema: here is an online example, there is probably some better-suited tool
Now the UnmarshalTypeError, contains the field name.