I've come up with a doubt around JSON files.
So, we're building a test case for a GWT application. The data it feeds from is in JSON files generated from a SQL database.
When testing the methods that work with data, we do it from sources held in String files, so to keep integrity with the original data, we just clone the original JSON values in to a String with escape sequences.
The result of this being that if a JSON entry shows like this:
{"country":"India","study_no":87}
The parsed result will come up like this in order for our tools to recognise them:
"[" + "{\"country\":\"India\",\"study_no\":87}" + "]"
The way we do it now is taking each JSON object and putting it between "" in IntelliJ, which automatically parses all double quotes in to escape sequences. This is ok if we only wanted a few objects, but What if we wanted a whole dataset?
So my question is, does anyone know or has created an opensource script to automate this tedious task?
One thing you could do is to wrap window.escape() using JsInterop or JSNI. For example:
#JsType(isNative="true", name="window")
public class window {
public native String escape(String toBeEscape);
}
and then apply to your results.
Related
I have a Json pattern string in a text file, I have to pharse the below string like below and put it in to a external file.
Please let me know how this can be handled with Informatica Powercenter or Unix or Python?
{"CONTACTID":"3b2a25b2","ANI":"+16146748702","DNIS":"+18006081123","START_TIME":"01/22/2023 03:31:42","MODULE":[{"Name":"MainIVR","Time":"01/22/2023 03:31:42",Dialog":[{"name":"offer_Spanish","dialogeresult":"(|raw:7|R|7|1.0|nm=0|ni=0|2023/22/21 03:02:01)"}],"backend":[{"Time":"01/22/2023)"}],"END_STATE":"XC"}
In The above sample string the special charcters should be removed and the values should be assigned to the corresponding columns like below 2 o/p formats
Output:
CONTACTID, ANI, DNIS, START_TIME, MODULE, Time,Dialog,dialogeresult,END_STATE
3b2a25b2,+16146748702 +18006081123 01/22/2023 03:31:42,Name:MainIVR,
or
Output:
CONTACTID : 3b2a25b2
ANI:16146748702
DNI :+18006081123
I tried this to read thru Informatica powercenter and using the expression tranformations but nothing worked and tried with Python too.
For a start, your JSON is invalid. The opening double quotes for Dialog are missing and it's not properly closed - MODULE array is not closed and root is not closed. Here's the fixed JSON:
{"CONTACTID":"3b2a25b2","ANI":"+16146748702","DNIS":"+18006081123","START_TIME":"01/22/2023 03:31:42","MODULE":[{"Name":"MainIVR","Time":"01/22/2023 03:31:42","Dialog":[{"name":"offer_Spanish","dialogeresult":"(|raw:7|R|7|1.0|nm=0|ni=0|2023/22/21 03:02:01)"}],"backend":[{"Time":"01/22/2023)"}],"END_STATE":"XC"}]}
Use some JSON validation tool, like this one - it helps a lot.
Next, here's some starter code you may use to achieve the required result:
import json
# some JSON:
x = '{"CONTACTID":"3b2a25b2","ANI":"+16146748702","DNIS":"+18006081123","START_TIME":"01/22/2023 03:31:42","MODULE":[{"Name":"MainIVR","Time":"01/22/2023 03:31:42","Dialog":[{"name":"offer_Spanish","dialogeresult":"(|raw:7|R|7|1.0|nm=0|ni=0|2023/22/21 03:02:01)"}],"backend":[{"Time":"01/22/2023)"}],"END_STATE":"XC"}]}'
# parse x:
y = json.loads(x)
# the result is a Python dictionary:
print(y.keys())
You may test it on Replit
Finally regarding Informatica Powercenter - it is a terrible choice for complex string processing. You would need a Hierarchy Parser Transformation. Long story short: it's very tedious, but possible. I would highly recommend picking up a differen approach, if this is not a regular data loading process you will need to build.
I am currently trying to format a JSON object using LabVIEW and have ran into the issue where it adds additional quotation marks invalidating my JSON formatting. I have not found a way around this so I thought just formatting the string manually would be enough.
Here is the JSON object that I have:
{
"contentType":"application/json",
"content":{
"msgType":2,
"objects":"["cat","dog","bird"]",
"count":3
}
}
Here is the JSON object I want with the quotation marks removed.
{
"contentType":"application/json",
"content":{
"msgType":2,
"objects":["cat","dog","bird"],
"count":3
}
}
I am still not an expert with regex and using a regex tester I was only able to grab the "objects" and "count" fields but I would still feel I would have to utilize substrings to remove the quotation marks.
Example I am using (would use a "count" to find the start of the next field and work backwards from there)
"([objects]*)"
Additionally, all the other Regex I have been looking at removes all instances of quotation marks whereas I only need a specific area trimmed. Thus, I feel that a specific regex replace would be a much more elegant solution.
If there is a better way to go about this I am happy to hear any suggestions!
Your question suggests that the built-in LabVIEW JSON tools are insufficient for your use case.
The built-in library converts LabVIEW clusters to JSON in a one-shot approach. Bundle all your data into a cluster and then convert it to JSON.
When it comes to parsing JSON, you use the path input terminal and the default type terminals to control what data is parsed from a JSON string.
If you need to handle JSON in a manner similar to say JavaScript, I would recommend something like the JSONText Toolkit which is free to use (and distribute) under the BSD licence. This allows more complex and iterative building of JSON strings from LabVIEW types and has text-path style element access along with many more features.
The Output controls from both my examples are identical - although JSONText provides a handy Pretty Print vi.
After using a regex from one of the comments, I ended up with this regex which allowed me to match the array itself.
(\[(?:"[^"]*"|[^"])+\])
I was able to split the the JSON string into before match, match and after match and removed the quotation marks from the end of 'before match' and start of 'after match' and concatenated the strings again to form a new output.
This may be an odd question as it's specific to the JSON strings themselves, not the objects they represent. Given a 'pretty printed' JSON string (representing any JSON-encodable model), how would one reformat it to the 'compact' format?
My first thought was to not consider it JSON, but rather just a string, then use RegEx to remove duplicate spaces, remove newlines, etc., but that's not context aware so it risks affecting the keys and values portions of the JSON if you don't properly test that you're inside quotes.
My next thought was to try and construct an object from the JSON, but without a type to convert to, I'm not sure how to do that without manually parsing the values as 'ANY', then testing if they're an array, and recurse into it if they are, repeating the process. Then once I have the final object, serialize the result in compact form. However, that seems like a lot of overkill.
Is there an easier way to accomplish this? We're using Swift 4 if it helps.
UPDATE:
as pointed by #Mark A. Donohoe, this removes ALL whitespaces. so even though it looks coooooool, it's a dumb answer. don't fall for it.
i needed the same thing and i ended up creating a String extension:
extension String {
func toCompactJSON() -> String {
self.filter { !$0.isWhitespace && !$0.isNewline }
}
}
in my case though it was for testing purposes and it turned out to be useless as the order in which the Javascript object/arrays appear is not the same as while generated through the JSONEncoder.
I'm using jsoncpp to store information in JSON format. I now have the need to store a json string inside of another json string... In other words, I need to store a sub_item inside of an item.
Using jsoncpp to generate the JSON string, I get this...
{"id":"1","name":"Advil","sub_item":"{\"id\":\"2\",\"name\":\"Liquid Gel Advil\"}\n"}
Which works perfectly fine during runtime. However, when my program saves this information into a MySQL database (on exit) and then loads it back up when I restart the program, it loads the same JSON string from the MySQL database, but it now looks like this...
{"id":"1","name":"Advil","sub_item":"{"id":"2","name":"Liquid Gel Advil"}"}
Which is an invalid JSON string. I'm not sure why this is happening can someone please tell me what the heck is going on...
My MySQL query string reads like so:
UPDATE json_string_test SET jsonstring='{"id":"1","name":"Advil","sub_item":"{\"id\":\"2\",\"name\":\"Liquid Gel Advil\"}\n"}';
Upon further research I found out FastWriter was depreciated, and StreamWriterBuilder was the recommended writer. However, it still produced the same problem as FastWrtier....
I managed to rig a fix by doing the following...
1) Before saving to the database, I replaced ALL substrings matching \" to \\" in ONLY the child JSON string (with id 2).
2) Upon loading the JSON string, I replaced ALL substrings matching \\" to \" in ONLY the child JSON string (with id 2).
I don't understand why the heck I have to do this, so if anyone has a better solution or an explanation... I'd love to hear it.
I think you need to transpose your last 2 characters (assuming your posted string is verbatim). You have
Advil\"}\n"}
but I think you need
Advil\"}\n}"
I'm working with Scala in IntelliJ IDEA 15 and trying to parse a large twitter record json file and count the total number of hashtags. I am very new to Scala and the idea of functional programming. Each line in the json file is a json object (representing a tweet). Each line in the file starts like so:
{"in_reply_to_status_id":null,"text":"To my followers sorry..
{"in_reply_to_status_id":null,"text":"#victory","in_reply_to_screen_name"..
{"in_reply_to_status_id":null,"text":"I'm so full I can't move"..
I am most interested in a property called "entities" which contains a property called "hastags" with a list of hashtags. Here is an example:
"entities":{"hashtags":[{"text":"thewayiseeit","indices":[0,13]}],"user_mentions":[],"urls":[]},
I've browsed the various scala frameworks for parsing json and have decided to use json4s. I have the following code in my Scala script.
import org.json4s.native.JsonMethods._
var json: String = ""
for (line <- io.Source.fromFile("twitter38.json").getLines) json += line
val data = parse(json)
My logic here is that I am trying to read each line from twitter38.json into a string and then parse the entire string with parse(). The parse function is throwing an error claiming:
"Type mismatch, expected: Nothing, found:String."
I have seen examples that use parse() on strings that hold json objects such as
val jsontest =
"""{
|"name" : "bob",
|"age" : "50",
|"gender" : "male"
|}
""".stripMargin
val data = parse(jsontest)
but I have received the same error. I am coming from an object oriented programming background, is there something fundamentally wrong with the way I am approaching this problem?
You have most likely incorrectly imported dependencies to your Intellij project or modules into your file. Make sure you have the following lines imported:
import org.json4s.native.JsonMethods._
Even if you correctly import this module, parse(String: json) will not work for you, because you have incorrectly formed a json. Your json String will look like this:
"""{"in_reply_...":"someValue1"}{"in_reply_...":"someValues2"}"""
but should look as follows to be a valid json that can be parsed:
"""{{"in_reply_...":"someValue1"},{"in_reply_...":"someValues2"}}"""
i.e. you need starting and ending brackets for the json, and a comma between each line of tweets. Please read the json4s documenation for more information.
Although being almost 6 years old, I think this question deserves another try.
JSON format has a few misunderstandings in people's minds, especially how they are stored and how they are read back.
JSON documents, are stored as either a single object having all the other fields, or an array of multiple object possibly in same format. this second part is important because arrays in almost every programming language are defined by angle brackets and values separated by commas (note here I used a person object as my single value):
[
{"name":"John","surname":"Doe"},
{"name":"Jane","surname":"Doe"}
]
also note that everything except brackets, numbers and booleans are enclosed in quotes when written into file.
however, there is another use that is not official but preferred to transfer datasets easily where every object, or document as in nosql/mongo language, are stored in a new line like this:
{"name":"John","surname":"Doe"}
{"name":"Jane","surname":"Doe"}
so for the question, OP has a document written in this second form, but tries an algorithm written to read the first form. following code has few simple changes to achieve this, and the user must read the file knowing that:
var json: String = "["
for (line <- io.Source.fromFile("twitter38.json").getLines) json += line + ","
json=json.splitAt(json.length()-1)._1
json+= "]"
val data = parse(json)
PS: although #sbrannon, has the correct idea, the example he/she gave has mistakenly curly braces instead of angle brackets to surround the data.
EDIT: I have added json=json.splitAt(json.length()-1)._1 because the code above ends with a trailing comma which will cause parse error per the JSON format definition.