I have a Json pattern string in a text file, I have to pharse the below string like below and put it in to a external file.
Please let me know how this can be handled with Informatica Powercenter or Unix or Python?
{"CONTACTID":"3b2a25b2","ANI":"+16146748702","DNIS":"+18006081123","START_TIME":"01/22/2023 03:31:42","MODULE":[{"Name":"MainIVR","Time":"01/22/2023 03:31:42",Dialog":[{"name":"offer_Spanish","dialogeresult":"(|raw:7|R|7|1.0|nm=0|ni=0|2023/22/21 03:02:01)"}],"backend":[{"Time":"01/22/2023)"}],"END_STATE":"XC"}
In The above sample string the special charcters should be removed and the values should be assigned to the corresponding columns like below 2 o/p formats
Output:
CONTACTID, ANI, DNIS, START_TIME, MODULE, Time,Dialog,dialogeresult,END_STATE
3b2a25b2,+16146748702 +18006081123 01/22/2023 03:31:42,Name:MainIVR,
or
Output:
CONTACTID : 3b2a25b2
ANI:16146748702
DNI :+18006081123
I tried this to read thru Informatica powercenter and using the expression tranformations but nothing worked and tried with Python too.
For a start, your JSON is invalid. The opening double quotes for Dialog are missing and it's not properly closed - MODULE array is not closed and root is not closed. Here's the fixed JSON:
{"CONTACTID":"3b2a25b2","ANI":"+16146748702","DNIS":"+18006081123","START_TIME":"01/22/2023 03:31:42","MODULE":[{"Name":"MainIVR","Time":"01/22/2023 03:31:42","Dialog":[{"name":"offer_Spanish","dialogeresult":"(|raw:7|R|7|1.0|nm=0|ni=0|2023/22/21 03:02:01)"}],"backend":[{"Time":"01/22/2023)"}],"END_STATE":"XC"}]}
Use some JSON validation tool, like this one - it helps a lot.
Next, here's some starter code you may use to achieve the required result:
import json
# some JSON:
x = '{"CONTACTID":"3b2a25b2","ANI":"+16146748702","DNIS":"+18006081123","START_TIME":"01/22/2023 03:31:42","MODULE":[{"Name":"MainIVR","Time":"01/22/2023 03:31:42","Dialog":[{"name":"offer_Spanish","dialogeresult":"(|raw:7|R|7|1.0|nm=0|ni=0|2023/22/21 03:02:01)"}],"backend":[{"Time":"01/22/2023)"}],"END_STATE":"XC"}]}'
# parse x:
y = json.loads(x)
# the result is a Python dictionary:
print(y.keys())
You may test it on Replit
Finally regarding Informatica Powercenter - it is a terrible choice for complex string processing. You would need a Hierarchy Parser Transformation. Long story short: it's very tedious, but possible. I would highly recommend picking up a differen approach, if this is not a regular data loading process you will need to build.
Related
Am having trouble identifying the propert format to store a json request body in csv format, then use the csv file value in a scenario.
This works properly within a scenario:
And request '{"contextURN":"urn:com.myco.here:env:booking:reservation:0987654321","individuals":[{"individualURN":"urn:com.myco.here:env:booking:reservation:0987654321:individual:12345678","name":{"firstName":"NUNYA","lastName":"BIDNESS"},"dateOfBirth":"1980-03-01","address":{"streetAddressLine1":"1 Myplace","streetAddressLine2":"","city":"LANDBRANCH","countrySubdivisionCode":"WV","postalCode":"25506","countryCode":"USA"},"objectType":"INDIVIDUAL"},{"individualURN":"urn:com.myco.here:env:booking:reservation:0987654321:individual:23456789","name":{"firstName":"NUNYA","lastName":"BIZNESS"},"dateOfBirth":"1985-03-01","address":{"streetAddressLine1":"1 Myplace","streetAddressLine2":"","city":"BRANCHLAND","countrySubdivisionCode":"WV","postalCode":"25506","countryCode":"USA"},"objectType":"INDIVIDUAL"}]}'
However, when stored in csv file as follows (I've tried quite a number other formatting variations)
'{"contextURN":"urn:com.myco.here:env:booking:reservation:0987654321","individuals":[{"individualURN":"urn:com.myco.here:env:booking:reservation:0987654321:individual:12345678","name":{"firstName":"NUNYA","lastName":"BIDNESS"},"dateOfBirth":"1980-03-01","address":{"streetAddressLine1":"1 Myplace","streetAddressLine2":"","city":"LANDBRANCH","countrySubdivisionCode":"WV","postalCode":"25506","countryCode":"USA"},"objectType":"INDIVIDUAL"},{"individualURN":"urn:com.myco.here:env:booking:reservation:0987654321:individual:23456789","name":{"firstName":"NUNYA","lastName":"BIZNESS"},"dateOfBirth":"1985-03-01","address":{"streetAddressLine1":"1 Myplace","streetAddressLine2":"","city":"BRANCHLAND","countrySubdivisionCode":"WV","postalCode":"25506","countryCode":"USA"},"objectType":"INDIVIDUAL"}]}',
and used in scenario as:
And request requestBody
my test returns an "javascript evaluation failed: " & the json above & :1:63 Missing close quote ^ in at line number 1 at column number 63
Can you please identify correct formatting or the usage errors I am missing? Thanks
We just use a basic CSV library behind the scenes. I suggest you roll your own Java helper class that does whatever processing / pre-processing you need.
Do read this answer as well: https://stackoverflow.com/a/54593057/143475
I can't make sense of your JSON but if you are trying to fit JSON into CSV, sorry - that's not a good idea. See this answer: https://stackoverflow.com/a/62449166/143475
I'm using jsoncpp to store information in JSON format. I now have the need to store a json string inside of another json string... In other words, I need to store a sub_item inside of an item.
Using jsoncpp to generate the JSON string, I get this...
{"id":"1","name":"Advil","sub_item":"{\"id\":\"2\",\"name\":\"Liquid Gel Advil\"}\n"}
Which works perfectly fine during runtime. However, when my program saves this information into a MySQL database (on exit) and then loads it back up when I restart the program, it loads the same JSON string from the MySQL database, but it now looks like this...
{"id":"1","name":"Advil","sub_item":"{"id":"2","name":"Liquid Gel Advil"}"}
Which is an invalid JSON string. I'm not sure why this is happening can someone please tell me what the heck is going on...
My MySQL query string reads like so:
UPDATE json_string_test SET jsonstring='{"id":"1","name":"Advil","sub_item":"{\"id\":\"2\",\"name\":\"Liquid Gel Advil\"}\n"}';
Upon further research I found out FastWriter was depreciated, and StreamWriterBuilder was the recommended writer. However, it still produced the same problem as FastWrtier....
I managed to rig a fix by doing the following...
1) Before saving to the database, I replaced ALL substrings matching \" to \\" in ONLY the child JSON string (with id 2).
2) Upon loading the JSON string, I replaced ALL substrings matching \\" to \" in ONLY the child JSON string (with id 2).
I don't understand why the heck I have to do this, so if anyone has a better solution or an explanation... I'd love to hear it.
I think you need to transpose your last 2 characters (assuming your posted string is verbatim). You have
Advil\"}\n"}
but I think you need
Advil\"}\n}"
I've come up with a doubt around JSON files.
So, we're building a test case for a GWT application. The data it feeds from is in JSON files generated from a SQL database.
When testing the methods that work with data, we do it from sources held in String files, so to keep integrity with the original data, we just clone the original JSON values in to a String with escape sequences.
The result of this being that if a JSON entry shows like this:
{"country":"India","study_no":87}
The parsed result will come up like this in order for our tools to recognise them:
"[" + "{\"country\":\"India\",\"study_no\":87}" + "]"
The way we do it now is taking each JSON object and putting it between "" in IntelliJ, which automatically parses all double quotes in to escape sequences. This is ok if we only wanted a few objects, but What if we wanted a whole dataset?
So my question is, does anyone know or has created an opensource script to automate this tedious task?
One thing you could do is to wrap window.escape() using JsInterop or JSNI. For example:
#JsType(isNative="true", name="window")
public class window {
public native String escape(String toBeEscape);
}
and then apply to your results.
I'm working with Scala in IntelliJ IDEA 15 and trying to parse a large twitter record json file and count the total number of hashtags. I am very new to Scala and the idea of functional programming. Each line in the json file is a json object (representing a tweet). Each line in the file starts like so:
{"in_reply_to_status_id":null,"text":"To my followers sorry..
{"in_reply_to_status_id":null,"text":"#victory","in_reply_to_screen_name"..
{"in_reply_to_status_id":null,"text":"I'm so full I can't move"..
I am most interested in a property called "entities" which contains a property called "hastags" with a list of hashtags. Here is an example:
"entities":{"hashtags":[{"text":"thewayiseeit","indices":[0,13]}],"user_mentions":[],"urls":[]},
I've browsed the various scala frameworks for parsing json and have decided to use json4s. I have the following code in my Scala script.
import org.json4s.native.JsonMethods._
var json: String = ""
for (line <- io.Source.fromFile("twitter38.json").getLines) json += line
val data = parse(json)
My logic here is that I am trying to read each line from twitter38.json into a string and then parse the entire string with parse(). The parse function is throwing an error claiming:
"Type mismatch, expected: Nothing, found:String."
I have seen examples that use parse() on strings that hold json objects such as
val jsontest =
"""{
|"name" : "bob",
|"age" : "50",
|"gender" : "male"
|}
""".stripMargin
val data = parse(jsontest)
but I have received the same error. I am coming from an object oriented programming background, is there something fundamentally wrong with the way I am approaching this problem?
You have most likely incorrectly imported dependencies to your Intellij project or modules into your file. Make sure you have the following lines imported:
import org.json4s.native.JsonMethods._
Even if you correctly import this module, parse(String: json) will not work for you, because you have incorrectly formed a json. Your json String will look like this:
"""{"in_reply_...":"someValue1"}{"in_reply_...":"someValues2"}"""
but should look as follows to be a valid json that can be parsed:
"""{{"in_reply_...":"someValue1"},{"in_reply_...":"someValues2"}}"""
i.e. you need starting and ending brackets for the json, and a comma between each line of tweets. Please read the json4s documenation for more information.
Although being almost 6 years old, I think this question deserves another try.
JSON format has a few misunderstandings in people's minds, especially how they are stored and how they are read back.
JSON documents, are stored as either a single object having all the other fields, or an array of multiple object possibly in same format. this second part is important because arrays in almost every programming language are defined by angle brackets and values separated by commas (note here I used a person object as my single value):
[
{"name":"John","surname":"Doe"},
{"name":"Jane","surname":"Doe"}
]
also note that everything except brackets, numbers and booleans are enclosed in quotes when written into file.
however, there is another use that is not official but preferred to transfer datasets easily where every object, or document as in nosql/mongo language, are stored in a new line like this:
{"name":"John","surname":"Doe"}
{"name":"Jane","surname":"Doe"}
so for the question, OP has a document written in this second form, but tries an algorithm written to read the first form. following code has few simple changes to achieve this, and the user must read the file knowing that:
var json: String = "["
for (line <- io.Source.fromFile("twitter38.json").getLines) json += line + ","
json=json.splitAt(json.length()-1)._1
json+= "]"
val data = parse(json)
PS: although #sbrannon, has the correct idea, the example he/she gave has mistakenly curly braces instead of angle brackets to surround the data.
EDIT: I have added json=json.splitAt(json.length()-1)._1 because the code above ends with a trailing comma which will cause parse error per the JSON format definition.
I'm using webwrite to post to an api. One of the field names in the json object I'm trying to setup for posting is odata.metadata. I'm making a struct that looks like this for the json object:
json = struct('odata.metadata', metadata, 'odata.type', type, 'Name', name,);
But I get an error
Error using struct
Invalid field name "odata.metadata"
Here's the json object I'm trying to use in Matlab. All strings for simplicity:
{
"odata.metadata": "https://website.com#Element",
"odata.type": "Blah.Blah.This.That",
"Name": "My Object"
}
Is there a way to submit this json object or is it a lost cause?
Field names are not allowed to have dots in them. The reason why is because this will be confused with accessing another nested structure within the structure itself.
For example, doing json.odata.metadata would be interpreted as json being a struct with a member whose field name is odata where odata has another member whose field name is metadata. This would not be interpreted as a member with the combined field name as odata.metadata. You're going to have to rename the field to something else or change the convention of your field name slightly.
Usually, the convention is to replace dots with underscores. An automated way to take care of this if you're not willing to manually rename the field names yourself is to use a function called matlab.lang.makeValidName that takes in a string and converts it into a valid field name. This function was introduced in R2014a. For older versions, it's called genvarname.
For example:
>> matlab.lang.makeValidName('odata.metadata')
ans =
odata_metadata
As such, either replace all dots with _ to ensure no ambiguities or use matlab.lang.makeValidName or genvarname to take care of this for you.
I would suggest using a a containers.Map instead of a struct to store your data, and then creating your JSON string by iterating over the Map filednames and appending them along with the data to your JSON.
Here's a quick demonstration of what I mean:
%// Prepare the Map and the Data:
metadata = 'https://website.com#Element';
type = 'Blah.Blah.This.That';
name = 'My Object';
example_map = containers.Map({'odata.metadata','odata.type','Name'},...
{metadata,type,name});
%// Convert to JSON:
JSONstr = '{'; %// Initialization
map_keys = keys(example_map);
map_vals = values(example_map);
for ind1 = 1:example_map.Count
JSONstr = [JSONstr '"' map_keys{ind1} '":"' map_vals{ind1} '",'];
end
JSONstr =[JSONstr(1:end-1) '}']; %// Finalization (get rid of the last ',' and close)
Which results in a valid JSON string.
Obviously if your values aren't strings you'll need to convert them using num2str etc.
Another alternative you might want to consider is the JSONlab FEX submission. I saw that its savejson.m is able to accept cell arrays - which can hold any string you like.
Other alternatives may include any of the numerous Java or python JSON libraries which you can call from MATLAB.
I probably shouldn't add this as an answer - but you can have '.' in a struct fieldname...
Before I go further - I do not advocate this and it will almost certainly cause bugs and a lot of trouble down the road... #rayryeng method is a better approach
If your struct is created by a mex function which creates a field that contains a "." -> then you will get what your after.
To create your own test see the Mathworks example and modify accordingly.
(I wont put the full code here to discourage the practice).
If you update the char example and compile to test_mex you get:
>> obj = test_mex
obj =
Doublestuff: [1x100 double]
odata.metadata: 'This is my char'
Note: You can only access your custom field in Matlab using dynamic fieldnames:
obj.('odata.metadata')
You need to use a mex capability to update it...