Parsing html data in params for xml response using CDATA - html

In the api, i'm generating a Xml response by hitting the url with request data in params. It contains some field which have html content and tags.The content is getting saved correctly in DB but when the response is getting generated the tags are being encoded which will happen as we need to skip those fields while parsing. I would like to know that how can i implement CDATA in order to skip the specific fields while parsing.
def generate_mobile_api_success_response(status_code, format, request_id, content = nil)
format_type_method, options_hash, content_type = get_format_method(format)
data = { "request_id" => request_id, "status" => status_code, "message" => status_message(status_code)}
data["data"] = content unless content.blank?
data = generate_data_format(format, data)
resp = [status_code, { "Content-Type" => content_type , "request_id" => request_id}, data.send(format_type_method, options_hash)]
generate_active_controller_response_format(resp)
resp
end
Content passed is a params hash, and format is xml. resp contains the following data when i tried to print it.Detailed description tag contains the encoded data
[201, {"request_id"=>"b425bce0-307d-012f-3e68-042b2b8686e6", "Content-Type"=>"application/xml"}, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<response>\n <data>\n <fine_print_line_3>line 3</fine_print_line_3>\n <available_start_date>2012-02-02T06:00:00+05:30</available_start_date>\n <status>inactive</status>\n <highlight_line_2>gfgf</highlight_line_2>\n <original_price>50.00</original_price>\n <category_id>bc210bb0-52b7-012e-8896-12313b077c61</category_id>\n <available_end_date>2012-03-25T00:00:00+05:30</available_end_date>\n <expiry_date>2012-08-25T00:00:00+05:30</expiry_date>\n <highlight_line_3></highlight_line_3>\n <product_service>food</product_service>\n <created_at>2012-02-03T15:43:56+05:30</created_at>\n <detailed_description><b>this is the testing detailed</b> </detailed_description>...
I would surely like to post some extra code if required.

So your data is in XML and you would like to know the content of these fields?
Use Nokogiri:
xml = data.send(format_type_method, options_hash)
doc = Nokogiri::XML(xml)
start_date = doc.xpath("//available_start_date").content
p start_date #=> "2012-02-02T06:00:00+05:30"
Once you have your fields in variables you can whatever you want.

Related

Groovy - parse/ convert x-www-form-urlencoded to something like JSON

this is my second try to explain a bit more precisely what I'm looking for ;-)
I set a webhook in Mailchimp that fires every time a new subscriber of an audience appears. Mailchimp sends a HTTP POST request to a Jira Sriptrunner REST endpoint.
The content type of this request is application/x-www-form-urlencoded.
Within the Jira endpoint I would like to read the request data. How can I do that?
The payload (raw body) I receive looks like this:
type=unsubscribe&fired_at=2020-05-26+07%3A04%3A42&data%5Baction%5D=unsub&data%5Breason%5D=manual&data%5Bid%5D=34f28a4516&data%5Bemail%5D=examlple%40bla.com&data%5Bemail_type%5D=html&data%5Bip_opt%5D=xx.xxx.xxx.198&data%5Bweb_id%5D=118321378&data%5Bmerges%5D%5BEMAIL%5D=example%40bla.com&data%5Bmerges%5D%5BFNAME%5D=Horst&data%5Bmerges%5D%5BLNAME%5D=Schlemmer&data%5Bmerges%5D%5BCOMPANY%5D=First&data%5Bmerges%5D%5BADDRESS%5D%5Baddr1%5D=XXX
Now I would like to parse the data of the raw body into a JSON or something similiar.
The result might look like this:
{
"web_id": 123,
"email": "example#bla.com",
"company": "First",
...
}
Meanwhile I searched around a little and found something like the node.js "querystring" module. It would be great if there is something similiar within Groovy or any other way to parse the data of application/x-www-form-urlencoded to json format.
Best regards and thanks in advance
Bernhard
def body = "type=unsubscribe&fired_at=2020-05-26+07%3A04%3A42&data%5Baction%5D=unsub&data%5Breason%5D=manual&data%5Bid%5D=34f28a4516&data%5Bemail%5D=examlple%40bla.com&data%5Bemail_type%5D=html&data%5Bip_opt%5D=xx.xxx.xxx.198&data%5Bweb_id%5D=118321378&data%5Bmerges%5D%5BEMAIL%5D=example%40bla.com&data%5Bmerges%5D%5BFNAME%5D=Horst&data%5Bmerges%5D%5BLNAME%5D=Schlemmer&data%5Bmerges%5D%5BCOMPANY%5D=First&data%5Bmerges%5D%5BADDRESS%5D%5Baddr1%5D=XXX"
def map = body.split('&').collectEntries{e->
e.split('=').collect{ URLDecoder.decode(it, "UTF-8") }
}
assert map.'data[merges][EMAIL]'=='example#bla.com'
map.each{println it}
prints:
type=unsubscribe
fired_at=2020-05-26 07:04:42
data[action]=unsub
data[reason]=manual
data[id]=34f28a4516
data[email]=examlple#bla.com
data[email_type]=html
data[ip_opt]=xx.xxx.xxx.198
data[web_id]=118321378
data[merges][EMAIL]=example#bla.com
data[merges][FNAME]=Horst
data[merges][LNAME]=Schlemmer
data[merges][COMPANY]=First
data[merges][ADDRESS][addr1]=XXX
A imple no-brainer groovy:
def a = '''
data[email_type]: html
data[web_id]: 123
fired_at: 2020-05-26 07:28:25
data[email]: example#bla.com
data[merges][COMPANY]: First
data[merges][FNAME]: Horst
data[ip_opt]: xx.xxx.xxx.xxx
data[merges][PHONE]: xxxxx
data[merges][ADDRESS][zip]: 33615
type: subscribe
data[list_id]: xxXXyyXX
data[merges][ADDRESS][addr1]: xxx.xxx'''
def res = [:]
a.eachLine{
def parts = it.split( /\s*:\s*/, 2 )
if( 2 != parts.size() ) return
def ( k, v ) = parts
def complexKey = ( k =~ /\[(\w+)\]/ ).findAll()
if( complexKey ) complexKey = complexKey.last().last()
res[ ( complexKey ?: k ).toLowerCase() ] = v
}
res
gives:
[email_type:html, web_id:123, fired_at:2020-05-26 07:28:25,
email:example#bla.com, company:First, fname:Horst, ip_opt:xx.xxx.xxx.xxx,
phone:xxxxx, zip:33615, type:subscribe, list_id:xxXXyyXX, addr1:xxx.xxx]
I found a solution finally. I hope you understand and maybe it helps others too ;-)
Starting from daggett's answer I did the following:
// Split body and remove unnecessary characters
def map = body.split('&').collectEntries{e->
e.split('=').collect{ URLDecoder.decode(it, "UTF-8") }
}
// Processing the map to readable stuff
def prettyMap = new JsonBuilder(map).toPrettyString()
// Convert the pretty map into a json object
def slurper = new JsonSlurper()
def jsonObject = slurper.parseText(prettyMap)
(The map looks pretty much like in daggett's answer.
prettyMap)
Then I extract the keys:
// Finally extracting customer data
def type = jsonObject['type']
And I get the data I need. For example
Type : subscribe
...
First Name : Heinz
...
Thanks to daggett!

Display JSON Array Result in Spotfire DataTable using ironPython

I have a JSON String array (Result from a REST API call) as shown below. That string array should be organized as Column Names and Respective Records in a Data Table in Spotfire Using IronPython Script.
All the attributes should be displayed as my column names (only for once) and values should be displayed as records (iterate).
Any Help is much Appreciated.
Please let me know if you need more Clarity.
JSON Data:
[{"actionType":"Adjust Tubing or Casing PSI","assignedTeam":"FS","exceptionStatus":"","eventEndDate":"1970-01-01T00:00:00.000Z","description":"Motor Temperature","lastEditedDate":"2019-07-19T02:49:00.945Z","well":{"apiNumber12":"423294206100","area":"MIDLAND BASIN","wellName":"C SCHARB 31WB","asset":"ODESSA EAST FMT","route":"4000003 - NOD","apiNumber14":"42329420610001"},"actionItemPastDueDate":"2019-07-27T02:48:59.000Z","lastEditedBy":"Agent(System-Queue-ServiceLevel.ProcessEvent)","priorityRangeMax":500.000000000,"currentStage":"ALT5","id":"E-4056","workOrder":"","assignedDate":"2019-07-15T05:00:00.000Z","impactedLpo":true,"alcrApprovalComments":"","createdDate":"2019-07-16T02:48:34.786Z","alcrApprovalDeadline":"2019-07-19T02:48:59.000Z","dueDate":"2019-07-22T02:48:59.000Z","chemicalAdvisorComments":"","recommendationsStripped":"Training Test","workStatus":"Pending-ALCRApproval","actionTakenStripped":"","recommendations":"Training Test","tier":"6","cancelationComments":"","priority":"5","woSubmitted":false,"daysTillActionItemDue":"","eventStartDate":"2019-07-16T02:48:37.276Z","assignedTo":"","currentCommentsStripped":"","fsActionItemNearDueDate":"1970-01-01T00:00:00.000Z","cancelDate":"1970-01-01T00:00:00.000Z","averageBoed":374,"fsActionItemPastDueDate":"1970-01-01T00:00:00.000Z","lpoUom":"barrels per day","lpoRate":374,"surfaceWorkRequired":false,"liftMethod":"ESP","originatorCai":"alwo.coeesp","actionItemNearDueDate":"2019-07-21T02:48:59.000Z","createdBy":"Centaur COE ESP","priorityRangeMin":250.000000000,"functionalGroup":"ESP COE"}]
(Above is the Edited JSON of one single record)
uri = "https://quantprice.herokuapp.com/api/v1.1/scoop/period?tickers=MSFT&begin=2012-02-19"
webRequest = HttpWebRequest.Create(uri)
response = webRequest.GetResponse()
from System.IO import StreamReader
from System.Web.Script.Serialization import JavaScriptSerializer
streamReader = StreamReader(response.GetResponseStream())
jsonData = streamReader.ReadToEnd()
js = JavaScriptSerializer()
dataDict = js.Deserialize(jsonData,object)
# build a string representing the data in tab-delimited text format
myColName = []
for val in dataDict["stations"]:
myColName.append(val["id"])
textData = "\t".join(myColName) + "\r\n"
print textData
for quote in dataDict["stations"]:
print "\t".join(str(val) for val in quote)
textData += "\t".join(str(val) for val in quote) + "\r\n"
print textData
If i do this, my Result set is showing [column name, column value] as rows.
But, i want "Column name" as my header list and "Column Values" as my Rows.

Using Ruby, grab value from HTML string inside a JSON response

I have searched what I can and I don't seem to be finding the answer I need. Granted I may not be wording it properly. I have tried using .find or even .rindex to count backwards, but no such luck. The value I receive from the JSON looks something like this:
"AdditionalData":"<Data><Entry Key=\"utm_campaign\" Value=\"j2c\" />
<Entry Key=\"utm_medium\" Value=\"cpc\" /><Entry Key=\"utm_source\"
Value=\"j2c\" /><Entry Key=\"job_id\" Value=\"300_xxxx_10703\" /></Data>"
I need to be able to grab the value for the key "job_id", so the "300_xxxx_11233". This value will change per object returned by the JSON response. Any help would be appreciated, and please let me know if this is already out there and I just missed it.
If the response format remains the same with every request, you could use a plain regexp expression to fetch your data, even without parsing JSON. Example:
response = "<Data><Entry Key=\"utm_campaign\" Value=\"j2c\" /><Entry Key=\"utm_medium\" Value=\"cpc\" /><Entry Key=\"utm_source\" Value=\"j2c\" /><Entry Key=\"job_id\" Value=\"300_xxxx_10703\" /></Data>"
match = response.match(%r{job_id\\?"\s+Value=\\?"(.+)\\?"}i)
match[1] if match # => "300_xxxx_10703"
If the response format can change (for example, if the order of the attributes of Entry element can change), then you need to parse JSON and use some HTML parser, such as Nokigiri, to fetch required attrbute. Code example:
parsed_response = JSON.parse(response)
doc = Nokogiri::HTML(parsed_response['AdditionalData'])
job_id = nil
doc.css('Entry').each do |el|
if el['Key'] == 'job_id'
job_id = el['Value']
break
end
end

Telegram bot in python

Currently, I'm doing a currency converter bot on telegram using python 3.
def on_callback_query(msg):
query_id, from_id, query_data = telepot.glance(msg, flavor='callback_query')
print('Callback Query:', query_id, from_id, query_data)
url = "http://api.fixer.io/latest?base=SGD&symbols="
symbols = query_data
response = requests.get(url + symbols)
data = response.json()
print(data)
if(query_data == 'SGD'):
bot.sendMessage(from_id, data)
elif (query_data == 'EUR'):
bot.sendMessage(from_id, data)
The data here will out {'rates': {'EUR': 0.62177}, 'base': 'SGD', 'date': '2017-09-18'} in my telegram chat. Is there anyway to beautify the output, i.e. take out the curly brackets and quotes?
This format named JSON and you can parse it as an object and simply print it with a custom format.
See this.
The result is in JSON format which is then converted to a dictionary, you can extract each element from the dictionary and use it to create a better format.
You can write in any format you like, here's an example
if(query_data == 'SGD'):
result = "Rates in EUR: {}\nBase: {}\nDate: {}".format(data['rates']['EUR'], data['base'], data['date'])
bot.sendMessage(from_id, result)
if(query_data == 'EUR'):
result = "Rates in SGD: {}\nBase: {}\nDate: {}".format(data['rates']['SGD'], data['base'], data['date'])
bot.sendMessage(from_id, result)
There's almost definitely a more efficient way of doing this than the way I did it, but it works.
str(output).replace("{","").replace("}","").replace("'","")

Keep Special Characters when parsing JSON response

I have retrieve this key/value from a hash using the facebook api
"message":"Next Practice:\n\nDate: 04.05.2014\nTime: 10:00-12:00\nVenue: Llandaff Fields\n\nAll welcome
but when i save it to my model i seem to lose all the special characters, i.e \n. Is there a way to save the value as it is returned so that i can use the \n when outputting to my view using .html_safe
This is how i am retrieving the data
def get_feed
fb_access_token = access_token
uri = URI(URI.escape "https://graph.facebook.com/#{VANDALS_ID}/posts/?#{fb_access_token}")
response = HTTParty.get(uri)
results = JSON.parse(response.body)
formatted_data(results)
end
anything i need to be doing to keep that string with \n left in it
Thanks
When I run the following code:
raw_json = '{"message":"Next Practice:\n\nDate: 04.05.2014\nTime: 10:00-12:00\nVenue: Llandaff Fields\n\nAll welcome"}'
parsed_json = JSON.parse(raw_json)
puts parsed_json['message']
# => Next Practice:
# => Date: 04.05.2014
# => Time: 10:00-12:00
# => Venue: Llandaff Fields
# => All welcome
So the \n is kept (it is parsed, and shown as real new-line). I also don't believe that saving this to your model erased the new lines.
Where I think your real problem lies is that in HTML new lines (\n) are not rendered as new lines at all, but as spaces. To render them as new lines, you need to replace them with breaks (<br>).
So you can try using the following on your ERB:
<div class=message><%= feed.message.gsub("\n", "<br>").html_safe %></div>
Your new-lines will now be rendered on the page.