Can JSON String format be converted to Actual format using groovy? - json

I have the following JSON String format getting from external source:-
What kind of format is this actually?
{
id=102,
brand=Disha,
book=[{
slr=EFTR,
description=Grammer,
data=TYR,
rate=true,
numberOfPages=345,
maxAllowed=12,
currentPage=345
},
{
slr=EFRE,
description=English,
data=TYR,
rate=true,
numberOfPages=345,
maxAllowed=12,
currentPage=345
}]
}
I want to convert this into actual JSON format like this: -
{
"id": "102",
"brand": "Disha",
"book": [{
"slr": "EFTR",
"description": "Grammer",
"data": "TYR",
"rate": true,
"numberOfPages": 345,
"maxAllowed": "12",
"currentPage": 345
},
{
"slr": "EFRE",
"description": "English",
"data": "TYR",
"rate": true,
"numberOfPages": 345,
"maxAllowed": "12",
"currentPage": 345
}]
}
Is this achievable using groovy command or code?

Couple of things:
You do not need Groovy Script test step which is currently there as step3
For step2, Add a 'Script Assertion` with given below script
Provide step name for nextStepName in the script below for which you want to add the request.
//Provide the test step name where you want to add the request
def nextStepName = 'step4'
def setRequestToStep = { stepName, requestContent ->
context.testCase.testSteps[stepName]?.httpRequest.requestContent = requestContent
}
//Check the response
assert context.response, 'Response is empty or null'
setRequestToStep(nextStepName, context.response)
EDIT: Based on the discussion with OP on the chat, OP want to update existing request of step4 for a key and its value as step2's response.
Using samples to demonstrate the change input and desired outputs.
Let us say, step2's response is:
{
"world": "test1"
}
And step4's existing request is :
{
"key" : "value",
"key2" : "value2"
}
Now, OP wants to update value of key with first response in ste4's request, and desired is :
{
"key": {
"world": "test1"
},
"key2": "value2"
}
Here is the updated script, use it in Script Assertion for step 2:
//Change the key name if required; the step2 response is updated for this key of step4
def keyName = 'key'
//Change the name of test step to expected to be updated with new request
def nextStepName = 'step4'
//Check response
assert context.response, 'Response is null or empty'
def getJson = { str ->
new groovy.json.JsonSlurper().parseText(str)
}
def getStringRequest = { json ->
new groovy.json.JsonBuilder(json).toPrettyString()
}
def setRequestToStep = { stepName, requestContent, key ->
def currentRequest = context.testCase.testSteps[stepName]?.httpRequest.requestContent
log.info "Existing request of step ${stepName} is ${currentRequest}"
def currentReqJson = getJson(currentRequest)
currentReqJson."$key" = getJson(requestContent)
context.testCase.testSteps[stepName]?.httpRequest.requestContent = getStringRequest(currentReqJson)
log.info "Updated request of step ${stepName} is ${getStringRequest(currentReqJson)}"
}
setRequestToStep(nextStepName, context.request, keyName)

We can convert the invalid JSON format to valid JSON format using this line of code:-
def validJSONString = JsonOutput.toJson(invalidJSONString).toString()

Related

Pulling specific Parent/Child JSON data with Python

I'm having a difficult time figuring out how to pull specific information from a json file.
So far I have this:
# Import json library
import json
# Open json database file
with open('jsondatabase.json', 'r') as f:
data = json.load(f)
# assign variables from json data and convert to usable information
identifier = data['ID']
identifier = str(identifier)
name = data['name']
name = str(name)
# Collect data from user to compare with data in json file
print("Please enter your numerical identifier and name: ")
user_id = input("Numerical identifier: ")
user_name = input("Name: ")
if user_id == identifier and user_name == name:
print("Your inputs matched. Congrats.")
else:
print("Your inputs did not match our data. Please try again.")
And that works great for a simple JSON file like this:
{
"ID": "123",
"name": "Bobby"
}
But ideally I need to create a more complex JSON file and can't find deeper information on how to pull specific information from something like this:
{
"Parent": [
{
"Parent_1": [
{
"Name": "Bobby",
"ID": "123"
}
],
"Parent_2": [
{
"Name": "Linda",
"ID": "321"
}
]
}
]
}
Here is an example that you might be able to pick apart.
You could either:
Make a custom de-jsonify object_hook as shown below and do something with it. There is a good tutorial here.
Just gobble up the whole dictionary that you get without a custom de-jsonify and drill down into it and make a list or set of the results. (not shown)
Example:
import json
from collections import namedtuple
data = '''
{
"Parents":
[
{
"Name": "Bobby",
"ID": "123"
},
{
"Name": "Linda",
"ID": "321"
}
]
}
'''
Parent = namedtuple('Parent', ['name', 'id'])
def dejsonify(json_str: dict):
if json_str.get("Name"):
parent = Parent(json_str.get('Name'), int(json_str.get('ID')))
return parent
return json_str
res = json.loads(data, object_hook=dejsonify)
print(res)
# then we can do whatever... if you need lookups by name/id,
# we could put the result into a dictionary
all_parents = {(p.name, p.id) : p for p in res['Parents']}
lookup_from_input = ('Bobby', 123)
print(f'found match: {all_parents.get(lookup_from_input)}')
Result:
{'Parents': [Parent(name='Bobby', id=123), Parent(name='Linda', id=321)]}
found match: Parent(name='Bobby', id=123)

TTN V3 (MQTT JSON) -> Telegraf -> Grafana / Sensor data from Dragino LSE01 does not apear

I have a problem with Telegraf. I have a Dragino LSE01-8 sensor which is registered on TTN v3. I can check the decoded payload by subscribing to the topic "v3/lse01-8#ttn/devices/+/up".
But when I want to grab the data from Influx, I can not get "temp_SOIL" and "water_SOIL", although the data appears in JSON. "conduct_SOIL" is no problem. But I don't know why. Can somebody give me a hint?
Another sensor (Dragino LHT 65) works fine with all data I want to access.
It's possible to get this data from the Influx-Database:
uplink_message_decoded_payload_BatV
uplink_message_decoded_payload_Mod
uplink_message_decoded_payload_conduct_SOIL
uplink_message_decoded_payload_i_flag
uplink_message_decoded_payload_s_flag
uplink_message_f_cnt
uplink_message_f_port
uplink_message_locations_user_latitude
uplink_message_locations_user_longitude
uplink_message_rx_metadata_0_channel_index
uplink_message_rx_metadata_0_channel_rssi
uplink_message_rx_metadata_0_location_altitude
uplink_message_rx_metadata_0_location_latitude
uplink_message_rx_metadata_0_location_longitude
uplink_message_rx_metadata_0_rssi
uplink_message_rx_metadata_0_snr
uplink_message_rx_metadata_0_timestamp
uplink_message_settings_data_rate_lora_bandwidth
uplink_message_settings_data_rate_lora_spreading_factor
uplink_message_settings_timestamp
## Feuchtigkeitssensor Dragino LSE01-8
[[inputs.mqtt_consumer]]
name_override = "TTN-LSE01"
servers = ["tcp://eu1.cloud.thethings.network:1883"]
qos = 0
connection_timeout = "30s"
topics = [ "v3/lse01-8#ttn/devices/+/up" ]
client_id = "telegraf"
username = "lse01-8#ttn"
password = "NNSXS.LLSNSE67AP..................P67Q.Q...........HPG............KJA..........." //
data_format = "json"
This is the JSON data I can get (I changed some data in order not to send any passwords or tokens).
{
"end_device_ids":{
"device_id":"eui-a8.40.141.bbe4",
"application_ids":{
"application_id":"lse01-8"
},
"dev_eui":"A8...40.BE...4",
"join_eui":"A8.40.010.1",
"dev_addr":"2.9F.....8"
},
"correlation_ids":[
"as:up:01G4WDNS..P3C3R...RK56VQ...KT7N076",
"gs:conn:01G4H2F.ETRG.V2QER...RQ.0K1MGZ44",
"gs:up:host:01G4H2F.ETWRZX.4PFN.A2M.6RDKD4",
"gs:uplink:01G4WDN.N7B6P.J8E.JS.503F1",
"ns:uplink:01G4WDNSFM.MCYYEZZ1.KY.4M78",
"rpc:/ttn.lorawan.v3.GsNs/HandleUplink:01G4W.NSFM29Z3.PABYW...43",
"rpc:/ttn.lorawan.v3.NsAs/HandleUplink:01G4W....VTQ4DMKBF"
],
"received_at":"2022-06-06T11:51:18.979353604Z",
"uplink_message":{
"session_key_id":"AYE...j+DM....A==",
"f_port":2,
"f_cnt":292,
"frm_payload":"DSQAAAcVB4AADBA=",
"decoded_payload":{
"BatV":3.364,
"Mod":0,
"conduct_SOIL":12,
"i_flag":0,
"s_flag":1,
"temp_DS18B20":"0.00",
"temp_SOIL":"19.20",
"water_SOIL":"18.13"
},
"rx_metadata":[
{
"gateway_ids":{
"gateway_id":"lr8",
"eui":"3.6201F0.058.....00"
},
"time":"2022-06-06T11:51:00.289713Z",
"timestamp":4283143007,
"rssi":-47,
"channel_rssi":-47,
"snr":7,
"location":{
"latitude":51.______________,
"longitude":6.__________________,
"altitude":25,
"source":"SOURCE_REGISTRY"
},
"uplink_token":"ChsKG________________________________",
"channel_index":2
}
],
"settings":{
"data_rate":{
"lora":{
"bandwidth":125000,
"spreading_factor":7
}
},
"coding_rate":"4/5",
"frequency":"868500000",
"timestamp":4283143007,
"time":"2022-06-06T11:51:00.289713Z"
},
"received_at":"2022-06-06T11:51:18.772518399Z",
"consumed_airtime":"0.061696s",
"locations":{
"user":{
"latitude":51._________________,
"longitude":6.__________________4,
"source":"SOURCE_REGISTRY"
}
},
"version_ids":{
"brand_id":"dragino",
"model_id":"lse01",
"hardware_version":"_unknown_hw_version_",
"firmware_version":"1.1.4",
"band_id":"EU_863_870"
},
"network_ids":{
"net_id":"000013",
"tenant_id":"ttn",
"cluster_id":"eu1",
"cluster_address":"eu1.cloud.thethings.network"
}
}
}

Incorrect properties displayed from Json

I am having an issue when trying to collect data from a json using SOAP UI and groovy scripting. Below is an example json:
{
"regions": [{
"hotels": [{
"roomInformation": [{
"hotelRoomId": xxx,
}],
"regionId": x,
"hotelId": xxx,
"providerInformation": {
"ratePlanCode": "xxx",
},
"providerHotelId": 0000001
},
{
"roomInformation": [{
"hotelRoomId": xx,
}],
"regionId": x,
"hotelId": xxx,
"providerInformation": {
"ratePlanCode": "ggg",
},
"providerHotelId": 0000002
}
],
"errors": null
}],
"errors": null
}
What I want to do is select the first instance of providerHotelId and ratePlanCode. To do this I have the groovy script below to tackle this:
def alert = com.eviware.soapui.support.UISupport
import groovy.json.JsonSlurper
def response = testRunner.testCase.getTestStepByName("Search Test").getProperty("Response").getValue();
def jsonRes = new JsonSlurper().parseText(response);
def providerhotelid = jsonRes.regions.hotels.providerHotelId[0].toString()
def rateplancode = jsonRes.regions.hotels.providerInformation[0].ratePlanCode.toString()
log.info providerhotelid
testRunner.testCase.setPropertyValue('providerhotelid', providerhotelid)
testRunner.testCase.setPropertyValue('rateplancode', rateplancode)
This outputs below in my custom properties:
providerhotelid - [0000001,0000002]
rateplancode - [xxx]
The above is incorrect because:
providerhotelid - it displays all provider hotel ids when I only want the first one which should be 0000001.
rateplancode - is correct but it displays a [] around it and I want this removed. Same goes for providerhotelid.
So for this example my custom properties should display:
providerhotelid - 0000001
rateplancode - xxx
How can this be achieved within my groovy script?
Here is what you need:
//Get all the values, falatten them and get the first one
def providerhotelid = jsonRes.regions.hotels.providerHotelId.flatten()[0]
def rateplancode = jsonRes.regions.hotels.providerInformation.ratePlanCode.flatten()[0]
log.info providerhotelid
log.info rateplancode
You can quickly try it online Demo

How to use MapReduce to parse a Json file?

Im new to json format. And Im trying to learn how to parse a Json file and pick up the data in it using a MapReduce programming model. Is there any Json Parser that can read multiple lines in records.
Here is my possible & maximum number of elements that be present in my Json file:
{
"type": "",
"format": "",
"version": "",
"id": "",
"start": "",
"cp": "",
message:{ "proto": "","protoVer": "","cliIP": "","reqPort": "","reqHost": "","reqMethod": "","reqPath": "","reqQuery": "","reqCT": "","reqLen": "","sslVer": "","status": "","redirURL": "","respCT": "","respLen": "","bytes": "","UA": "","fwdHost":},
reqHdr:{"accEnc": "","accLang": "","auth": "","cacheCtl": "","conn": "","contMD5": "","cookie": "","DNT": "","expect": "","ifMatch": "","ifMod": "","ifNone": "","ifRange": "","ifUnmod": "","range": "","referer": "","te": "","upgrade": "","via": "","xFrwdFor": "","xReqWith": ""},
"respHdr": {"accRange": "","allowOrigin": "","age": "","allow": "","cacheCtl": "","conn": "","contEnc": "","contLang": "","contMD5": "","contDisp": "","contRange": "","date": "","eTag": "","expires": "","lastMod": "","link": "","p3p": "","retry": "","server": "","trailer": "","transEnc": "","vary": "","via": "","warning": "","wwwAuth": "","xPwrdBy": "","setCookie": ""},
"netPerf": {"downloadTime": "","originName": "","originIP": "","originInitIP": "","originRetry": "","lastMileRTT": "","midMileLatency": "","netOriginLatency": "","lastMileBW": "","cacheStatus": "","firstByte": "","lastByte": "","asnum": "","network": "","netType": "","edgeIP": ""},
"geo": {"country": "","region": "","city": ""},
"waf" : {"logVer" : "1.0","ipRules" : "","appRules" : "","warn" : "","deny" : ""},
"content": {"custom_name": "custom_value"},
}
These are my sample values in the json file.
`{"type":"cloud_monitor","format":"default","version":"1.0","id":"71101cb85441995d11a43bb","start":"1413585245.921","cp":"254623","message":{"proto":"http","protoVer":"1.1","status":"403","cliIP":"23.79.231.14","reqPort":"80","reqHost":"ksd.metareactor.com","reqMethod":"GET","reqPath":"%2findex.php","reqQuery":"path%3d57%26product_id%3d49%26route%3d%255Cwinnt%255Cwin.ini%2500.","respCT":"text/html","respLen":"286","bytes":"286","UA":"mozilla-saturn","fwdHost":"origin-demo2-akamaized.scoe-sil.net"}`,
"reqHdr":{"accEnc":"gzip,%20deflate","cookie":"PHPSESSID%3dkkqoodvfe0rt9l7lbvqghk6e15%3bcurrency%3dUSD%3blanguage%3den"},"netPerf":{"downloadTime":"8","lastMileRTT":"20","cacheStatus":"0","firstByte":"1","lastByte":"1","asnum":"12222","edgeIP":"184.28.16.109"},"geo":{"country":"US","region":"CA","city":"SANFRANCISCO","lat":"37.7795","long":"-122.4195"},"network":{"edgeIP":"184.28.16.109","asnum":"12222","network":"","networkType":""},"waf":{"ver":"2.0","policy":"qik1_12418","ruleSet":"KRS%201.0","mode":"scr","rsr":"1","dor":"0","oft":"0","riskGroups":":INBOUND-ANOMALY","riskTuples":":-3000002","riskScores":":-1000","pAction":"","pRate":"","warnRules":"3000002","warnSlrs":"ARGS%3aroute","warnData":"d2lubnQvd2luLmluaQ%3d%3d","warnTags":"AKAMAI%2fWEB_ATTACK%2fFILE_INJECTION","denyRules":"INBOUND-ANOMALY","denyData":"U2NvcmU6IDEwMDAsIERFTlkgdGhyZXNob2xkOiAyNSwgQWxlcnQgUnVsZXM6IDMwMDAwMDIsIERlbnkgUnVsZTogLCBMYXN0IE1hdGNoZWQgTWVzc2FnZTogTG9jYWwgU3lzdGVtIEZpbGUgQWNjZXNzIEF0dGVtcHQ%3d"}}
I have a Java Json Parser but I can use it to read a single line. How can I identify a multiple line record in a Json file and use it in the MapReduce code to extract the data.
My Json parser class:
String[] tuple = value.toString().split("\n");
try {
for(int i=0; i<tuple.length; i++) {
JSONObject jsonobj = new JSONObject(tuple[i]);
type = (String) jsonobj.get("type");
format = (String) jsonobj.get("format");
version = (String) jsonobj.get("version");
id = (String) jsonobj.get("id");
start = (String) jsonobj.get("start");
cp = (String) jsonobj.get("cp");
message = (String) jsonobj.get("message");
}
} catch (JSONException e) {
e.printStackTrace();
}
Can anyone help me on writing code to read Json records which are in multiple lines in MapReduce ?

i got empty value when print json key in lua code

this is the json response plus "\x00" in the end from server :
{
"STATUS": [{
"STATUS":"S",
"When":1470180059,
"Code":11,
"Msg":"Summary",
"Description":"nsgminer 0.9.2"
}],"SUMMARY": [{
"Elapsed":2061,
"MHS av":0.00,
"Found Blocks":0,
"Getworks":76,
"Accepted":0,
"Rejected":0,
"Hardware Errors":0,
"Utility":0.00,
"Discarded":209,
"Stale":0,
"Get Failures":3,
"Local Work":293,
"Remote Failures":0,
"Network Blocks":14,
"Total MH":0.0000,
"Work Utility":0.00,
"Difficulty Accepted":0.00000000,
"Difficulty Rejected":0.00000000,
"Difficulty Stale":0.00000000,
"Best Share":0
}],
"id":1
}\x00
i want to use the json in lua code :
local output = stdnse.output_table()
local json_string = tostring(result:sub(1, -2))
local pos, value = json.parse(json_string)
output["Description"] = value["STATUS"][0]["Description"]
return output
when i print it out, i got null value
i solve that with covert json to string and convert string into json table
local pos, value = json.parse(tostring(json_string))
output["Description"] = value["STATUS"][1]["Description"]