I have following code to create a JSON for making a call to Adobe Analytics API (method segment.save)
item <-
list(definition = list(
container = list (
type = "hits",
operator = "or",
rules=I(list(
list(value= "test1 test2",
operator = "contains_any",
element = "page")))
)
),
owner="test",
reportSuiteID="test",
description="API Generated Segment",
name="test segment"
)
Once prettyfied and auto-unboxed, the result is:
> jsonlite::toJSON(item, pretty = T, auto_unbox= T)
{
"definition": {
"container": {
"type": "hits",
"operator": "or",
"rules": [
{
"value": "test1 test2",
"operator": "contains_any",
"element": "page"
}
]
}
},
"owner": "test",
"reportSuiteID": "test",
"description": "API Generated Segment",
"name": "test segment"
}
Good for creating new segments, but not so good for editing them
The JSON structure is valid, as I am able to create the new segment. However, I would like to check if the segment already exists (using f.i. the GetSegments() function from randyzwitch RSiteCatalyst package and check if name coincides already with a created segment). If the segment already exists, I want to pass the id to the API call, which is the method used for editing already existing segments. It should then look like:
> jsonlite::toJSON(item, pretty = T, auto_unbox= T)
{
"definition": {
...
},
"owner": "test",
"reportSuiteID": "test",
"description": "API Generated Segment",
"name": "test segment",
"id": "s1982XXXXXXXXX_XXXXX_XXXXX",
}
It is possible to make an if alike statement within the list() definition provided in the first piece of code? I would like to reach a solution that does not need an if statement that checks if segmentID exists and, depending on it, generates a call with id or a call without id.
Once a "JSON alike structure" is created using list function:
item <-
list(definition = list(
container = list (
type = "hits",
operator = "or",
rules=I(list(
list(value= "test1 test2",
operator = "contains_any",
element = "page")))
)
),
owner="test",
reportSuiteID="test",
description="API Generated Segment",
name="test segment"
)
We can push new elements to this list using the needed conditions. For example, if we have our segment IDs in a dataframe with name segments, we can push this ID to item this way:
if (!is.na(segments$segmentID[i])) {
item <- c(item, id=segments$segmentID[i])
}
Related
PostgreSQL 13
The goal is to rename all src keys in the photos array to image.
I have a table plans which has a column json with a simplified structure similar to the below sample.
{
"id": "some id",
"name": "some name",
"tags": [
{
"keyId": 123,
"valueId": 123
},
{
"keyId": 123,
"valueId": 123
}
],
"score": 123,
"photos": [
{
"src": "someString"
},
{
"src": "someString"
}
],
"payment": true
}
The number of objects in the photos array varies, but in general, it is less than 10, so a non-iterating method would be fine, too.
I tried something like this, but it is only good for modifying the value of a key, not the name of the key itself.
UPDATE
plans
SET
json = jsonb_set(json::jsonb, '{photos, 0, src}', '"image"')
;
With the following attempt, I was actually able to rename the key but it overwrites everything else, so only an object with {"image": "someUrl"} is left:
UPDATE
plans
SET
json = (json -> 'photos' ->> 0)::jsonb - 'src' || jsonb_build_object ('image',
json::jsonb -> 'photos' -> 0 -> 'src')
WHERE json::jsonb ? 'photos' = true;
Is there a way to rename keys as expected?
So in the end I used a variation of my initial jsonb_set method. The solution isn't elegant or efficient, but since it is a one-time operation, it was only important to work:
UPDATE
plans
SET
json = jsonb_set(json::jsonb, '{photos, 0, imageUrl}', (json->'photos'->0->'src')::jsonb)
WHERE
json->'photos'->0->'src' IS NOT NULL
;
This query would add the imageUrl key with the existing value of the src key for the first object (position 0) in the photos array. So it left me with src and imageUrl key.
To remove the src key, I ran the following query
UPDATE
plans
SET
json = json::jsonb #- '{ photos, 0, src}'
;
Repeating this as many times as the maximum number of elements in a photos array eventually solved the issue for me.
I am creating an API for a game where users get points for labelling images. If a user has labeled an image with the same label as another, the score gets updated by 5 points (and 25 if that player is their opponent). I have declared the field score inside my serializer as a SerializerMethodField.
In the get_score method, I am trying to access the tag__name field inside the list of tags_to_compare and simply extract the strings and store them in a list.
I have tried the following inside my serializer:
Update
I have updated my get_score method as follows:
serializers.py
score = serializers.SerializerMethodField('get_score')
...
def get_score(self, tagging):
score = 0
resource_id = tagging.resource.id
# A previously played gameround for this resource is coordinated for Tag verification
coordinated_gameround = Gameround.objects.all().filter(taggings__resource_id=resource_id).order_by("?").first()
# list of tag_name from coordinated gameround
coordinated_gameround_tags = coordinated_gameround.taggings.all().values_list("tag__name", flat=True)
if Tag.objects.all().filter(name=tagging) in coordinated_gameround_tags:
score += 25
return score
I tested the following line and it retrieves a list of taggings.
coordinated_gameround_tags = coordinated_gameround.taggings.all().values_list("tag__name", flat=True)
This is the JSON Object where the list of tags is:
{
"gameround": {
"id": 2014305513,
"user": {
"id": 1,
"username": "carina",
"is_superuser": true
},
"score": 50,
"tags_to_compare": [
{
"tag_id": 153,
"tag__name": "night",
"tag__language": "en",
"resource_id": 323570
},
{
"tag_id": 10437,
"tag__name": "dress",
"tag__language": "en",
"resource_id": 323570
},
{
"tag_id": 9598,
"tag__name": "sleep",
"tag__language": "en",
"resource_id": 323570
}
]
}
}
coordinated_gameround is a Gameround instance of your model. Not a Dict. So you cannot access it with coordinated_gameround[key].
If your Gameround has a field named "name" or "langage", you can get those values with
name = coordinated_gameround.name
langage = coordinated_gameround.langage
Also, there might be an error with this line
Tag.objects.all().filter(name=tagging) in coordinated_gameround_tags
Do you expect coordinated_gameround_tags to contain a list of queryset ?
I just discovered glom and the tutorial makes sense, but I can't figure out the right spec to use for chrome BrowserHistory.json entries to create a data structure grouped by client_id or if this is even the right use of glom. I think I can accomplish this using other methods by looping over the json, but was hoping to learn more about glom and its capabilities.
The json has Browser_History with a list for each history entry as follows:
{
"Browser_History": [
{
"favicon_url": "https://www.google.com/favicon.ico",
"page_transition": "LINK",
"title": "Google Takeout",
"url": "https://takeout.google.com",
"client_id": "abcd1234",
"time_usec": 1424794867875291
},
...
I'd like a data structure where everything is grouped by the client_id, like with the client_id as the key to a list of dicts, something like:
{ 'client_ids' : {
'abcd1234' : [ {
"title" : "Google Takeout",
"url" : "https://takeout.google.com",
...
},
...
],
'wxyz9876' : [ {
"title" : "Google",
"url" : "https://www.google.com",
...
},
...
}
}
Is this something glom is suited for? I've been playing around with it and reading, but I can't seem to get the spec correct to accomplish what I need. Best I've got without error is:
with open(history_json) as f:
history_list = json.load(f)['Browser_History']
spec = {
'client_ids' : ['client_id']
}
pprint(glom(data, spec))
which gets me a list of all the client_ids, but I can't figure out how to group them together as keys rather than have them as a big list. any help would be appreciated, thanks!
This should do the trick although I'm not sure if this is the most "glom"-ic way to achieve this.
import glom
grouping_key = "client_ids"
def group_combine (existing,incoming):
# existing is a dictionary used for accumulating the data
# incoming is each item in the list (your input)
if incoming[grouping_key] not in existing:
existing[incoming[grouping_key]] = []
if grouping_key in incoming:
existing[incoming[grouping_key]].append(incoming)
return existing
data ={ 'Browser_History': [{}] } # your data structure
fold_spec = glom.Fold(glom.T,init = dict, op = group_combine )
results = glom.glom(data["Browser_History"] ,{ grouping_key:fold_spec })
I have a sqlite database and in one of the fields I have stored complete json object . I have to make some json select requests . If you see my json
the ALL key has value which is an array . We need to extract some data like all comments where "pod" field is fb . How to extract properly when sqlite json has value as an array ?
select json_extract(data,'$."json"') from datatable ; gives me entire thing . Then I do
select json_extract(data,'$."json"[0]') but i dont want to do it manually . i want to iterate .
kindly suggest some source where i can study and work on it .
MY JSON
{
"ALL": [{
"comments": "your site is awesome",
"pod": "passcode",
"originalDirectory": "case1"
},
{
"comments": "your channel is good",
"data": ["youTube"],
"pod": "library"
},
{
"comments": "you like everything",
"data": ["facebook"],
"pod": "fb"
},
{
"data": ["twitter"],
"pod": "tw",
"ALL": [{
"data": [{
"codeLevel": "3"
}],
"pod": "mo",
"pod2": "p"
}]
}
]
}
create table datatable ( path string , data json1 );
insert into datatable values("1" , json('<abovejson in a single line>'));
Simple List
Where your JSON represents a "simple" list of comments, you want something like:
select key, value
from datatable, json_each( datatable.data, '$.ALL' )
where json_extract( value, '$.pod' ) = 'fb' ;
which, using your sample data, returns:
2|{"comments":"you like everything","data":["facebook"],"pod":"fb"}
The use of json_each() returns a row for every element of the input JSON (datatable.data), starting at the path $.ALL (where $ is the top-level, and ALL is the name of your array: the path can be omitted if the top-level of the JSON object is required). In your case, this returns one row for each comment entry.
The fields of this row are documented at 4.13. The json_each() and json_tree() table-valued functions in the SQLite documentation: the two we're interested in are key (very roughly, the "row number") and value (the JSON for the current element). The latter will contain elements called comment and pod, etc..
Because we are only interested in elements where pod is equal to fb, we add a where clause, using json_extract() to get at pod (where $.pod is relative to value returned by the json_each function).
Nested List
If your JSON contains nested elements (something I didn't notice at first), then you need to use the json_tree() function instead of json_each(). Whereas the latter will only iterate over the immediate children of the node specified, json_tree() will descend recursively through all children from the node specified.
To give us some data to work with, I have augmented your test data with an extra element:
create table datatable ( path string , data json1 );
insert into datatable values("1" , json('
{
"ALL": [{
"comments": "your site is awesome",
"pod": "passcode",
"originalDirectory": "case1"
},
{
"comments": "your channel is good",
"data": ["youTube"],
"pod": "library"
},
{
"comments": "you like everything",
"data": ["facebook"],
"pod": "fb"
},
{
"data": ["twitter"],
"pod": "tw",
"ALL": [{
"data": [{
"codeLevel": "3"
}],
"pod": "mo",
"pod2": "p"
},
{
"comments": "inserted by TripeHound",
"data": ["facebook"],
"pod": "fb"
}]
}
]
}
'));
If we were to simply switch to using json_each(), then we see that a simple query (with no where clause) will return all elements of the source JSON:
select key, value
from datatable, json_tree( datatable.data, '$.ALL' ) limit 10 ;
ALL|[{"comments":"your site is awesome","pod":"passcode","originalDirectory":"case1"},{"comments":"your channel is good","data":["youTube"],"pod":"library"},{"comments":"you like everything","data":["facebook"],"pod":"fb"},{"data":["twitter"],"pod":"tw","ALL":[{"data":[{"codeLevel":"3"}],"pod":"mo","pod2":"p"},{"comments":"inserted by TripeHound","data":["facebook"],"pod":"fb"}]}]
0|{"comments":"your site is awesome","pod":"passcode","originalDirectory":"case1"}
comments|your site is awesome
pod|passcode
originalDirectory|case1
1|{"comments":"your channel is good","data":["youTube"],"pod":"library"}
comments|your channel is good
data|["youTube"]
0|youTube
pod|library
Because JSON objects are mixed in with simple values, we can no longer simply add where json_extract( value, '$.pod' ) = 'fb' because this produces errors when value does not represent an object. The simplest way around this is to look at the type values returned by json_each()/json_tree(): these will be the string object if the row represents a JSON object (see above documentation for other values).
Adding this to the where clause (and relying on "short-circuit evaluation" to prevent json_extract() being called on non-object rows), we get:
select key, value
from datatable, json_tree( datatable.data, '$.ALL' )
where type = 'object'
and json_extract( value, '$.pod' ) = 'fb' ;
which returns:
2|{"comments":"you like everything","data":["facebook"],"pod":"fb"}
1|{"comments":"inserted by TripeHound","data":["facebook"],"pod":"fb"}
If desired, we could use json_extract() to break apart the returned objects:
.mode column
.headers on
.width 30 15 5
select json_extract( value, '$.comments' ) as Comments,
json_extract( value, '$.data' ) as Data,
json_extract( value, '$.pod' ) as POD
from datatable, json_tree( datatable.data, '$.ALL' )
where type = 'object'
and json_extract( value, '$.pod' ) = 'fb' ;
Comments Data POD
------------------------------ --------------- -----
you like everything ["facebook"] fb
inserted by TripeHound ["facebook"] fb
Note: If your structure contained other objects, of different formats, it may not be sufficient to simply select for type = 'object': you may have to devise a more subtle filtering process.
I have this input parameter (complex list of inputs) that needs to be sent as part of a HTTP POST request. I have been searching for an AutoIt equivalent to encapsulate the following data, but unsuccessful.
Below given working code is failing with 400 - Bad Request for obvious reason of incomplete input (unable to send ip_list and other array type parameters).
Questions
How to encapsulate the below given data for $oHttp.Send()?
How to encapsulate data types like:
null - aligned_device_template parameter
white-spaced values - description and
empty array - hostname or device_groups
Input parameter in JSON format to be converted & sent in AutoIt
{
"name": "SnmpSIM",
"description": "EM7 device created by BDD test case",
"credentials":"{{feature.credential.body.result_set[*].URI}}",
"organization": "/api/organization/0",
"aligned_device_template": null,
"aligned_collector": "{{feature.appliance_id[0].id[0]}}{{feature.appliance_id[0].id[1]}}",
"discover_non_snmp": "1",
"scan_ports": [
"21",
"22",
"23",
"25",
"80"
],
"ip_lists": [
{
"start_ip": "{{feature.json.ip}}",
"end_ip": "{{feature.json.ip}}"
}
],
"dhcp_enabled": "0",
"duplicate_protection": "1",
"model_device": "1",
"log_all": "1",
"scan_all_ips": null,
"port_scan_timeout": null,
"initial_scan_level": null,
"scan_throttle": null,
"interface_inventory_timeout": "600000",
"max_interface_inventory_count": "10000",
"bypass_interface_inventory": "0",
"hostnames": [],
"device_groups": []
}
Working code
; The data to be sent
$sPD = 'name=SnmpSIM&description=EM7 device created by BDD test case&credentials=37&organization=/api/organization/0&aligned_device_template=null&aligned_collector=1&discover_non_snmp=1&dhcp_enabled=0&duplicate_protection=1&model_device=1&log_all=1&scan_all_ips= null&port_scan_timeout= null&initial_scan_level= null&scan_throttle= null&interface_inventory_timeout=600000&max_interface_inventory_count=10000&bypass_interface_inventory=0&hostnames=&device_groups=&scan_ports=21'
; Creating the object
$oHTTP = ObjCreate("winhttp.winhttprequest.5.1")
$oHTTP.Open("POST", "http://10.2.4.18/api/discovery_session", False)
$oHTTP.SetCredentials("username","password",0)
$oHTTP.SetRequestHeader("Content-Type", "application/em7-resource-uri")
; Performing the Request
$oHTTP.Send($sPD)
; Download the body response if any, and get the server status response code.
$oReceived = $oHTTP.ResponseText
$oStatusCode = $oHTTP.Status
If $oStatusCode <> 200 then
MsgBox(4096, "Response code", $oStatusCode)
EndIf
; Saves the body response regardless of the Response code
$file = FileOpen("Received.html", 2) ; The value of 2 overwrites the file if it already exists
FileWrite($file, $oReceived)
FileClose($file)
Sample VBA code for reference
Dim Items As New Collection
Dim Item As Dictionary
Dim Id As Long
For Id = 1 To 2
Set Item = New Dictionary
Item("iditem") = Id
Item("amount") = 1
Items.Add Item
Next Id
Request.AddBodyParameter "id", 5633
Request.AddBodyParameter "items", Items
$oDictionary = ObjCreate("Scripting.Dictionary")
$oDictionary.ADD("start_ip", "10.20.7.31")
$oDictionary.ADD("end_ip", "10.20.7.33")