converting R dataframes to json object - json

Say I have the following dataframes:
df1 <- data.frame(Name = c("Harry","George"), color=c("#EA0001", "#EEEEEE"))
Name color
1 Harry #EA0001
2 George #EEEEEE
df.details <- data.frame(Name = c(rep("Harry",each=3), rep("George", each=3)),
age=21:23,
total=c(14,19,24,1,9,4)
)
Name age total
1 Harry 21 14
2 Harry 22 19
3 Harry 23 24
4 George 21 1
5 George 22 9
6 George 23 4
I know how to convert each df to json like this:
library(jsonlite)
toJSON(df.details)
[{"Name":"Harry","age":21,"total":14},{"Name":"Harry","age":22,"total":19},{"Name":"Harry","age":23,"total":24},{"Name":"George","age":21,"total":1},{"Name":"George","age":22,"total":9},{"Name":"George","age":23,"total":4}]
However, I am looking to get the following structure to my JSON data:
{
"myjsondata": [
{
"Name": "Harry",
"color": "#EA0001",
"details": [
{
"age": 21,
"total": 14
},
{
"age": 22,
"total": 19
},
{
"age": 23,
"total": 24
}
]
},
{
"Name": "George",
"color": "#EEEEEE",
"details": [
{
"age": 21,
"total": 1
},
{
"age": 22,
"total": 9
},
{
"age": 23,
"total": 4
}
]
}
]
}
I think the answer may be in how I store the data in a list in R before converting, but not sure.

Try this format:
df1$details <- split(df.details[-1], df.details$Name)[df1$Name]
df1
# Name color details
#1 Harry #EA0001 21, 22, 23, 14, 19, 24
#2 George #EEEEEE 21, 22, 23, 1, 9, 4
toJSON(df1)
#[{
#"Name":"Harry",
#"color":"#EA0001",
#"details":[
# {"age":21,"total":14},
# {"age":22,"total":19},
# {"age":23,"total":24}]},
#{
#"Name":"George",
#"color":"#EEEEEE",
#"details":[
# {"age":21,"total":1},
# {"age":22,"total":9},
# {"age":23,"total":4}]}
#]

Related

Adding more data in array of object in PostgreSQL

I have a table of cart with 2 columns (user_num, data).
user_num will have the phone number of user and
data will have an array of object like [{ "id": 1, "quantity": 1 }, { "id": 2, "quantity": 2 }, { "id": 3, "quantity": 3 }] here id is product id.
user_num | data
----------+--------------------------------------------------------------------------------------
1 | [{ "id": 1, "quantity": 1 }, { "id": 2, "quantity": 2 }, { "id": 3, "quantity": 3 }]
I want to add more data of products in above array of objects in PostgreSQL.
Thanks!
To add the value use the JSONB array append operator ||
Demo
update
test
set
data = data || '[{"id": 4, "quantity": 4}, {"id": 5, "quantity": 5}]'
where
user_num = 1;

jq - parse structure and save values in bash variable

I have a json input as follow
{
"unique": 1924,
"coordinates": [
{
"time": "2015-01-25T00:00:01.683",
"xyz": [
{
"z": 4,
"y": 2,
"x": 1,
"id": 99,
"inner_arr" : [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
}
]
},
{
"z": 9,
"y": 9,
"x": 8,
"id": 100,
"inner_arr" : [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
}
]
},
{
"z": 9,
"y": 6,
"x": 10,
"id": 101,
"inner_arr" : [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
}
]
}
]
},
{
"time": "2015-01-25T00:00:02.790",
"xyz": [
{
"z": 0,
"y": 3,
"x": 7,
"id": 99,
"inner_arr" : [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
}
]
},
{
"z": 4,
"y": 6,
"x": 2,
"id": 100,
"inner_arr" : [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
}
]
},
{
"z": 2,
"y": 9,
"x": 51,
"id": 101,
"inner_arr" : [
{
"a": 1,
"b": 2
},
{
"a": 3,
"b": 4
}
]
}
]
}
]
}
I want to parse this input with jq and store values in bash arrays:
#!/bin/bash
z=()
x=()
y=()
id=()
a=()
b=()
jq --raw-output '.coordinates[] | .xyz[] | (.z) as $z, (.y) as $y,7 (.x) as $x, (.id) as $id, .inner_arr[].a $a, .inner_arr[].b as $b | $z, $y, $x, $id, $a, $b' <<< "$input"
echo -e "${z}"
Expected output for above echo command:
4
9
9
0
4
2
echo -e "${a}"
Expected output for above echo command:
1
3
1
3
1
3
1
3
1
3
1
3
How can I do it with jq with a single jq call looping through all arrays in a cascading fashion?
I want to save CPU by calling jq just once and extract all single or array values.
You cannot set environment variable directly from jq (cf. manual). What you can do is to generate a series of bash declarations for the declare builtin. I suggest to store the declarations in an intermediate bash array (with mapfile) processed directly by declare so that you can stay away from hazardous commands like eval.
mapfile -t < <(
jq --raw-output '
def m(exp): first(.[0] | path(exp)[-1]) + "=(" + (map(exp) | #sh) + ")";
[ .coordinates[].xyz[] ]
| m(.x), m(.y), m(.z), m(.id), m(.inner_arr[].a), m(.inner_arr[].b)
' input
)
declare -a "${MAPFILE[#]}"
The jq script packs all xyz objects in a single array and filters it with the m function for each field represented as a path expression. The function returns a string formatted as field=(val1 val2... valN), where the field name is the last component of the path expression, i.e. x for .x and a for .inner_arr[].a (extracted on the first item of the array).
Then you can check the shell variables with declare -p var or ${var[#]}. ${var} refers to the first element only.
declare -p MAPFILE
declare -p z
echo a: "${a[#]}" / size = ${#a[#]}
declare -a MAPFILE=([0]="x=(1 8 10 7 2 51)" [1]="y=(2 9 6 3 6 9)" [2]="z=(4 9 9 0 4 2)" [3]="id=(99 100 101 99 100 101)" [4]="a=(1 3 1 3 1 3 1 3 1 3 1 3)" [5]="b=(2 4 2 4 2 4 2 4 2 4 2 4)")
declare -a z=([0]="4" [1]="9" [2]="9" [3]="0" [4]="4" [5]="2")
a: 1 3 1 3 1 3 1 3 1 3 1 3 / size = 12

How to convert JSON column to list in R

I have data set like this
genres
[{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 14, "name": "Fantasy"}, {"id": 878, "name": "Science Fiction"}]
[{"id": 12, "name": "Adventure"}, {"id": 14, "name": "Fantasy"}, {"id": 28, "name": "Action"}]
[{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 80, "name": "Crime"}]
[{"id": 28, "name": "Action"}, {"id": 80, "name": "Crime"}, {"id": 18, "name": "Drama"}, {"id": 53, "name": "Thriller"}]
[{"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}, {"id": 878, "name": "Science Fiction"}]
[{"id": 14, "name": "Fantasy"}, {"id": 28, "name": "Action"}, {"id": 12, "name": "Adventure"}]
[{"id": 16, "name": "Animation"}, {"id": 10751, "name": "Family"}]
Now I want to populate a drop down list of Shiny application so I want to convert the column JSON to data table, I use apply functions but does not get desired result, can someone please help me out.
Code:
lapply(dt, fromJSON(dt$genres))
I am new to R so it may be not best solution but here my finding which is working in my case, please let me know if any feedback
genre <- raw$genres
genreList <- lapply(genre,function(x) fromJSON(x))
genreList <- genreList[sapply(genreList, function(x) as.numeric(dim(x)[1])) > 0]
genreList <- genreList[!sapply(genreList, is.null)]
finalGenre <- unique(Reduce(function(...) merge(..., all=T), genreList))
And the output
id name
1 28 Action
2 53 Thriller
3 10769 Foreign
4 12 Adventure
5 10751 Family
6 878 Science Fiction
7 27 Horror
8 16 Animation
9 80 Crime
10 35 Comedy
11 10770 TV Movie
12 18 Drama
13 99 Documentary
14 10752 War
15 10402 Music
16 10749 Romance
17 14 Fantasy
18 37 Western
19 36 History
20 9648 Mystery

Mysql / Eloquent / Laravel aggregate data returned by query in usable form for view

I have this situation:
$x = user->waist_id;
$y = user->pants_size_id;
$sizes = DB::select(DB::raw("SELECT model_id,
pants_sizes.id AS size_id,
Count(pants_sizes.id) AS Total FROM `pants_sizes`
INNER JOIN `owned_items`
ON `pants_sizes`.`id` = `owned_items`.`model_size_id`
INNER JOIN `owned_item_user`
ON `owned_items`.`id` = `owned_item_user`.`owned_item_id`
INNER JOIN `users`
ON `users`.`id` = `owned_item_user`.`user_id` WHERE `waist_id` = $x
AND `pants_size_id` = $y
AND `owned_items`.`model_id` IN ( 218, 219 ) GROUP BY `model_id`,
`pants_sizes`.`id` "))
return $sizes;
with this query I am joining some tables to get the sizes of a specific pants model owned by users based on same attributes (example waist and pants_size).
this returns the data in this form:
[
{
"model_id": 218,
"size_id": 4,
"Total": 3
},
{
"model_id": 218,
"size_id": 5,
"Total": 7
},
{
"model_id": 219,
"size_id": 4,
"Total": 3
},
{
"model_id": 219,
"size_id": 5,
"Total": 7
}
]
what I need is the data to be returned like this instead:
[
{
"model_id": 218,
"size_id": 4,
"Total_size_id_4": 3
"size_id": 5,
"Total_size_id_5": 7
"Total": 10
},
"model_id": 219,
"size_id": 4,
"Total_size_id_4": 3
"size_id": 5,
"Total_size_id_5": 7
"Total": 10
},
]
or in a way that I can use to produce statistics in the view, example:
[
{
"model_id": 218,
"4": "30%",
"5": "70%",
"Total": 10
},
"model_id": 219,
"4": "30%",
"5": "70%",
"Total": 10
},
]
"model 218 is owned by 10 users, 30% has size 4 and 70% has size 5"
Can this be done modifying the above query? with some group concat and sum stuff maybe?

Constructing request payload in R using rjson/jsonlite

My current code as seen below attempts to construct a request payload (body), but isn't giving me the desired result.
library(df2json)
library(rjson)
y = rjson::fromJSON((df2json::df2json(dataframe)))
globalparam = ""
req = list(
Inputs = list(
input1 = y
)
,GlobalParameters = paste("{",globalparam,"}",sep="")#globalparam
)
body = enc2utf8((rjson::toJSON(req)))
body currently turns out to be
{
"Inputs": {
"input1": [
{
"X": 7,
"Y": 5,
"month": "mar",
"day": "fri",
"FFMC": 86.2,
"DMC": 26.2,
"DC": 94.3,
"ISI": 5.1,
"temp": 8.2,
"RH": 51,
"wind": 6.7,
"rain": 0,
"area": 0
}
]
},
"GlobalParameters": "{}"
}
However, I need it to look like this:
{
"Inputs": {
"input1": [
{
"X": 7,
"Y": 5,
"month": "mar",
"day": "fri",
"FFMC": 86.2,
"DMC": 26.2,
"DC": 94.3,
"ISI": 5.1,
"temp": 8.2,
"RH": 51,
"wind": 6.7,
"rain": 0,
"area": 0
}
]
},
"GlobalParameters": {}
}
So basically global parameters have to be {}, but not hardcoded. It seemed like a fairly simple problem, but I couldn't fix it. Please help!
EDIT:
This is the dataframe
X Y month day FFMC DMC DC ISI temp RH wind rain area
1 7 5 mar fri 86.2 26.2 94.3 5.1 8.2 51 6.7 0.0 0
2 7 4 oct tue 90.6 35.4 669.1 6.7 18.0 33 0.9 0.0 0
3 7 4 oct sat 90.6 43.7 686.9 6.7 14.6 33 1.3 0.0 0
4 8 6 mar fri 91.7 33.3 77.5 9.0 8.3 97 4.0 0.2 0
This is an example of another data frame
> a = data.frame("col1" = c(81, 81, 81, 81), "col2" = c(72, 69, 79, 84))
Using this sample data
dd<-read.table(text=" X Y month day FFMC DMC DC ISI temp RH wind rain area
1 7 5 mar fri 86.2 26.2 94.3 5.1 8.2 51 6.7 0.0 0", header=T)
You can do
globalparam = setNames(list(), character(0))
req = list(
Inputs = list(
input1 = dd
)
,GlobalParameters = globalparam
)
body = enc2utf8((rjson::toJSON(req)))
Note that globalparam looks a bit funny because we need to force it to a named list for rjson to treat it properly. We only have to do this when it's empty.